Hello everyone,
V8 for basic support of the Video Input Processor found on the Rockchip PX30 SoC
The v6 is based on the fifth iteration of the series introducing the
driver: sent 29 Dec 2020 [1]
Most of this driver was written following the BSP driver from rockchip,
removing the parts that either didn't fit correctly the guidelines, or
that couldn't be tested.
In the BSP, this driver is known as the "cif" driver, but this was
renamed to "vip" to better fit the controller denomination in the
datasheet.
This version of the driver supports ONLY the parallel interface BT656
and was tested/implemented using an SDTV video decoder
media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
V7 => V8:
vip/capture.c:
- fixed a warning: unused variable reported by the kernel test robot
V6 => V7:
vip/capture.c vip/dev.c vip/dev.h
- renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
- added some error when rk_vip_get_buffer() returns NULL
- removed a WARN_ON
- made the irq NOT shared
- dropped of_match_ptr
- added the rk_vip_get_resource() function
rockchip,px30-vip.yaml:
- changed filename to match the compatible
- dropped the mention of the other rockchip SoC in the dt-binding
description and added a more detailed description of VIP
- removed unused labels in the example
V5 [1] => V6:
vip/capture.c vip/dev.c vip/dev.h
- added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
supported stds in rk_vip_enum_input callback
- added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
- added the supported video_device->tvnorms
- s_std will now update the format as this depends on the standard
NTSC/PAL (as suggested by Hans in [1])
- removed STD_ATSC
- moved the colorimetry information to come from the subdev
- removed the core s_power subdev calls
- dropped cropping in rk_vip_stream struct
rockchip-vip.yaml:
- fixed a mistake in the name of third clock plckin -> plck
- changed the reg maxItems 2 -> 1
[1] https://lore.kernel.org/linux-media/[email protected]/
I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
# v4l2-compliance
v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
Compliance test for rk_vip device /dev/video0:
Driver Info:
Driver name : rk_vip
Card type : rk_vip
Bus info : platform:ff490000.vip
Driver version : 6.6.0
Capabilities : 0x84201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : rk_vip
Model : rk_vip
Serial :
Bus info : platform:ff490000.vip
Media version : 6.6.0
Hardware revision: 0x00000000 (0)
Driver version : 6.6.0
Interface Info:
ID : 0x03000002
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : video_rkvip
Function : V4L2 I/O
Pad 0x01000004 : 0: Sink
Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
Mehdi Djait (3):
media: dt-bindings: media: add bindings for Rockchip VIP
media: rockchip: Add a driver for Rockhip's camera interface
arm64: dts: rockchip: Add the camera interface
.../bindings/media/rockchip,px30-vip.yaml | 93 ++
arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
drivers/media/platform/rockchip/vip/Kconfig | 14 +
drivers/media/platform/rockchip/vip/Makefile | 3 +
drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
drivers/media/platform/rockchip/vip/dev.c | 346 +++++
drivers/media/platform/rockchip/vip/dev.h | 163 +++
drivers/media/platform/rockchip/vip/regs.h | 260 ++++
10 files changed, 2103 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
create mode 100644 drivers/media/platform/rockchip/vip/Makefile
create mode 100644 drivers/media/platform/rockchip/vip/capture.c
create mode 100644 drivers/media/platform/rockchip/vip/dev.c
create mode 100644 drivers/media/platform/rockchip/vip/dev.h
create mode 100644 drivers/media/platform/rockchip/vip/regs.h
--
2.41.0
The PX30 has a camera interface, supporting CSI2 and BT656
modes. Add a DT description for this interface.
Signed-off-by: Mehdi Djait <[email protected]>
---
arch/arm64/boot/dts/rockchip/px30.dtsi | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/px30.dtsi b/arch/arm64/boot/dts/rockchip/px30.dtsi
index 42ce78beb413..7aaa88a15d07 100644
--- a/arch/arm64/boot/dts/rockchip/px30.dtsi
+++ b/arch/arm64/boot/dts/rockchip/px30.dtsi
@@ -1281,6 +1281,18 @@ isp_mmu: iommu@ff4a8000 {
#iommu-cells = <0>;
};
+ vip: vip@ff490000 {
+ compatible = "rockchip,px30-vip";
+ reg = <0x0 0xff490000 0x0 0x200>;
+ interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru ACLK_CIF>, <&cru HCLK_CIF>, <&cru PCLK_CIF>;
+ clock-names = "aclk", "hclk", "pclk";
+ power-domains = <&power PX30_PD_VI>;
+ resets = <&cru SRST_CIF_A>, <&cru SRST_CIF_H>, <&cru SRST_CIF_PCLKIN>;
+ reset-names = "axi", "ahb", "pclkin";
+ status = "disabled";
+ };
+
qos_gmac: qos@ff518000 {
compatible = "rockchip,px30-qos", "syscon";
reg = <0x0 0xff518000 0x0 0x20>;
--
2.41.0
Hi Mehdi,
On Mon 16 Oct 23, 11:00, Mehdi Djait wrote:
> In the BSP, this driver is known as the "cif" driver, but this was
> renamed to "vip" to better fit the controller denomination in the
> datasheet.
So I have spent a bit of time trying to figure out the history of this unit
and in which platform in was used. The takeaway is that the earliest Rockchip
SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568
(2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is
a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit
(mixed with VICAP_MIPI) which also seems significantly different.
Over the course of the existence of this unit, it is most often referred to
as CIF. Since this is also the name for the driver in the Rockchip tree,
I feel like it is best to use CIF as the mainline terminology instead of VIP.
Note that the unit is also called VICAP in RK3566/RK3568.
Here is the detail of my research on the concerned chips. The + at the beginning
of the line indicate support in Rockchip's 4.4 tree:
- RK3566/RK3568 (2020): CIF pins + VICAP terminology
+ RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
+ PX30 (2017): VIP pins + VIP registers
+ RK3328 (2017): CIF pins + VIP terminology
- RK3326 (2017): CIF pins + VIP terminology
- RK3399 (2016): CIF pins
- RK3368 (2015): CIF pins
- PX2 (2014-11): CIF pins + CIF registers
+ RK3126/RK3128 (2014-10): CIF pins + registers
+ RK3288 (2014-05): CIF pins + VIP terminology
- RK3026 (2013): CIF pins + CIF registers
- RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
- RK3066 (2012): CIF pins + CIF registers
Note that there are a few variations over time (added/removed registers), but
the offsets of crucial registers are always the same, so we can safely
assume this is the same unit in different generations.
Since the RK3066 is the first model starting the RK30 lineup I think we can
safely use that for the "base" compatible to be used for e.g. the bindings
document, instead of px30 which is just one of the many SoCs that use this unit.
Cheers,
Paul
> This version of the driver supports ONLY the parallel interface BT656
> and was tested/implemented using an SDTV video decoder
>
> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
>
> V7 => V8:
> vip/capture.c:
> - fixed a warning: unused variable reported by the kernel test robot
>
> V6 => V7:
> vip/capture.c vip/dev.c vip/dev.h
> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
> - added some error when rk_vip_get_buffer() returns NULL
> - removed a WARN_ON
> - made the irq NOT shared
> - dropped of_match_ptr
> - added the rk_vip_get_resource() function
>
> rockchip,px30-vip.yaml:
> - changed filename to match the compatible
> - dropped the mention of the other rockchip SoC in the dt-binding
> description and added a more detailed description of VIP
> - removed unused labels in the example
>
>
> V5 [1] => V6:
> vip/capture.c vip/dev.c vip/dev.h
> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
> supported stds in rk_vip_enum_input callback
> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
> - added the supported video_device->tvnorms
> - s_std will now update the format as this depends on the standard
> NTSC/PAL (as suggested by Hans in [1])
> - removed STD_ATSC
> - moved the colorimetry information to come from the subdev
> - removed the core s_power subdev calls
> - dropped cropping in rk_vip_stream struct
>
> rockchip-vip.yaml:
> - fixed a mistake in the name of third clock plckin -> plck
> - changed the reg maxItems 2 -> 1
>
> [1] https://lore.kernel.org/linux-media/[email protected]/
>
> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
>
> # v4l2-compliance
> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
>
> Compliance test for rk_vip device /dev/video0:
>
> Driver Info:
> Driver name : rk_vip
> Card type : rk_vip
> Bus info : platform:ff490000.vip
> Driver version : 6.6.0
> Capabilities : 0x84201000
> Video Capture Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04201000
> Video Capture Multiplanar
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : rk_vip
> Model : rk_vip
> Serial :
> Bus info : platform:ff490000.vip
> Media version : 6.6.0
> Hardware revision: 0x00000000 (0)
> Driver version : 6.6.0
> Interface Info:
> ID : 0x03000002
> Type : V4L Video
> Entity Info:
> ID : 0x00000001 (1)
> Name : video_rkvip
> Function : V4L2 I/O
> Pad 0x01000004 : 0: Sink
> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
> test invalid ioctls: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 1 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Input 0):
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls (Input 0):
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls (Input 0):
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Input 0):
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
>
> Mehdi Djait (3):
> media: dt-bindings: media: add bindings for Rockchip VIP
> media: rockchip: Add a driver for Rockhip's camera interface
> arm64: dts: rockchip: Add the camera interface
>
> .../bindings/media/rockchip,px30-vip.yaml | 93 ++
> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
> drivers/media/platform/rockchip/Kconfig | 1 +
> drivers/media/platform/rockchip/Makefile | 1 +
> drivers/media/platform/rockchip/vip/Kconfig | 14 +
> drivers/media/platform/rockchip/vip/Makefile | 3 +
> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
> drivers/media/platform/rockchip/vip/dev.c | 346 +++++
> drivers/media/platform/rockchip/vip/dev.h | 163 +++
> drivers/media/platform/rockchip/vip/regs.h | 260 ++++
> 10 files changed, 2103 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
> create mode 100644 drivers/media/platform/rockchip/vip/Makefile
> create mode 100644 drivers/media/platform/rockchip/vip/capture.c
> create mode 100644 drivers/media/platform/rockchip/vip/dev.c
> create mode 100644 drivers/media/platform/rockchip/vip/dev.h
> create mode 100644 drivers/media/platform/rockchip/vip/regs.h
>
> --
> 2.41.0
>
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Mehdi,
On Mon 16 Oct 23, 11:00, Mehdi Djait wrote:
> The PX30 has a camera interface, supporting CSI2 and BT656
> modes. Add a DT description for this interface.
The "vip" node name is not very standard but the generic names recommendation
doesn't have anything that really fits video capture:
https://devicetree-specification.readthedocs.io/en/v0.3/devicetree-basics.html#generic-names-recommendation
You might want to call it "video-capture" or "camera-controller" which seems
like a good middle-ground that could be included in the list.
Please keep the name (vip/cif) as label though.
Other than that, please rename vip to cif and this will be:
Reviewed-by: Paul Kocialkowski <[email protected]>
Cheers,
Paul
> Signed-off-by: Mehdi Djait <[email protected]>
> ---
> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/rockchip/px30.dtsi b/arch/arm64/boot/dts/rockchip/px30.dtsi
> index 42ce78beb413..7aaa88a15d07 100644
> --- a/arch/arm64/boot/dts/rockchip/px30.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/px30.dtsi
> @@ -1281,6 +1281,18 @@ isp_mmu: iommu@ff4a8000 {
> #iommu-cells = <0>;
> };
>
> + vip: vip@ff490000 {
> + compatible = "rockchip,px30-vip";
> + reg = <0x0 0xff490000 0x0 0x200>;
> + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&cru ACLK_CIF>, <&cru HCLK_CIF>, <&cru PCLK_CIF>;
> + clock-names = "aclk", "hclk", "pclk";
> + power-domains = <&power PX30_PD_VI>;
> + resets = <&cru SRST_CIF_A>, <&cru SRST_CIF_H>, <&cru SRST_CIF_PCLKIN>;
> + reset-names = "axi", "ahb", "pclkin";
> + status = "disabled";
> + };
> +
> qos_gmac: qos@ff518000 {
> compatible = "rockchip,px30-qos", "syscon";
> reg = <0x0 0xff518000 0x0 0x20>;
> --
> 2.41.0
>
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Mehdi and Paul,
Let me begin by saying how great it is to see this series moving
forward. After nearly three years of silence I was not expecting that
:-) Thanks for your efforts!
On 10/19/23 17:33, Paul Kocialkowski wrote:
> Hi Mehdi,
>
> On Mon 16 Oct 23, 11:00, Mehdi Djait wrote:
>> In the BSP, this driver is known as the "cif" driver, but this was
>> renamed to "vip" to better fit the controller denomination in the
>> datasheet.
>
> So I have spent a bit of time trying to figure out the history of this unit
> and in which platform in was used. The takeaway is that the earliest Rockchip
> SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568
> (2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is
> a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit
> (mixed with VICAP_MIPI) which also seems significantly different.
The RK3588 VICAP may be significantly more complex, but it is supported
by the same driver in the downstream kernel [0]. The DVP part (which is
in the scope of this series) should be more or less the same deal
(although Rockchip mixed the register positions once again, the
registers itself are similar to e.g., RK356X when it comes to DVP).
> Over the course of the existence of this unit, it is most often referred to
> as CIF. Since this is also the name for the driver in the Rockchip tree,
> I feel like it is best to use CIF as the mainline terminology instead of VIP.
> Note that the unit is also called VICAP in RK3566/RK3568.
But even if we take the RK3588 VICAP unit into account, I'd agree that
CIF seems to be the better name (precisly because the downstream driver
is called "cif").
The individual compatibles can be named "rockchip,px30-vip",
"rockchip,rk3568-vicap", ..., right? This would be in contrast to the
downstream driver (which uses "-cif" for all units) but here the
alignment with the respective data sheet comes into play (which will be
helpful).
> Here is the detail of my research on the concerned chips. The + at the beginning
> of the line indicate support in Rockchip's 4.4 tree:
>
> - RK3566/RK3568 (2020): CIF pins + VICAP terminology
> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
> + PX30 (2017): VIP pins + VIP registers
> + RK3328 (2017): CIF pins + VIP terminology
> - RK3326 (2017): CIF pins + VIP terminology
> - RK3399 (2016): CIF pins
> - RK3368 (2015): CIF pins
> - PX2 (2014-11): CIF pins + CIF registers
> + RK3126/RK3128 (2014-10): CIF pins + registers
> + RK3288 (2014-05): CIF pins + VIP terminology
> - RK3026 (2013): CIF pins + CIF registers
> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
> - RK3066 (2012): CIF pins + CIF registers
>
> Note that there are a few variations over time (added/removed registers), but
> the offsets of crucial registers are always the same, so we can safely
> assume this is the same unit in different generations.
>
> Since the RK3066 is the first model starting the RK30 lineup I think we can
> safely use that for the "base" compatible to be used for e.g. the bindings
> document, instead of px30 which is just one of the many SoCs that use this unit.
Once the name of the driver is defined and adjusted in v9, I can try to
give the series a shot on my RK3568 board. First attempts to do so
basing on Maxime's v5 showed that with a few modifications the DVP
feature works fine. In a subsequent step, we could discuss the inclusion
of the MIPI CSI-2 things in order to keep the driver sufficiently general.
@Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
Best regards,
Michael
[0]
https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968
>
>> This version of the driver supports ONLY the parallel interface BT656
>> and was tested/implemented using an SDTV video decoder
>>
>> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
>>
>> V7 => V8:
>> vip/capture.c:
>> - fixed a warning: unused variable reported by the kernel test robot
>>
>> V6 => V7:
>> vip/capture.c vip/dev.c vip/dev.h
>> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
>> - added some error when rk_vip_get_buffer() returns NULL
>> - removed a WARN_ON
>> - made the irq NOT shared
>> - dropped of_match_ptr
>> - added the rk_vip_get_resource() function
>>
>> rockchip,px30-vip.yaml:
>> - changed filename to match the compatible
>> - dropped the mention of the other rockchip SoC in the dt-binding
>> description and added a more detailed description of VIP
>> - removed unused labels in the example
>>
>>
>> V5 [1] => V6:
>> vip/capture.c vip/dev.c vip/dev.h
>> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
>> supported stds in rk_vip_enum_input callback
>> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
>> - added the supported video_device->tvnorms
>> - s_std will now update the format as this depends on the standard
>> NTSC/PAL (as suggested by Hans in [1])
>> - removed STD_ATSC
>> - moved the colorimetry information to come from the subdev
>> - removed the core s_power subdev calls
>> - dropped cropping in rk_vip_stream struct
>>
>> rockchip-vip.yaml:
>> - fixed a mistake in the name of third clock plckin -> plck
>> - changed the reg maxItems 2 -> 1
>>
>> [1] https://lore.kernel.org/linux-media/[email protected]/
>>
>> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
>>
>> # v4l2-compliance
>> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
>>
>> Compliance test for rk_vip device /dev/video0:
>>
>> Driver Info:
>> Driver name : rk_vip
>> Card type : rk_vip
>> Bus info : platform:ff490000.vip
>> Driver version : 6.6.0
>> Capabilities : 0x84201000
>> Video Capture Multiplanar
>> Streaming
>> Extended Pix Format
>> Device Capabilities
>> Device Caps : 0x04201000
>> Video Capture Multiplanar
>> Streaming
>> Extended Pix Format
>> Media Driver Info:
>> Driver name : rk_vip
>> Model : rk_vip
>> Serial :
>> Bus info : platform:ff490000.vip
>> Media version : 6.6.0
>> Hardware revision: 0x00000000 (0)
>> Driver version : 6.6.0
>> Interface Info:
>> ID : 0x03000002
>> Type : V4L Video
>> Entity Info:
>> ID : 0x00000001 (1)
>> Name : video_rkvip
>> Function : V4L2 I/O
>> Pad 0x01000004 : 0: Sink
>> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
>>
>> Required ioctls:
>> test MC information (see 'Media Driver Info' above): OK
>> test VIDIOC_QUERYCAP: OK
>> test invalid ioctls: OK
>>
>> Allow for multiple opens:
>> test second /dev/video0 open: OK
>> test VIDIOC_QUERYCAP: OK
>> test VIDIOC_G/S_PRIORITY: OK
>> test for unlimited opens: OK
>>
>> Debug ioctls:
>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>
>> Input ioctls:
>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>> test VIDIOC_G/S/ENUMINPUT: OK
>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>> Inputs: 1 Audio Inputs: 0 Tuners: 0
>>
>> Output ioctls:
>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>
>> Input/Output configuration ioctls:
>> test VIDIOC_ENUM/G/S/QUERY_STD: OK
>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>
>> Control ioctls (Input 0):
>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>> Standard Controls: 0 Private Controls: 0
>>
>> Format ioctls (Input 0):
>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>> test VIDIOC_G/S_PARM: OK (Not Supported)
>> test VIDIOC_G_FBUF: OK (Not Supported)
>> test VIDIOC_G_FMT: OK
>> test VIDIOC_TRY_FMT: OK
>> test VIDIOC_S_FMT: OK
>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>> test Cropping: OK (Not Supported)
>> test Composing: OK (Not Supported)
>> test Scaling: OK (Not Supported)
>>
>> Codec ioctls (Input 0):
>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>
>> Buffer ioctls (Input 0):
>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>> test VIDIOC_EXPBUF: OK
>> test Requests: OK (Not Supported)
>>
>> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
>>
>> Mehdi Djait (3):
>> media: dt-bindings: media: add bindings for Rockchip VIP
>> media: rockchip: Add a driver for Rockhip's camera interface
>> arm64: dts: rockchip: Add the camera interface
>>
>> .../bindings/media/rockchip,px30-vip.yaml | 93 ++
>> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
>> drivers/media/platform/rockchip/Kconfig | 1 +
>> drivers/media/platform/rockchip/Makefile | 1 +
>> drivers/media/platform/rockchip/vip/Kconfig | 14 +
>> drivers/media/platform/rockchip/vip/Makefile | 3 +
>> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
>> drivers/media/platform/rockchip/vip/dev.c | 346 +++++
>> drivers/media/platform/rockchip/vip/dev.h | 163 +++
>> drivers/media/platform/rockchip/vip/regs.h | 260 ++++
>> 10 files changed, 2103 insertions(+)
>> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
>> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
>> create mode 100644 drivers/media/platform/rockchip/vip/Makefile
>> create mode 100644 drivers/media/platform/rockchip/vip/capture.c
>> create mode 100644 drivers/media/platform/rockchip/vip/dev.c
>> create mode 100644 drivers/media/platform/rockchip/vip/dev.h
>> create mode 100644 drivers/media/platform/rockchip/vip/regs.h
>>
>> --
>> 2.41.0
>>
>
Hi Michael,
On Mon 23 Oct 23, 15:07, Michael Riesch wrote:
> > So I have spent a bit of time trying to figure out the history of this unit
> > and in which platform in was used. The takeaway is that the earliest Rockchip
> > SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568
> > (2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is
> > a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit
> > (mixed with VICAP_MIPI) which also seems significantly different.
>
> The RK3588 VICAP may be significantly more complex, but it is supported
> by the same driver in the downstream kernel [0]. The DVP part (which is
> in the scope of this series) should be more or less the same deal
> (although Rockchip mixed the register positions once again, the
> registers itself are similar to e.g., RK356X when it comes to DVP).
After a closer look I tend to agree with you: they moved some registers around
but it still seems to be the same base unit.
> > Over the course of the existence of this unit, it is most often referred to
> > as CIF. Since this is also the name for the driver in the Rockchip tree,
> > I feel like it is best to use CIF as the mainline terminology instead of VIP.
> > Note that the unit is also called VICAP in RK3566/RK3568.
>
> But even if we take the RK3588 VICAP unit into account, I'd agree that
> CIF seems to be the better name (precisly because the downstream driver
> is called "cif").
>
> The individual compatibles can be named "rockchip,px30-vip",
> "rockchip,rk3568-vicap", ..., right? This would be in contrast to the
> downstream driver (which uses "-cif" for all units) but here the
> alignment with the respective data sheet comes into play (which will be
> helpful).
Yeah I think it would make sense to keep the SoC-specific terminology in the
compatible, so I agree that "rockchip,px30-vip" and "rockchip,rk3568-vicap"
feel like the most relevant choices here.
> > Here is the detail of my research on the concerned chips. The + at the beginning
> > of the line indicate support in Rockchip's 4.4 tree:
> >
> > - RK3566/RK3568 (2020): CIF pins + VICAP terminology
> > + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
> > + PX30 (2017): VIP pins + VIP registers
> > + RK3328 (2017): CIF pins + VIP terminology
> > - RK3326 (2017): CIF pins + VIP terminology
> > - RK3399 (2016): CIF pins
> > - RK3368 (2015): CIF pins
> > - PX2 (2014-11): CIF pins + CIF registers
> > + RK3126/RK3128 (2014-10): CIF pins + registers
> > + RK3288 (2014-05): CIF pins + VIP terminology
> > - RK3026 (2013): CIF pins + CIF registers
> > - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
> > - RK3066 (2012): CIF pins + CIF registers
> >
> > Note that there are a few variations over time (added/removed registers), but
> > the offsets of crucial registers are always the same, so we can safely
> > assume this is the same unit in different generations.
> >
> > Since the RK3066 is the first model starting the RK30 lineup I think we can
> > safely use that for the "base" compatible to be used for e.g. the bindings
> > document, instead of px30 which is just one of the many SoCs that use this unit.
>
> Once the name of the driver is defined and adjusted in v9, I can try to
> give the series a shot on my RK3568 board. First attempts to do so
> basing on Maxime's v5 showed that with a few modifications the DVP
> feature works fine. In a subsequent step, we could discuss the inclusion
> of the MIPI CSI-2 things in order to keep the driver sufficiently general.
Nice! I guess there will be a need to introduce a variant structure associated
to each compatible to express the differences betweens these different
generations.
Note that we will also probably need to convert the driver over to a MC-centric
approach, but this is of course outside of the scope of this series.
Cheers,
Paul
> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
>
> Best regards,
> Michael
>
> [0]
> https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968
>
> >
> >> This version of the driver supports ONLY the parallel interface BT656
> >> and was tested/implemented using an SDTV video decoder
> >>
> >> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
> >>
> >> V7 => V8:
> >> vip/capture.c:
> >> - fixed a warning: unused variable reported by the kernel test robot
> >>
> >> V6 => V7:
> >> vip/capture.c vip/dev.c vip/dev.h
> >> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
> >> - added some error when rk_vip_get_buffer() returns NULL
> >> - removed a WARN_ON
> >> - made the irq NOT shared
> >> - dropped of_match_ptr
> >> - added the rk_vip_get_resource() function
> >>
> >> rockchip,px30-vip.yaml:
> >> - changed filename to match the compatible
> >> - dropped the mention of the other rockchip SoC in the dt-binding
> >> description and added a more detailed description of VIP
> >> - removed unused labels in the example
> >>
> >>
> >> V5 [1] => V6:
> >> vip/capture.c vip/dev.c vip/dev.h
> >> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
> >> supported stds in rk_vip_enum_input callback
> >> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
> >> - added the supported video_device->tvnorms
> >> - s_std will now update the format as this depends on the standard
> >> NTSC/PAL (as suggested by Hans in [1])
> >> - removed STD_ATSC
> >> - moved the colorimetry information to come from the subdev
> >> - removed the core s_power subdev calls
> >> - dropped cropping in rk_vip_stream struct
> >>
> >> rockchip-vip.yaml:
> >> - fixed a mistake in the name of third clock plckin -> plck
> >> - changed the reg maxItems 2 -> 1
> >>
> >> [1] https://lore.kernel.org/linux-media/[email protected]/
> >>
> >> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
> >>
> >> # v4l2-compliance
> >> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
> >>
> >> Compliance test for rk_vip device /dev/video0:
> >>
> >> Driver Info:
> >> Driver name : rk_vip
> >> Card type : rk_vip
> >> Bus info : platform:ff490000.vip
> >> Driver version : 6.6.0
> >> Capabilities : 0x84201000
> >> Video Capture Multiplanar
> >> Streaming
> >> Extended Pix Format
> >> Device Capabilities
> >> Device Caps : 0x04201000
> >> Video Capture Multiplanar
> >> Streaming
> >> Extended Pix Format
> >> Media Driver Info:
> >> Driver name : rk_vip
> >> Model : rk_vip
> >> Serial :
> >> Bus info : platform:ff490000.vip
> >> Media version : 6.6.0
> >> Hardware revision: 0x00000000 (0)
> >> Driver version : 6.6.0
> >> Interface Info:
> >> ID : 0x03000002
> >> Type : V4L Video
> >> Entity Info:
> >> ID : 0x00000001 (1)
> >> Name : video_rkvip
> >> Function : V4L2 I/O
> >> Pad 0x01000004 : 0: Sink
> >> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
> >>
> >> Required ioctls:
> >> test MC information (see 'Media Driver Info' above): OK
> >> test VIDIOC_QUERYCAP: OK
> >> test invalid ioctls: OK
> >>
> >> Allow for multiple opens:
> >> test second /dev/video0 open: OK
> >> test VIDIOC_QUERYCAP: OK
> >> test VIDIOC_G/S_PRIORITY: OK
> >> test for unlimited opens: OK
> >>
> >> Debug ioctls:
> >> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>
> >> Input ioctls:
> >> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >> test VIDIOC_G/S/ENUMINPUT: OK
> >> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >> Inputs: 1 Audio Inputs: 0 Tuners: 0
> >>
> >> Output ioctls:
> >> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>
> >> Input/Output configuration ioctls:
> >> test VIDIOC_ENUM/G/S/QUERY_STD: OK
> >> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>
> >> Control ioctls (Input 0):
> >> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >> Standard Controls: 0 Private Controls: 0
> >>
> >> Format ioctls (Input 0):
> >> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >> test VIDIOC_G/S_PARM: OK (Not Supported)
> >> test VIDIOC_G_FBUF: OK (Not Supported)
> >> test VIDIOC_G_FMT: OK
> >> test VIDIOC_TRY_FMT: OK
> >> test VIDIOC_S_FMT: OK
> >> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >> test Cropping: OK (Not Supported)
> >> test Composing: OK (Not Supported)
> >> test Scaling: OK (Not Supported)
> >>
> >> Codec ioctls (Input 0):
> >> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>
> >> Buffer ioctls (Input 0):
> >> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >> test VIDIOC_EXPBUF: OK
> >> test Requests: OK (Not Supported)
> >>
> >> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
> >>
> >> Mehdi Djait (3):
> >> media: dt-bindings: media: add bindings for Rockchip VIP
> >> media: rockchip: Add a driver for Rockhip's camera interface
> >> arm64: dts: rockchip: Add the camera interface
> >>
> >> .../bindings/media/rockchip,px30-vip.yaml | 93 ++
> >> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
> >> drivers/media/platform/rockchip/Kconfig | 1 +
> >> drivers/media/platform/rockchip/Makefile | 1 +
> >> drivers/media/platform/rockchip/vip/Kconfig | 14 +
> >> drivers/media/platform/rockchip/vip/Makefile | 3 +
> >> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
> >> drivers/media/platform/rockchip/vip/dev.c | 346 +++++
> >> drivers/media/platform/rockchip/vip/dev.h | 163 +++
> >> drivers/media/platform/rockchip/vip/regs.h | 260 ++++
> >> 10 files changed, 2103 insertions(+)
> >> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
> >> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
> >> create mode 100644 drivers/media/platform/rockchip/vip/Makefile
> >> create mode 100644 drivers/media/platform/rockchip/vip/capture.c
> >> create mode 100644 drivers/media/platform/rockchip/vip/dev.c
> >> create mode 100644 drivers/media/platform/rockchip/vip/dev.h
> >> create mode 100644 drivers/media/platform/rockchip/vip/regs.h
> >>
> >> --
> >> 2.41.0
> >>
> >
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul,
On 10/25/23 10:43, Paul Kocialkowski wrote:
> [...]
>>> Here is the detail of my research on the concerned chips. The + at the beginning
>>> of the line indicate support in Rockchip's 4.4 tree:
>>>
>>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology
>>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
>>> + PX30 (2017): VIP pins + VIP registers
>>> + RK3328 (2017): CIF pins + VIP terminology
>>> - RK3326 (2017): CIF pins + VIP terminology
>>> - RK3399 (2016): CIF pins
>>> - RK3368 (2015): CIF pins
>>> - PX2 (2014-11): CIF pins + CIF registers
>>> + RK3126/RK3128 (2014-10): CIF pins + registers
>>> + RK3288 (2014-05): CIF pins + VIP terminology
>>> - RK3026 (2013): CIF pins + CIF registers
>>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
>>> - RK3066 (2012): CIF pins + CIF registers
>>>
>>> Note that there are a few variations over time (added/removed registers), but
>>> the offsets of crucial registers are always the same, so we can safely
>>> assume this is the same unit in different generations.
>>>
>>> Since the RK3066 is the first model starting the RK30 lineup I think we can
>>> safely use that for the "base" compatible to be used for e.g. the bindings
>>> document, instead of px30 which is just one of the many SoCs that use this unit.
>>
>> Once the name of the driver is defined and adjusted in v9, I can try to
>> give the series a shot on my RK3568 board. First attempts to do so
>> basing on Maxime's v5 showed that with a few modifications the DVP
>> feature works fine. In a subsequent step, we could discuss the inclusion
>> of the MIPI CSI-2 things in order to keep the driver sufficiently general.
>
> Nice! I guess there will be a need to introduce a variant structure associated
> to each compatible to express the differences betweens these different
> generations.
Indeed. If Mehdi and you suggest something, I'd be happy to review.
Otherwise, I'll try to come up with something reasonable. IMHO it would
make sense (as a first step) to have the clocks and the resets in this
structure, as well as a sub-structure that describes the DVP. The latter
consists of registers mainly, but maybe supported input/output formats
and other things should go in there as well. Also, downstream code has a
significant number of
if (some condition including chip_id) A; else B;
things that we should probably get rid of with this variant structure.
As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X
will have one of those, RK3588 will feature even six of them. So we
should add a const array to the variant structure.
> Note that we will also probably need to convert the driver over to a MC-centric
> approach, but this is of course outside of the scope of this series.
That would absolutely make sense. What is missing, though? (I was
wondering that the driver calls media_device_(un)register but no
/dev/mediaX device pops up.)
Best regards,
Michael
>
> Cheers,
>
> Paul
>
>> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
>>
>> Best regards,
>> Michael
>>
>> [0]
>> https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968
>>
>>>
>>>> This version of the driver supports ONLY the parallel interface BT656
>>>> and was tested/implemented using an SDTV video decoder
>>>>
>>>> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
>>>>
>>>> V7 => V8:
>>>> vip/capture.c:
>>>> - fixed a warning: unused variable reported by the kernel test robot
>>>>
>>>> V6 => V7:
>>>> vip/capture.c vip/dev.c vip/dev.h
>>>> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
>>>> - added some error when rk_vip_get_buffer() returns NULL
>>>> - removed a WARN_ON
>>>> - made the irq NOT shared
>>>> - dropped of_match_ptr
>>>> - added the rk_vip_get_resource() function
>>>>
>>>> rockchip,px30-vip.yaml:
>>>> - changed filename to match the compatible
>>>> - dropped the mention of the other rockchip SoC in the dt-binding
>>>> description and added a more detailed description of VIP
>>>> - removed unused labels in the example
>>>>
>>>>
>>>> V5 [1] => V6:
>>>> vip/capture.c vip/dev.c vip/dev.h
>>>> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
>>>> supported stds in rk_vip_enum_input callback
>>>> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
>>>> - added the supported video_device->tvnorms
>>>> - s_std will now update the format as this depends on the standard
>>>> NTSC/PAL (as suggested by Hans in [1])
>>>> - removed STD_ATSC
>>>> - moved the colorimetry information to come from the subdev
>>>> - removed the core s_power subdev calls
>>>> - dropped cropping in rk_vip_stream struct
>>>>
>>>> rockchip-vip.yaml:
>>>> - fixed a mistake in the name of third clock plckin -> plck
>>>> - changed the reg maxItems 2 -> 1
>>>>
>>>> [1] https://lore.kernel.org/linux-media/[email protected]/
>>>>
>>>> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
>>>>
>>>> # v4l2-compliance
>>>> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
>>>>
>>>> Compliance test for rk_vip device /dev/video0:
>>>>
>>>> Driver Info:
>>>> Driver name : rk_vip
>>>> Card type : rk_vip
>>>> Bus info : platform:ff490000.vip
>>>> Driver version : 6.6.0
>>>> Capabilities : 0x84201000
>>>> Video Capture Multiplanar
>>>> Streaming
>>>> Extended Pix Format
>>>> Device Capabilities
>>>> Device Caps : 0x04201000
>>>> Video Capture Multiplanar
>>>> Streaming
>>>> Extended Pix Format
>>>> Media Driver Info:
>>>> Driver name : rk_vip
>>>> Model : rk_vip
>>>> Serial :
>>>> Bus info : platform:ff490000.vip
>>>> Media version : 6.6.0
>>>> Hardware revision: 0x00000000 (0)
>>>> Driver version : 6.6.0
>>>> Interface Info:
>>>> ID : 0x03000002
>>>> Type : V4L Video
>>>> Entity Info:
>>>> ID : 0x00000001 (1)
>>>> Name : video_rkvip
>>>> Function : V4L2 I/O
>>>> Pad 0x01000004 : 0: Sink
>>>> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
>>>>
>>>> Required ioctls:
>>>> test MC information (see 'Media Driver Info' above): OK
>>>> test VIDIOC_QUERYCAP: OK
>>>> test invalid ioctls: OK
>>>>
>>>> Allow for multiple opens:
>>>> test second /dev/video0 open: OK
>>>> test VIDIOC_QUERYCAP: OK
>>>> test VIDIOC_G/S_PRIORITY: OK
>>>> test for unlimited opens: OK
>>>>
>>>> Debug ioctls:
>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>
>>>> Input ioctls:
>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>> test VIDIOC_G/S/ENUMINPUT: OK
>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>> Inputs: 1 Audio Inputs: 0 Tuners: 0
>>>>
>>>> Output ioctls:
>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>
>>>> Input/Output configuration ioctls:
>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK
>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>
>>>> Control ioctls (Input 0):
>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>> Standard Controls: 0 Private Controls: 0
>>>>
>>>> Format ioctls (Input 0):
>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>> test VIDIOC_G_FMT: OK
>>>> test VIDIOC_TRY_FMT: OK
>>>> test VIDIOC_S_FMT: OK
>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>> test Cropping: OK (Not Supported)
>>>> test Composing: OK (Not Supported)
>>>> test Scaling: OK (Not Supported)
>>>>
>>>> Codec ioctls (Input 0):
>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>>
>>>> Buffer ioctls (Input 0):
>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>>> test VIDIOC_EXPBUF: OK
>>>> test Requests: OK (Not Supported)
>>>>
>>>> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
>>>>
>>>> Mehdi Djait (3):
>>>> media: dt-bindings: media: add bindings for Rockchip VIP
>>>> media: rockchip: Add a driver for Rockhip's camera interface
>>>> arm64: dts: rockchip: Add the camera interface
>>>>
>>>> .../bindings/media/rockchip,px30-vip.yaml | 93 ++
>>>> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
>>>> drivers/media/platform/rockchip/Kconfig | 1 +
>>>> drivers/media/platform/rockchip/Makefile | 1 +
>>>> drivers/media/platform/rockchip/vip/Kconfig | 14 +
>>>> drivers/media/platform/rockchip/vip/Makefile | 3 +
>>>> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
>>>> drivers/media/platform/rockchip/vip/dev.c | 346 +++++
>>>> drivers/media/platform/rockchip/vip/dev.h | 163 +++
>>>> drivers/media/platform/rockchip/vip/regs.h | 260 ++++
>>>> 10 files changed, 2103 insertions(+)
>>>> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
>>>> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
>>>> create mode 100644 drivers/media/platform/rockchip/vip/Makefile
>>>> create mode 100644 drivers/media/platform/rockchip/vip/capture.c
>>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.c
>>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.h
>>>> create mode 100644 drivers/media/platform/rockchip/vip/regs.h
>>>>
>>>> --
>>>> 2.41.0
>>>>
>>>
>
Michael,
On Wed 25 Oct 23, 11:17, Michael Riesch wrote:
> Hi Paul,
>
> On 10/25/23 10:43, Paul Kocialkowski wrote:
> > [...]
> >>> Here is the detail of my research on the concerned chips. The + at the beginning
> >>> of the line indicate support in Rockchip's 4.4 tree:
> >>>
> >>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology
> >>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
> >>> + PX30 (2017): VIP pins + VIP registers
> >>> + RK3328 (2017): CIF pins + VIP terminology
> >>> - RK3326 (2017): CIF pins + VIP terminology
> >>> - RK3399 (2016): CIF pins
> >>> - RK3368 (2015): CIF pins
> >>> - PX2 (2014-11): CIF pins + CIF registers
> >>> + RK3126/RK3128 (2014-10): CIF pins + registers
> >>> + RK3288 (2014-05): CIF pins + VIP terminology
> >>> - RK3026 (2013): CIF pins + CIF registers
> >>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
> >>> - RK3066 (2012): CIF pins + CIF registers
> >>>
> >>> Note that there are a few variations over time (added/removed registers), but
> >>> the offsets of crucial registers are always the same, so we can safely
> >>> assume this is the same unit in different generations.
> >>>
> >>> Since the RK3066 is the first model starting the RK30 lineup I think we can
> >>> safely use that for the "base" compatible to be used for e.g. the bindings
> >>> document, instead of px30 which is just one of the many SoCs that use this unit.
> >>
> >> Once the name of the driver is defined and adjusted in v9, I can try to
> >> give the series a shot on my RK3568 board. First attempts to do so
> >> basing on Maxime's v5 showed that with a few modifications the DVP
> >> feature works fine. In a subsequent step, we could discuss the inclusion
> >> of the MIPI CSI-2 things in order to keep the driver sufficiently general.
> >
> > Nice! I guess there will be a need to introduce a variant structure associated
> > to each compatible to express the differences betweens these different
> > generations.
>
> Indeed. If Mehdi and you suggest something, I'd be happy to review.
Well the be honest the scope of work on our side is really centered on PX30
and merging this first version.
> Otherwise, I'll try to come up with something reasonable. IMHO it would
> make sense (as a first step) to have the clocks and the resets in this
> structure, as well as a sub-structure that describes the DVP. The latter
> consists of registers mainly, but maybe supported input/output formats
> and other things should go in there as well. Also, downstream code has a
> significant number of
> if (some condition including chip_id) A; else B;
> things that we should probably get rid of with this variant structure.
Indeed I think we want to try avoid that. Another common option is to define
capability flags to represent differences between generations in a more
practical and clean way than explicitly checking chip ids or so.
> As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X
> will have one of those, RK3588 will feature even six of them. So we
> should add a const array to the variant structure.
>
> > Note that we will also probably need to convert the driver over to a MC-centric
> > approach, but this is of course outside of the scope of this series.
>
> That would absolutely make sense. What is missing, though? (I was
> wondering that the driver calls media_device_(un)register but no
> /dev/mediaX device pops up.)
Switching from video node-centric to MC-centric is more of a semantic change.
In the first case we expect that subdevs are configured by the video device
driver and userspace is not expected to change anything in the media topology
or to configure media entities explicitly.
In the latter case it's the opposite : the driver should never try to push
configuration to a subdev and should instead validate that the current
configuration makes sense.
Still, I believe should be a media device registered and visible to userspace.
Mehdi could you take a look at this? Do you see a media device in `media-ctl -p`
and /dev/mediaX?
Cheers,
Paul
> Best regards,
> Michael
>
> >
> > Cheers,
> >
> > Paul
> >
> >> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
> >>
> >> Best regards,
> >> Michael
> >>
> >> [0]
> >> https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968
> >>
> >>>
> >>>> This version of the driver supports ONLY the parallel interface BT656
> >>>> and was tested/implemented using an SDTV video decoder
> >>>>
> >>>> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
> >>>>
> >>>> V7 => V8:
> >>>> vip/capture.c:
> >>>> - fixed a warning: unused variable reported by the kernel test robot
> >>>>
> >>>> V6 => V7:
> >>>> vip/capture.c vip/dev.c vip/dev.h
> >>>> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
> >>>> - added some error when rk_vip_get_buffer() returns NULL
> >>>> - removed a WARN_ON
> >>>> - made the irq NOT shared
> >>>> - dropped of_match_ptr
> >>>> - added the rk_vip_get_resource() function
> >>>>
> >>>> rockchip,px30-vip.yaml:
> >>>> - changed filename to match the compatible
> >>>> - dropped the mention of the other rockchip SoC in the dt-binding
> >>>> description and added a more detailed description of VIP
> >>>> - removed unused labels in the example
> >>>>
> >>>>
> >>>> V5 [1] => V6:
> >>>> vip/capture.c vip/dev.c vip/dev.h
> >>>> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
> >>>> supported stds in rk_vip_enum_input callback
> >>>> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
> >>>> - added the supported video_device->tvnorms
> >>>> - s_std will now update the format as this depends on the standard
> >>>> NTSC/PAL (as suggested by Hans in [1])
> >>>> - removed STD_ATSC
> >>>> - moved the colorimetry information to come from the subdev
> >>>> - removed the core s_power subdev calls
> >>>> - dropped cropping in rk_vip_stream struct
> >>>>
> >>>> rockchip-vip.yaml:
> >>>> - fixed a mistake in the name of third clock plckin -> plck
> >>>> - changed the reg maxItems 2 -> 1
> >>>>
> >>>> [1] https://lore.kernel.org/linux-media/[email protected]/
> >>>>
> >>>> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
> >>>>
> >>>> # v4l2-compliance
> >>>> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
> >>>>
> >>>> Compliance test for rk_vip device /dev/video0:
> >>>>
> >>>> Driver Info:
> >>>> Driver name : rk_vip
> >>>> Card type : rk_vip
> >>>> Bus info : platform:ff490000.vip
> >>>> Driver version : 6.6.0
> >>>> Capabilities : 0x84201000
> >>>> Video Capture Multiplanar
> >>>> Streaming
> >>>> Extended Pix Format
> >>>> Device Capabilities
> >>>> Device Caps : 0x04201000
> >>>> Video Capture Multiplanar
> >>>> Streaming
> >>>> Extended Pix Format
> >>>> Media Driver Info:
> >>>> Driver name : rk_vip
> >>>> Model : rk_vip
> >>>> Serial :
> >>>> Bus info : platform:ff490000.vip
> >>>> Media version : 6.6.0
> >>>> Hardware revision: 0x00000000 (0)
> >>>> Driver version : 6.6.0
> >>>> Interface Info:
> >>>> ID : 0x03000002
> >>>> Type : V4L Video
> >>>> Entity Info:
> >>>> ID : 0x00000001 (1)
> >>>> Name : video_rkvip
> >>>> Function : V4L2 I/O
> >>>> Pad 0x01000004 : 0: Sink
> >>>> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
> >>>>
> >>>> Required ioctls:
> >>>> test MC information (see 'Media Driver Info' above): OK
> >>>> test VIDIOC_QUERYCAP: OK
> >>>> test invalid ioctls: OK
> >>>>
> >>>> Allow for multiple opens:
> >>>> test second /dev/video0 open: OK
> >>>> test VIDIOC_QUERYCAP: OK
> >>>> test VIDIOC_G/S_PRIORITY: OK
> >>>> test for unlimited opens: OK
> >>>>
> >>>> Debug ioctls:
> >>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> >>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
> >>>>
> >>>> Input ioctls:
> >>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> >>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> >>>> test VIDIOC_G/S/ENUMINPUT: OK
> >>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> >>>> Inputs: 1 Audio Inputs: 0 Tuners: 0
> >>>>
> >>>> Output ioctls:
> >>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> >>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> >>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> >>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> >>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
> >>>>
> >>>> Input/Output configuration ioctls:
> >>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK
> >>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> >>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> >>>> test VIDIOC_G/S_EDID: OK (Not Supported)
> >>>>
> >>>> Control ioctls (Input 0):
> >>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> >>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
> >>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
> >>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> >>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> >>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> >>>> Standard Controls: 0 Private Controls: 0
> >>>>
> >>>> Format ioctls (Input 0):
> >>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> >>>> test VIDIOC_G/S_PARM: OK (Not Supported)
> >>>> test VIDIOC_G_FBUF: OK (Not Supported)
> >>>> test VIDIOC_G_FMT: OK
> >>>> test VIDIOC_TRY_FMT: OK
> >>>> test VIDIOC_S_FMT: OK
> >>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> >>>> test Cropping: OK (Not Supported)
> >>>> test Composing: OK (Not Supported)
> >>>> test Scaling: OK (Not Supported)
> >>>>
> >>>> Codec ioctls (Input 0):
> >>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> >>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >>>>
> >>>> Buffer ioctls (Input 0):
> >>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >>>> test VIDIOC_EXPBUF: OK
> >>>> test Requests: OK (Not Supported)
> >>>>
> >>>> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
> >>>>
> >>>> Mehdi Djait (3):
> >>>> media: dt-bindings: media: add bindings for Rockchip VIP
> >>>> media: rockchip: Add a driver for Rockhip's camera interface
> >>>> arm64: dts: rockchip: Add the camera interface
> >>>>
> >>>> .../bindings/media/rockchip,px30-vip.yaml | 93 ++
> >>>> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
> >>>> drivers/media/platform/rockchip/Kconfig | 1 +
> >>>> drivers/media/platform/rockchip/Makefile | 1 +
> >>>> drivers/media/platform/rockchip/vip/Kconfig | 14 +
> >>>> drivers/media/platform/rockchip/vip/Makefile | 3 +
> >>>> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
> >>>> drivers/media/platform/rockchip/vip/dev.c | 346 +++++
> >>>> drivers/media/platform/rockchip/vip/dev.h | 163 +++
> >>>> drivers/media/platform/rockchip/vip/regs.h | 260 ++++
> >>>> 10 files changed, 2103 insertions(+)
> >>>> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/Makefile
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/capture.c
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.c
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.h
> >>>> create mode 100644 drivers/media/platform/rockchip/vip/regs.h
> >>>>
> >>>> --
> >>>> 2.41.0
> >>>>
> >>>
> >
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul, Hi Michael
On Wed, Oct 25, 2023 at 11:54:27AM +0200, Paul Kocialkowski wrote:
> Michael,
>
> On Wed 25 Oct 23, 11:17, Michael Riesch wrote:
> > Hi Paul,
> >
> > On 10/25/23 10:43, Paul Kocialkowski wrote:
> > > [...]
> > >>> Here is the detail of my research on the concerned chips. The + at the beginning
> > >>> of the line indicate support in Rockchip's 4.4 tree:
> > >>>
> > >>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology
> > >>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
> > >>> + PX30 (2017): VIP pins + VIP registers
> > >>> + RK3328 (2017): CIF pins + VIP terminology
> > >>> - RK3326 (2017): CIF pins + VIP terminology
> > >>> - RK3399 (2016): CIF pins
> > >>> - RK3368 (2015): CIF pins
> > >>> - PX2 (2014-11): CIF pins + CIF registers
> > >>> + RK3126/RK3128 (2014-10): CIF pins + registers
> > >>> + RK3288 (2014-05): CIF pins + VIP terminology
> > >>> - RK3026 (2013): CIF pins + CIF registers
> > >>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
> > >>> - RK3066 (2012): CIF pins + CIF registers
> > >>>
> > >>> Note that there are a few variations over time (added/removed registers), but
> > >>> the offsets of crucial registers are always the same, so we can safely
> > >>> assume this is the same unit in different generations.
> > >>>
> > >>> Since the RK3066 is the first model starting the RK30 lineup I think we can
> > >>> safely use that for the "base" compatible to be used for e.g. the bindings
> > >>> document, instead of px30 which is just one of the many SoCs that use this unit.
> > >>
> > >> Once the name of the driver is defined and adjusted in v9, I can try to
> > >> give the series a shot on my RK3568 board. First attempts to do so
This sounds good!
> > >> basing on Maxime's v5 showed that with a few modifications the DVP
> > >> feature works fine. In a subsequent step, we could discuss the inclusion
> > >> of the MIPI CSI-2 things in order to keep the driver sufficiently general.
> > >
> > > Nice! I guess there will be a need to introduce a variant structure associated
> > > to each compatible to express the differences betweens these different
> > > generations.
> >
> > Indeed. If Mehdi and you suggest something, I'd be happy to review.
>
> Well the be honest the scope of work on our side is really centered on PX30
> and merging this first version.
>
> > Otherwise, I'll try to come up with something reasonable. IMHO it would
> > make sense (as a first step) to have the clocks and the resets in this
> > structure, as well as a sub-structure that describes the DVP. The latter
> > consists of registers mainly, but maybe supported input/output formats
> > and other things should go in there as well. Also, downstream code has a
> > significant number of
> > if (some condition including chip_id) A; else B;
> > things that we should probably get rid of with this variant structure.
>
> Indeed I think we want to try avoid that. Another common option is to define
> capability flags to represent differences between generations in a more
> practical and clean way than explicitly checking chip ids or so.
>
> > As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X
> > will have one of those, RK3588 will feature even six of them. So we
> > should add a const array to the variant structure.
> >
> > > Note that we will also probably need to convert the driver over to a MC-centric
> > > approach, but this is of course outside of the scope of this series.
> >
> > That would absolutely make sense. What is missing, though? (I was
> > wondering that the driver calls media_device_(un)register but no
> > /dev/mediaX device pops up.)
>
> Switching from video node-centric to MC-centric is more of a semantic change.
> In the first case we expect that subdevs are configured by the video device
> driver and userspace is not expected to change anything in the media topology
> or to configure media entities explicitly.
>
> In the latter case it's the opposite : the driver should never try to push
> configuration to a subdev and should instead validate that the current
> configuration makes sense.
>
> Still, I believe should be a media device registered and visible to userspace.
> Mehdi could you take a look at this? Do you see a media device in `media-ctl -p`
> and /dev/mediaX?
Yes I do have a media device
media-ctl -p
Media device information
------------------------
driver rockchip-cif
model rk_cif
serial
bus info platform:ff490000.video-capture
hw revision 0x0
driver version 6.6.0
Device topology
- entity 1: rockchip_cif (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "tw9900 2-0044":0 [ENABLED]
- entity 5: tw9900 2-0044 (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Source
[fmt:UYVY8_2X8/720x480 field:none colorspace:smpte170m xfer:709 ycbcr:601]
-> "rockchip_cif":0 [ENABLED]
>
> Cheers,
>
> Paul
>
> > Best regards,
> > Michael
> >
> > >
> > > Cheers,
> > >
> > > Paul
> > >
> > >> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
Of course I will :)
--
Kind Regards
Mehdi Djait
Hi Mehdi,
On 10/25/23 12:33, Mehdi Djait wrote:
> Hi Paul, Hi Michael
>
> On Wed, Oct 25, 2023 at 11:54:27AM +0200, Paul Kocialkowski wrote:
>> Michael,
>>
>> On Wed 25 Oct 23, 11:17, Michael Riesch wrote:
>>> Hi Paul,
>>>
>>> On 10/25/23 10:43, Paul Kocialkowski wrote:
>>>> [...]
>>>>>> Here is the detail of my research on the concerned chips. The + at the beginning
>>>>>> of the line indicate support in Rockchip's 4.4 tree:
>>>>>>
>>>>>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology
>>>>>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers
>>>>>> + PX30 (2017): VIP pins + VIP registers
>>>>>> + RK3328 (2017): CIF pins + VIP terminology
>>>>>> - RK3326 (2017): CIF pins + VIP terminology
>>>>>> - RK3399 (2016): CIF pins
>>>>>> - RK3368 (2015): CIF pins
>>>>>> - PX2 (2014-11): CIF pins + CIF registers
>>>>>> + RK3126/RK3128 (2014-10): CIF pins + registers
>>>>>> + RK3288 (2014-05): CIF pins + VIP terminology
>>>>>> - RK3026 (2013): CIF pins + CIF registers
>>>>>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers
>>>>>> - RK3066 (2012): CIF pins + CIF registers
>>>>>>
>>>>>> Note that there are a few variations over time (added/removed registers), but
>>>>>> the offsets of crucial registers are always the same, so we can safely
>>>>>> assume this is the same unit in different generations.
>>>>>>
>>>>>> Since the RK3066 is the first model starting the RK30 lineup I think we can
>>>>>> safely use that for the "base" compatible to be used for e.g. the bindings
>>>>>> document, instead of px30 which is just one of the many SoCs that use this unit.
>>>>>
>>>>> Once the name of the driver is defined and adjusted in v9, I can try to
>>>>> give the series a shot on my RK3568 board. First attempts to do so
>
> This sounds good!
>
>>>>> basing on Maxime's v5 showed that with a few modifications the DVP
>>>>> feature works fine. In a subsequent step, we could discuss the inclusion
>>>>> of the MIPI CSI-2 things in order to keep the driver sufficiently general.
>>>>
>>>> Nice! I guess there will be a need to introduce a variant structure associated
>>>> to each compatible to express the differences betweens these different
>>>> generations.
>>>
>>> Indeed. If Mehdi and you suggest something, I'd be happy to review.
>>
>> Well the be honest the scope of work on our side is really centered on PX30
>> and merging this first version.
>>
>>> Otherwise, I'll try to come up with something reasonable. IMHO it would
>>> make sense (as a first step) to have the clocks and the resets in this
>>> structure, as well as a sub-structure that describes the DVP. The latter
>>> consists of registers mainly, but maybe supported input/output formats
>>> and other things should go in there as well. Also, downstream code has a
>>> significant number of
>>> if (some condition including chip_id) A; else B;
>>> things that we should probably get rid of with this variant structure.
>>
>> Indeed I think we want to try avoid that. Another common option is to define
>> capability flags to represent differences between generations in a more
>> practical and clean way than explicitly checking chip ids or so.
>>
>>> As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X
>>> will have one of those, RK3588 will feature even six of them. So we
>>> should add a const array to the variant structure.
>>>
>>>> Note that we will also probably need to convert the driver over to a MC-centric
>>>> approach, but this is of course outside of the scope of this series.
>>>
>>> That would absolutely make sense. What is missing, though? (I was
>>> wondering that the driver calls media_device_(un)register but no
>>> /dev/mediaX device pops up.)
Sorry, had a false memory of this. The media device *does* pop up.
>> Switching from video node-centric to MC-centric is more of a semantic change.
>> In the first case we expect that subdevs are configured by the video device
>> driver and userspace is not expected to change anything in the media topology
>> or to configure media entities explicitly.
>>
>> In the latter case it's the opposite : the driver should never try to push
>> configuration to a subdev and should instead validate that the current
>> configuration makes sense.
>>
>> Still, I believe should be a media device registered and visible to userspace.
>> Mehdi could you take a look at this? Do you see a media device in `media-ctl -p`
>> and /dev/mediaX?
>
> Yes I do have a media device
>
> media-ctl -p
> Media device information
> ------------------------
> driver rockchip-cif
> model rk_cif
> serial
> bus info platform:ff490000.video-capture
> hw revision 0x0
> driver version 6.6.0
>
> Device topology
> - entity 1: rockchip_cif (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video0
> pad0: Sink
> <- "tw9900 2-0044":0 [ENABLED]
>
> - entity 5: tw9900 2-0044 (1 pad, 1 link)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev0
> pad0: Source
> [fmt:UYVY8_2X8/720x480 field:none colorspace:smpte170m xfer:709 ycbcr:601]
> -> "rockchip_cif":0 [ENABLED]
>
>>
>> Cheers,
>>
>> Paul
>>
>>> Best regards,
>>> Michael
>>>
>>>>
>>>> Cheers,
>>>>
>>>> Paul
>>>>
>>>>> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated.
>
> Of course I will :)
Cool, thanks! Looking forward to it!
Best regards,
Michael
>
> --
> Kind Regards
> Mehdi Djait