Hi,
This series is the v7 series that attempts to support the Camera Subsystem
found on StarFive JH7110 SoC.
The following are the media graph for the device and the v4l2-compliance
output.
===========================================================================
[the media graph]:
digraph board {
rankdir=TB
n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
n00000001:port1 -> n0000000d:port0 [style=dashed]
n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
n00000004:port1 -> n00000007 [style=bold]
n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
n0000000d:port1 -> n00000010 [style=bold]
n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
n00000018:port1 -> n00000004:port0 [style=dashed]
n00000018:port1 -> n00000001:port0 [style=dashed]
n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
n00000022:port0 -> n00000018:port0 [style=bold]
}
[the device topology]:
Media controller API version 6.4.0
Media device information
------------------------
driver starfive-camss
model Starfive Camera Subsystem
serial
bus info platform:19840000.camss
hw revision 0x0
driver version 6.4.0
Device topology
- entity 1: stf_isp0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
crop.bounds:(0,0)/1920x1080
crop:(0,0)/1920x1080]
<- "cdns_csi2rx.19800000.csi-bridge":1 []
pad1: Source
[fmt:Y12_1X12/1920x1080 field:none colorspace:srgb
crop.bounds:(0,0)/1920x1080
crop:(0,0)/1920x1080]
-> "stf_vin0_isp0":0 []
- entity 4: stf_vin0_wr (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
<- "cdns_csi2rx.19800000.csi-bridge":1 []
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
-> "stf_vin0_wr_video0":0 [ENABLED,IMMUTABLE]
- entity 7: stf_vin0_wr_video0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "stf_vin0_wr":1 [ENABLED,IMMUTABLE]
- entity 13: stf_vin0_isp0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
<- "stf_isp0":1 []
pad1: Source
[fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
-> "stf_vin0_isp0_video1":0 [ENABLED,IMMUTABLE]
- entity 16: stf_vin0_isp0_video1 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "stf_vin0_isp0":1 [ENABLED,IMMUTABLE]
- entity 24: cdns_csi2rx.19800000.csi-bridge (5 pads, 3 links)
type V4L2 subdev subtype Unknown flags 0
pad0: Sink
<- "imx219 6-0010":0 [ENABLED,IMMUTABLE]
pad1: Source
-> "stf_vin0_wr":0 []
-> "stf_isp0":0 []
pad2: Source
pad3: Source
pad4: Source
- entity 34: imx219 6-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev3
pad0: Source
[fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
crop.bounds:(8,8)/3280x2464
crop:(8,8)/3280x2464]
-> "cdns_csi2rx.19800000.csi-bridge":0 [ENABLED,IMMUTABLE]
===========================================================================
[the v4l2-compliance output]:
v4l2-compliance -s -d /dev/video1
v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
Compliance test for stf camss device /dev/video1:
Driver Info:
Driver name : stf camss
Card type : Starfive Camera Subsystem
Bus info : platform:19840000.camss
Driver version : 6.4.0
Capabilities : 0x85200001
Video Capture
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x05200001
Video Capture
Read/Write
Streaming
Extended Pix Format
Media Driver Info:
Driver name : starfive-camss
Model : Starfive Camera Subsystem
Serial :
Bus info : platform:19840000.camss
Media version : 6.4.0
Hardware revision: 0x00000000 (0)
Driver version : 6.4.0
Interface Info:
ID : 0x03000012
Type : V4L Video
Entity Info:
ID : 0x00000010 (16)
Name : stf_vin0_isp0_video1
Function : V4L2 I/O
Pad 0x01000011 : 0: Sink
Link 0x02000014: from remote pad 0x100000f of entity 'stf_vin0_isp0' (Video Pixel Formatter): Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Test input 0:
Streaming ioctls:
test read/write: OK
test blocking wait: OK
test MMAP (no poll): OK
test MMAP (select): OK
test MMAP (epoll): OK
test USERPTR (no poll): OK (Not Supported)
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for stf camss device /dev/video1: 53, Succeeded: 53, Failed: 0, Warnings: 0
===========================================================================
Changes in v7:
- HAS_DMA is used instead of DMA_CMA in Kconfig.
- Dropped some non-essential member variables.
- Used v4l2_async_nf_add_fwnode_remote() to simplify the relevant code.
- Modified some Local variable types in the function.
- Used v4l2_create_fwnode_links_to_pad() to simplify the relevant code.
- Added error handling for clk_prepare_enable().
- Simplified stfcamss_format_info struct and modified the relevant code.
- Dropped enum_input, g_input and s_input.
- Unified v4l2_ioctl_ops struct.
- Used v4l2_fh_open()/vb2_fop_release to replace deprecated APIs.
- Added a camss directory under the starfive directory and modified the
patch title.
v6 link: https://lore.kernel.org/all/[email protected]/
Changes in v6:
- Added 'bus-type' in bindings example.
- Corrected spelling errors.
- As reviewed by Bryan, used 'nclks' and 'nrsts' variables.
- Added lccf config for ISP.
v5 link: https://lore.kernel.org/all/[email protected]/
Changes in v5:
- Rebased on v6.4-rc1.
- Added new patch.
- Modified ISP driver.
v4 link: https://lore.kernel.org/all/[email protected]/
Previous cover letter from v4:
This patch series adds support for the StarFive Camera Subsystem
found on StarFive JH7110 SoC.
The driver implements V4L2, Media controller and V4L2 subdev interfaces.
Camera sensor using V4L2 subdev interface in the kernel is supported.
The driver is tested on VisionFive V2 board with IMX219 camera sensor.
GStreamer 1.18.5 with v4l2src plugin is supported.
Changes since v3:
Patch 1:
- Modified port@0 and port@1 properties.
- Extended the port@0 example with appropriate properties.
- Added 'port@0' for 'required'
Patch 2:
- Modified spelling errors.
Patch 3:
- Merged patch 5 into the patch with an explanation for compatible in
commit msg.
Patch 6:
- Asserted pixel_rst[i] reset in the loop after the err_disable_pixclk
label.
- Modified Code Style for getting sys_rst and p_rst.
- Renamed clk_name to name and modified the relevant code.
Patch 9:
- Added static for stfcamss_get_mem_res function.
- Added static for isp_close function.
- Fixed implicit conversion warning for stf_vin_map_isp_pad function.
- Dropped unused variables.
v3: https://lore.kernel.org/all/[email protected]/
Changes since v2:
- Rebased on v6.3-rc1.
Patch 1:
- Modified spelling errors.
- Added port@0.
- Modified '$ref' of port.
- Added 'ports' to 'required'.
- Dropped 'stfcamss' label in example.
- Added port@0 in example.
- Added MAINTAINERS file.
Patch 2:
- Split this patch into three new patches.
- Modified compatible property.
- Replaced clock names with the existing names.
- Modified 'bus-type' and 'clock-lanes'
- Added port@2 - port@4
- Dropped 'csi2rx' label in example.
Patch 3:
- Updated rst and dot file as three pipelines were deleted.
Patch 4:
- Split this patch into three new patches.
- Dropped .s_power() and .get_fmt().
- Dropped CSI-2 DT support.
- Dropped v4l2_device_register_subdev_nodes().
- Used assigned-clock-rates in DT to set clk value.
- Modified 'compatible' field.
Patch 5:
- Deleted three pipelines.
- Modified 'stfcamss_clocks'/'stfcamss_resets' struct.
- Dropped stfcamss_find_sensor() function.
- Removed redundant code from stfcamss_of_parse_endpoint_node().
- Modified spelling errors.
- Rewrote stfcamss_reg_media_subdev_node() function.
- Modified stfcamss_subdev_notifier_bound().
- Modified stfcamss_probe() function.
- Dropped stfcamss_suspend() and stfcamss_resume().
- Dropped dev_info() in stfcamss_remove() function.
- Added 'stf_' prefix for enum subdev_type.
- Moved all includes to the top in stf_camss.h file.
- Dropped unused fields in stfcamss struct.
- Replaced Custom logging macros with regular macros.
- Rewrote register read and write functions.
- Used lowercase for all hex constants.
- Used macro to name registers.
- Dropped unused ioctl and stf_isp_ioctl.h file.
v2: https://lore.kernel.org/all/[email protected]/
Changes since v1:
- Deleted starfive,jh7110-mipi-csi2.yaml.
- Converted cdns,csi2rx.txt to cdns,csi2rx.yaml and added ‘resets’
properties.
- Added ‘cdns,csi2rx.yaml’ in ‘CADENCE MIPI-CSI2 BRIDGES’ entry.
- The following contents were modified in starfive,jh7110-camss.yaml:
dropped quotes from ’id’ and ‘schema’; dropped ‘|’ for ‘description’;
corrected the wrong or redundant words: ‘a ISP’, ‘PD ISP’;
dropped ‘minItems’ for ‘reg’, ‘clocks’, ‘resets’ and ‘interrupts’;
dropped the '_clk' and 'rst_' prefix about the 'clock-names' and
'reset-names';
changed ‘endpoint@1’ to ‘endpoint’; updated examples;
- Updated Subject for some patches.
- Merged patch 6, 7, 8, 9, 10, 11 into one patch.
Jack Zhu (6):
media: dt-bindings: Add JH7110 Camera Subsystem
media: admin-guide: Add starfive_camss.rst for Starfive Camera
Subsystem
media: starfive: camss: Add basic driver
media: starfive: camss: Add video driver
media: starfive: camss: Add ISP driver
media: starfive: camss: Add VIN driver
.../admin-guide/media/starfive_camss.rst | 57 +
.../media/starfive_camss_graph.dot | 16 +
.../admin-guide/media/v4l-drivers.rst | 1 +
.../bindings/media/starfive,jh7110-camss.yaml | 180 +++
MAINTAINERS | 9 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/starfive/Kconfig | 5 +
drivers/media/platform/starfive/Makefile | 2 +
drivers/media/platform/starfive/camss/Kconfig | 16 +
.../media/platform/starfive/camss/Makefile | 14 +
.../media/platform/starfive/camss/stf_camss.c | 444 +++++++
.../media/platform/starfive/camss/stf_camss.h | 151 +++
.../media/platform/starfive/camss/stf_isp.c | 519 ++++++++
.../media/platform/starfive/camss/stf_isp.h | 479 ++++++++
.../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++
.../media/platform/starfive/camss/stf_video.c | 724 +++++++++++
.../media/platform/starfive/camss/stf_video.h | 92 ++
.../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
.../media/platform/starfive/camss/stf_vin.h | 173 +++
.../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
21 files changed, 4613 insertions(+)
create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
create mode 100644 Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
create mode 100644 drivers/media/platform/starfive/Kconfig
create mode 100644 drivers/media/platform/starfive/Makefile
create mode 100644 drivers/media/platform/starfive/camss/Kconfig
create mode 100644 drivers/media/platform/starfive/camss/Makefile
create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
--
2.34.1
Add basic platform driver for StarFive Camera Subsystem.
Reviewed-by: Bryan O'Donoghue <[email protected]>
Signed-off-by: Jack Zhu <[email protected]>
---
MAINTAINERS | 1 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/starfive/Kconfig | 5 +
drivers/media/platform/starfive/Makefile | 2 +
drivers/media/platform/starfive/camss/Kconfig | 16 +
.../media/platform/starfive/camss/Makefile | 8 +
.../media/platform/starfive/camss/stf_camss.c | 338 ++++++++++++++++++
.../media/platform/starfive/camss/stf_camss.h | 146 ++++++++
9 files changed, 518 insertions(+)
create mode 100644 drivers/media/platform/starfive/Kconfig
create mode 100644 drivers/media/platform/starfive/Makefile
create mode 100644 drivers/media/platform/starfive/camss/Kconfig
create mode 100644 drivers/media/platform/starfive/camss/Makefile
create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 71291dc58671..62f9e3593eab 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -20077,6 +20077,7 @@ L: [email protected]
S: Maintained
F: Documentation/admin-guide/media/starfive_camss.rst
F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
+F: drivers/media/platform/starfive/camss
STARFIVE DEVICETREES
M: Emil Renner Berthing <[email protected]>
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ee579916f874..627eaa0ab3ee 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -80,6 +80,7 @@ source "drivers/media/platform/renesas/Kconfig"
source "drivers/media/platform/rockchip/Kconfig"
source "drivers/media/platform/samsung/Kconfig"
source "drivers/media/platform/st/Kconfig"
+source "drivers/media/platform/starfive/Kconfig"
source "drivers/media/platform/sunxi/Kconfig"
source "drivers/media/platform/ti/Kconfig"
source "drivers/media/platform/verisilicon/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 5453bb868e67..5a038498a370 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -23,6 +23,7 @@ obj-y += renesas/
obj-y += rockchip/
obj-y += samsung/
obj-y += st/
+obj-y += starfive/
obj-y += sunxi/
obj-y += ti/
obj-y += verisilicon/
diff --git a/drivers/media/platform/starfive/Kconfig b/drivers/media/platform/starfive/Kconfig
new file mode 100644
index 000000000000..7955c2a0a4a3
--- /dev/null
+++ b/drivers/media/platform/starfive/Kconfig
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+comment "StarFive media platform drivers"
+
+source "drivers/media/platform/starfive/camss/Kconfig"
diff --git a/drivers/media/platform/starfive/Makefile b/drivers/media/platform/starfive/Makefile
new file mode 100644
index 000000000000..4639fa1bca32
--- /dev/null
+++ b/drivers/media/platform/starfive/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-y += camss/
diff --git a/drivers/media/platform/starfive/camss/Kconfig b/drivers/media/platform/starfive/camss/Kconfig
new file mode 100644
index 000000000000..dafe1d24324b
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/Kconfig
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_STARFIVE_CAMSS
+ tristate "Starfive Camera Subsystem driver"
+ depends on V4L_PLATFORM_DRIVERS
+ depends on VIDEO_DEV && OF
+ depends on HAS_DMA
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_FWNODE
+ help
+ Enable this to support for the Starfive Camera subsystem
+ found on Starfive JH7110 SoC.
+
+ To compile this driver as a module, choose M here: the
+ module will be called stf-camss.
diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
new file mode 100644
index 000000000000..d56ddd078a71
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for StarFive Camera Subsystem driver
+#
+
+starfive-camss-objs += stf_camss.o
+
+obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
new file mode 100644
index 000000000000..dc2b5dba7bd4
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_camss.c
@@ -0,0 +1,338 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_camss.c
+ *
+ * Starfive Camera Subsystem driver
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-mc.h>
+
+#include "stf_camss.h"
+
+static const char * const stfcamss_clocks[] = {
+ "clk_apb_func",
+ "clk_wrapper_clk_c",
+ "clk_dvp_inv",
+ "clk_axiwr",
+ "clk_mipi_rx0_pxl",
+ "clk_ispcore_2x",
+ "clk_isp_axi",
+};
+
+static const char * const stfcamss_resets[] = {
+ "rst_wrapper_p",
+ "rst_wrapper_c",
+ "rst_axird",
+ "rst_axiwr",
+ "rst_isp_top_n",
+ "rst_isp_top_axi",
+};
+
+static int stfcamss_get_mem_res(struct platform_device *pdev,
+ struct stfcamss *stfcamss)
+{
+ stfcamss->syscon_base =
+ devm_platform_ioremap_resource_byname(pdev, "syscon");
+ if (IS_ERR(stfcamss->syscon_base))
+ return PTR_ERR(stfcamss->syscon_base);
+
+ stfcamss->isp_base =
+ devm_platform_ioremap_resource_byname(pdev, "isp");
+ if (IS_ERR(stfcamss->isp_base))
+ return PTR_ERR(stfcamss->isp_base);
+
+ return 0;
+}
+
+/*
+ * stfcamss_of_parse_endpoint_node - Parse port endpoint node
+ * @dev: Device
+ * @node: Device node to be parsed
+ * @csd: Parsed data from port endpoint node
+ *
+ * Return 0 on success or a negative error code on failure
+ */
+static int stfcamss_of_parse_endpoint_node(struct device *dev,
+ struct device_node *node,
+ struct stfcamss_async_subdev *csd)
+{
+ struct v4l2_fwnode_endpoint vep = { { 0 } };
+
+ v4l2_fwnode_endpoint_parse(of_fwnode_handle(node), &vep);
+ dev_dbg(dev, "vep.base.port = %d, id = %d\n",
+ vep.base.port, vep.base.id);
+
+ csd->port = vep.base.port;
+
+ return 0;
+}
+
+/*
+ * stfcamss_of_parse_ports - Parse ports node
+ * @stfcamss: STFCAMSS device
+ *
+ * Return number of "port" nodes found in "ports" node
+ */
+static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
+{
+ struct device *dev = stfcamss->dev;
+ struct device_node *node = NULL;
+ int ret, num_subdevs = 0;
+
+ for_each_endpoint_of_node(dev->of_node, node) {
+ struct stfcamss_async_subdev *csd;
+
+ if (!of_device_is_available(node))
+ continue;
+
+ csd = v4l2_async_nf_add_fwnode_remote(&stfcamss->notifier,
+ of_fwnode_handle(node),
+ struct stfcamss_async_subdev);
+ if (IS_ERR(csd)) {
+ ret = PTR_ERR(csd);
+ goto err_cleanup;
+ }
+
+ ret = stfcamss_of_parse_endpoint_node(dev, node, csd);
+ if (ret < 0)
+ goto err_cleanup;
+
+ num_subdevs++;
+ }
+
+ return num_subdevs;
+
+err_cleanup:
+ of_node_put(node);
+ return ret;
+}
+
+static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_subdev *asd)
+{
+ struct media_pad *pad[STF_PADS_NUM];
+ unsigned int i, pad_num = 0;
+
+ for (i = 0; i < pad_num; ++i) {
+ int ret;
+
+ ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int stfcamss_subdev_notifier_complete(struct v4l2_async_notifier *ntf)
+{
+ struct stfcamss *stfcamss =
+ container_of(ntf, struct stfcamss, notifier);
+
+ return v4l2_device_register_subdev_nodes(&stfcamss->v4l2_dev);
+}
+
+static const struct v4l2_async_notifier_operations
+stfcamss_subdev_notifier_ops = {
+ .bound = stfcamss_subdev_notifier_bound,
+ .complete = stfcamss_subdev_notifier_complete,
+};
+
+static const struct media_device_ops stfcamss_media_ops = {
+ .link_notify = v4l2_pipeline_link_notify,
+};
+
+static void stfcamss_mc_init(struct platform_device *pdev,
+ struct stfcamss *stfcamss)
+{
+ stfcamss->media_dev.dev = stfcamss->dev;
+ strscpy(stfcamss->media_dev.model, "Starfive Camera Subsystem",
+ sizeof(stfcamss->media_dev.model));
+ stfcamss->media_dev.ops = &stfcamss_media_ops;
+ media_device_init(&stfcamss->media_dev);
+
+ stfcamss->v4l2_dev.mdev = &stfcamss->media_dev;
+}
+
+/*
+ * stfcamss_probe - Probe STFCAMSS platform device
+ * @pdev: Pointer to STFCAMSS platform device
+ *
+ * Return 0 on success or a negative error code on failure
+ */
+static int stfcamss_probe(struct platform_device *pdev)
+{
+ struct stfcamss *stfcamss;
+ struct device *dev = &pdev->dev;
+ int ret, num_subdevs;
+ unsigned int i;
+
+ stfcamss = devm_kzalloc(dev, sizeof(*stfcamss), GFP_KERNEL);
+ if (!stfcamss)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(stfcamss->irq); ++i) {
+ stfcamss->irq[i] = platform_get_irq(pdev, i);
+ if (stfcamss->irq[i] < 0)
+ return dev_err_probe(&pdev->dev, stfcamss->irq[i],
+ "Failed to get irq%d", i);
+ }
+
+ stfcamss->nclks = ARRAY_SIZE(stfcamss->sys_clk);
+ for (i = 0; i < stfcamss->nclks; ++i)
+ stfcamss->sys_clk[i].id = stfcamss_clocks[i];
+ ret = devm_clk_bulk_get(dev, stfcamss->nclks, stfcamss->sys_clk);
+ if (ret) {
+ dev_err(dev, "Failed to get clk controls\n");
+ return ret;
+ }
+
+ stfcamss->nrsts = ARRAY_SIZE(stfcamss->sys_rst);
+ for (i = 0; i < stfcamss->nrsts; ++i)
+ stfcamss->sys_rst[i].id = stfcamss_resets[i];
+ ret = devm_reset_control_bulk_get_shared(dev, stfcamss->nrsts,
+ stfcamss->sys_rst);
+ if (ret) {
+ dev_err(dev, "Failed to get reset controls\n");
+ return ret;
+ }
+
+ ret = stfcamss_get_mem_res(pdev, stfcamss);
+ if (ret) {
+ dev_err(dev, "Could not map registers\n");
+ return ret;
+ }
+
+ stfcamss->dev = dev;
+ platform_set_drvdata(pdev, stfcamss);
+
+ v4l2_async_nf_init(&stfcamss->notifier);
+
+ num_subdevs = stfcamss_of_parse_ports(stfcamss);
+ if (num_subdevs < 0) {
+ ret = -ENODEV;
+ goto err_cleanup_notifier;
+ }
+
+ stfcamss_mc_init(pdev, stfcamss);
+
+ ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
+ goto err_cleanup_notifier;
+ }
+
+ ret = media_device_register(&stfcamss->media_dev);
+ if (ret) {
+ dev_err(dev, "Failed to register media device: %d\n", ret);
+ goto err_unregister_device;
+ }
+
+ pm_runtime_enable(dev);
+
+ stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
+ ret = v4l2_async_nf_register(&stfcamss->v4l2_dev, &stfcamss->notifier);
+ if (ret) {
+ dev_err(dev, "Failed to register async subdev nodes: %d\n",
+ ret);
+ goto err_unregister_media_dev;
+ }
+
+ return 0;
+
+err_unregister_media_dev:
+ media_device_unregister(&stfcamss->media_dev);
+err_unregister_device:
+ v4l2_device_unregister(&stfcamss->v4l2_dev);
+err_cleanup_notifier:
+ v4l2_async_nf_cleanup(&stfcamss->notifier);
+ return ret;
+}
+
+/*
+ * stfcamss_remove - Remove STFCAMSS platform device
+ * @pdev: Pointer to STFCAMSS platform device
+ *
+ * Always returns 0.
+ */
+static int stfcamss_remove(struct platform_device *pdev)
+{
+ struct stfcamss *stfcamss = platform_get_drvdata(pdev);
+
+ v4l2_device_unregister(&stfcamss->v4l2_dev);
+ media_device_cleanup(&stfcamss->media_dev);
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+static const struct of_device_id stfcamss_of_match[] = {
+ { .compatible = "starfive,jh7110-camss" },
+ { /* sentinel */ },
+};
+
+MODULE_DEVICE_TABLE(of, stfcamss_of_match);
+
+static int __maybe_unused stfcamss_runtime_suspend(struct device *dev)
+{
+ struct stfcamss *stfcamss = dev_get_drvdata(dev);
+
+ reset_control_assert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
+ reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
+ reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
+ clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
+ clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
+
+ return 0;
+}
+
+static int __maybe_unused stfcamss_runtime_resume(struct device *dev)
+{
+ struct stfcamss *stfcamss = dev_get_drvdata(dev);
+ int ret;
+
+ ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
+ if (ret)
+ return ret;
+
+ ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
+ if (ret)
+ return ret;
+
+ reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
+ reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
+ reset_control_deassert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
+
+ return 0;
+}
+
+static const struct dev_pm_ops stfcamss_pm_ops = {
+ SET_RUNTIME_PM_OPS(stfcamss_runtime_suspend,
+ stfcamss_runtime_resume,
+ NULL)
+};
+
+static struct platform_driver stfcamss_driver = {
+ .probe = stfcamss_probe,
+ .remove = stfcamss_remove,
+ .driver = {
+ .name = "starfive-camss",
+ .pm = &stfcamss_pm_ops,
+ .of_match_table = of_match_ptr(stfcamss_of_match),
+ },
+};
+
+module_platform_driver(stfcamss_driver);
+
+MODULE_AUTHOR("StarFive Corporation");
+MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
new file mode 100644
index 000000000000..15c4f34b9054
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_camss.h
@@ -0,0 +1,146 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * stf_camss.h
+ *
+ * Starfive Camera Subsystem driver
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_CAMSS_H
+#define STF_CAMSS_H
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/reset.h>
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-device.h>
+
+#define STF_DVP_NAME "stf_dvp"
+#define STF_CSI_NAME "cdns_csi2rx"
+#define STF_ISP_NAME "stf_isp"
+#define STF_VIN_NAME "stf_vin"
+
+#define STF_PAD_SINK 0
+#define STF_PAD_SRC 1
+#define STF_PADS_NUM 2
+
+enum stf_port_num {
+ STF_PORT_DVP = 0,
+ STF_PORT_CSI2RX
+};
+
+enum stf_clk {
+ STF_CLK_APB_FUNC = 0,
+ STF_CLK_WRAPPER_CLK_C,
+ STF_CLK_DVP_INV,
+ STF_CLK_AXIWR,
+ STF_CLK_MIPI_RX0_PXL,
+ STF_CLK_ISPCORE_2X,
+ STF_CLK_ISP_AXI,
+ STF_CLK_NUM
+};
+
+enum stf_rst {
+ STF_RST_WRAPPER_P = 0,
+ STF_RST_WRAPPER_C,
+ STF_RST_AXIRD,
+ STF_RST_AXIWR,
+ STF_RST_ISP_TOP_N,
+ STF_RST_ISP_TOP_AXI,
+ STF_RST_NUM
+};
+
+enum stf_irq {
+ STF_IRQ_VINWR = 0,
+ STF_IRQ_ISP,
+ STF_IRQ_ISPCSIL,
+ STF_IRQ_NUM
+};
+
+struct stfcamss {
+ struct v4l2_device v4l2_dev;
+ struct media_device media_dev;
+ struct media_pipeline pipe;
+ struct device *dev;
+ struct v4l2_async_notifier notifier;
+ void __iomem *syscon_base;
+ void __iomem *isp_base;
+ int irq[STF_IRQ_NUM];
+ struct clk_bulk_data sys_clk[STF_CLK_NUM];
+ int nclks;
+ struct reset_control_bulk_data sys_rst[STF_RST_NUM];
+ int nrsts;
+};
+
+struct stfcamss_async_subdev {
+ struct v4l2_async_subdev asd; /* must be first */
+ enum stf_port_num port;
+};
+
+static inline u32 stf_isp_reg_read(struct stfcamss *stfcamss, u32 reg)
+{
+ return ioread32(stfcamss->isp_base + reg);
+}
+
+static inline void stf_isp_reg_write(struct stfcamss *stfcamss,
+ u32 reg, u32 val)
+{
+ iowrite32(val, stfcamss->isp_base + reg);
+}
+
+static inline void stf_isp_reg_write_delay(struct stfcamss *stfcamss,
+ u32 reg, u32 val, u32 delay)
+{
+ iowrite32(val, stfcamss->isp_base + reg);
+ usleep_range(1000 * delay, 1000 * delay + 100);
+}
+
+static inline void stf_isp_reg_set_bit(struct stfcamss *stfcamss,
+ u32 reg, u32 mask, u32 val)
+{
+ u32 value;
+
+ value = ioread32(stfcamss->isp_base + reg) & ~mask;
+ val &= mask;
+ val |= value;
+ iowrite32(val, stfcamss->isp_base + reg);
+}
+
+static inline void stf_isp_reg_set(struct stfcamss *stfcamss, u32 reg, u32 mask)
+{
+ iowrite32(ioread32(stfcamss->isp_base + reg) | mask,
+ stfcamss->isp_base + reg);
+}
+
+static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32 reg)
+{
+ return ioread32(stfcamss->syscon_base + reg);
+}
+
+static inline void stf_syscon_reg_write(struct stfcamss *stfcamss,
+ u32 reg, u32 val)
+{
+ iowrite32(val, stfcamss->syscon_base + reg);
+}
+
+static inline void stf_syscon_reg_set_bit(struct stfcamss *stfcamss,
+ u32 reg, u32 bit_mask)
+{
+ u32 value;
+
+ value = ioread32(stfcamss->syscon_base + reg);
+ iowrite32(value | bit_mask, stfcamss->syscon_base + reg);
+}
+
+static inline void stf_syscon_reg_clear_bit(struct stfcamss *stfcamss,
+ u32 reg, u32 bit_mask)
+{
+ u32 value;
+
+ value = ioread32(stfcamss->syscon_base + reg);
+ iowrite32(value & ~bit_mask, stfcamss->syscon_base + reg);
+}
+#endif /* STF_CAMSS_H */
--
2.34.1
Add the bindings documentation for Starfive JH7110 Camera Subsystem
which is used for handing image sensor data.
Reviewed-by: Krzysztof Kozlowski <[email protected]>
Reviewed-by: Laurent Pinchart <[email protected]>
Signed-off-by: Jack Zhu <[email protected]>
---
.../bindings/media/starfive,jh7110-camss.yaml | 180 ++++++++++++++++++
MAINTAINERS | 7 +
2 files changed, 187 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
diff --git a/Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml b/Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
new file mode 100644
index 000000000000..c66586d90fa2
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
@@ -0,0 +1,180 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/starfive,jh7110-camss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Starfive SoC CAMSS ISP
+
+maintainers:
+ - Jack Zhu <[email protected]>
+ - Changhuang Liang <[email protected]>
+
+description:
+ The Starfive CAMSS ISP is a Camera interface for Starfive JH7110 SoC. It
+ consists of a VIN controller (Video In Controller, a top-level control unit)
+ and an ISP.
+
+properties:
+ compatible:
+ const: starfive,jh7110-camss
+
+ reg:
+ maxItems: 2
+
+ reg-names:
+ items:
+ - const: syscon
+ - const: isp
+
+ clocks:
+ maxItems: 7
+
+ clock-names:
+ items:
+ - const: apb_func
+ - const: wrapper_clk_c
+ - const: dvp_inv
+ - const: axiwr
+ - const: mipi_rx0_pxl
+ - const: ispcore_2x
+ - const: isp_axi
+
+ resets:
+ maxItems: 6
+
+ reset-names:
+ items:
+ - const: wrapper_p
+ - const: wrapper_c
+ - const: axird
+ - const: axiwr
+ - const: isp_top_n
+ - const: isp_top_axi
+
+ power-domains:
+ items:
+ - description: JH7110 ISP Power Domain Switch Controller.
+
+ interrupts:
+ maxItems: 4
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description: Input port for receiving DVP data.
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ bus-type:
+ enum: [5, 6]
+
+ bus-width:
+ enum: [8, 10, 12]
+
+ data-shift:
+ enum: [0, 2]
+ default: 0
+
+ hsync-active:
+ enum: [0, 1]
+ default: 1
+
+ vsync-active:
+ enum: [0, 1]
+ default: 1
+
+ required:
+ - bus-type
+ - bus-width
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Input port for receiving CSI data.
+
+ required:
+ - port@0
+ - port@1
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - clock-names
+ - resets
+ - reset-names
+ - power-domains
+ - interrupts
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ isp@19840000 {
+ compatible = "starfive,jh7110-camss";
+ reg = <0x19840000 0x10000>,
+ <0x19870000 0x30000>;
+ reg-names = "syscon", "isp";
+ clocks = <&ispcrg 0>,
+ <&ispcrg 13>,
+ <&ispcrg 2>,
+ <&ispcrg 12>,
+ <&ispcrg 1>,
+ <&syscrg 51>,
+ <&syscrg 52>;
+ clock-names = "apb_func",
+ "wrapper_clk_c",
+ "dvp_inv",
+ "axiwr",
+ "mipi_rx0_pxl",
+ "ispcore_2x",
+ "isp_axi";
+ resets = <&ispcrg 0>,
+ <&ispcrg 1>,
+ <&ispcrg 10>,
+ <&ispcrg 11>,
+ <&syscrg 41>,
+ <&syscrg 42>;
+ reset-names = "wrapper_p",
+ "wrapper_c",
+ "axird",
+ "axiwr",
+ "isp_top_n",
+ "isp_top_axi";
+ power-domains = <&pwrc 5>;
+ interrupts = <92>, <87>, <88>, <90>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ vin_from_sc2235: endpoint {
+ remote-endpoint = <&sc2235_to_vin>;
+ bus-type = <5>;
+ bus-width = <8>;
+ data-shift = <2>;
+ hsync-active = <1>;
+ vsync-active = <0>;
+ pclk-sample = <1>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ vin_from_csi2rx: endpoint {
+ remote-endpoint = <&csi2rx_to_vin>;
+ };
+ };
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 869e1c7fc869..9ef5d0e5dff0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -20070,6 +20070,13 @@ M: Ion Badulescu <[email protected]>
S: Odd Fixes
F: drivers/net/ethernet/adaptec/starfire*
+STARFIVE CAMERA SUBSYSTEM DRIVER
+M: Jack Zhu <[email protected]>
+M: Changhuang Liang <[email protected]>
+L: [email protected]
+S: Maintained
+F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
+
STARFIVE DEVICETREES
M: Emil Renner Berthing <[email protected]>
S: Maintained
--
2.34.1
Add Video In Controller driver for StarFive Camera Subsystem.
Signed-off-by: Jack Zhu <[email protected]>
---
.../media/platform/starfive/camss/Makefile | 4 +-
.../media/platform/starfive/camss/stf_camss.c | 42 +-
.../media/platform/starfive/camss/stf_camss.h | 2 +
.../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
.../media/platform/starfive/camss/stf_vin.h | 173 +++
.../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
6 files changed, 1478 insertions(+), 4 deletions(-)
create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
index cdf57e8c9546..ef574e01ca47 100644
--- a/drivers/media/platform/starfive/camss/Makefile
+++ b/drivers/media/platform/starfive/camss/Makefile
@@ -7,6 +7,8 @@ starfive-camss-objs += \
stf_camss.o \
stf_isp.o \
stf_isp_hw_ops.o \
- stf_video.o
+ stf_video.o \
+ stf_vin.o \
+ stf_vin_hw_ops.o
obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
index 6f56b45f57db..834ea63eb833 100644
--- a/drivers/media/platform/starfive/camss/stf_camss.c
+++ b/drivers/media/platform/starfive/camss/stf_camss.c
@@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
return ret;
}
+ ret = stf_vin_subdev_init(stfcamss);
+ if (ret < 0) {
+ dev_err(stfcamss->dev, "Failed to init vin subdev: %d\n", ret);
+ return ret;
+ }
return ret;
}
static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
{
int ret;
+ struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
if (ret < 0) {
dev_err(stfcamss->dev,
"Failed to register stf isp%d entity: %d\n", 0, ret);
- return ret;
+ goto err_reg_isp;
+ }
+
+ ret = stf_vin_register(vin_dev, &stfcamss->v4l2_dev);
+ if (ret < 0) {
+ dev_err(stfcamss->dev,
+ "Failed to register vin entity: %d\n", ret);
+ goto err_reg_vin;
}
+ ret = media_create_pad_link(&isp_dev->subdev.entity,
+ STF_ISP_PAD_SRC,
+ &vin_dev->line[VIN_LINE_ISP].subdev.entity,
+ STF_VIN_PAD_SINK,
+ 0);
+ if (ret < 0) {
+ dev_err(stfcamss->dev,
+ "Failed to link %s->%s entities: %d\n",
+ isp_dev->subdev.entity.name,
+ vin_dev->line[VIN_LINE_ISP].subdev.entity.name, ret);
+ goto err_link;
+ }
+
+ return ret;
+
+err_link:
+ stf_vin_unregister(&stfcamss->vin_dev);
+err_reg_vin:
+ stf_isp_unregister(&stfcamss->isp_dev);
+err_reg_isp:
return ret;
}
static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
{
stf_isp_unregister(&stfcamss->isp_dev);
+ stf_vin_unregister(&stfcamss->vin_dev);
}
static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
@@ -164,12 +198,14 @@ static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
container_of(asd, struct stfcamss_async_subdev, asd);
enum stf_port_num port = csd->port;
struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
+ struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
struct media_pad *pad[STF_PADS_NUM];
unsigned int i, pad_num;
if (port == STF_PORT_CSI2RX) {
- pad[0] = &isp_dev->pads[STF_PAD_SINK];
- pad_num = 1;
+ pad[0] = &vin_dev->line[VIN_LINE_WR].pads[STF_PAD_SINK];
+ pad[1] = &isp_dev->pads[STF_PAD_SINK];
+ pad_num = 2;
} else if (port == STF_PORT_DVP) {
dev_err(stfcamss->dev, "Not support DVP sensor\n");
return -EPERM;
diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
index 9482081288fa..a14f22bc0742 100644
--- a/drivers/media/platform/starfive/camss/stf_camss.h
+++ b/drivers/media/platform/starfive/camss/stf_camss.h
@@ -19,6 +19,7 @@
#include <media/v4l2-device.h>
#include "stf_isp.h"
+#include "stf_vin.h"
#define STF_DVP_NAME "stf_dvp"
#define STF_CSI_NAME "cdns_csi2rx"
@@ -67,6 +68,7 @@ struct stfcamss {
struct media_device media_dev;
struct media_pipeline pipe;
struct device *dev;
+ struct stf_vin_dev vin_dev;
struct stf_isp_dev isp_dev;
struct v4l2_async_notifier notifier;
void __iomem *syscon_base;
diff --git a/drivers/media/platform/starfive/camss/stf_vin.c b/drivers/media/platform/starfive/camss/stf_vin.c
new file mode 100644
index 000000000000..0efa4bbb079c
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_vin.c
@@ -0,0 +1,1069 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_vin.c
+ *
+ * StarFive Camera Subsystem - VIN Module
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/pm_runtime.h>
+
+#include "stf_camss.h"
+
+#define vin_line_array(ptr_line) \
+ ((const struct vin_line (*)[]) &(ptr_line)[-((ptr_line)->id)])
+
+#define line_to_vin_dev(ptr_line) \
+ container_of(vin_line_array(ptr_line), struct stf_vin_dev, line)
+
+#define VIN_FRAME_DROP_MIN_VAL 4
+
+/* ISP ctrl need 1 sec to let frames become stable. */
+#define VIN_FRAME_DROP_SEC_FOR_ISP_CTRL 1
+
+static const struct vin_format vin_formats_wr[] = {
+ { MEDIA_BUS_FMT_SRGGB10_1X10, 10},
+ { MEDIA_BUS_FMT_SGRBG10_1X10, 10},
+ { MEDIA_BUS_FMT_SGBRG10_1X10, 10},
+ { MEDIA_BUS_FMT_SBGGR10_1X10, 10},
+};
+
+static const struct vin_format vin_formats_uo[] = {
+ { MEDIA_BUS_FMT_Y12_1X12, 8},
+};
+
+static const struct vin_format_table vin_formats_table[] = {
+ /* VIN_LINE_WR */
+ { vin_formats_wr, ARRAY_SIZE(vin_formats_wr) },
+ /* VIN_LINE_ISP */
+ { vin_formats_uo, ARRAY_SIZE(vin_formats_uo) },
+};
+
+static void vin_buffer_done(struct vin_line *line);
+static void vin_change_buffer(struct vin_line *line);
+static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output);
+static void vin_output_init_addrs(struct vin_line *line);
+static struct v4l2_mbus_framefmt *
+__vin_get_format(struct vin_line *line,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ enum v4l2_subdev_format_whence which);
+
+static char *vin_get_line_subdevname(int line_id)
+{
+ char *name = NULL;
+
+ switch (line_id) {
+ case VIN_LINE_WR:
+ name = "wr";
+ break;
+ case VIN_LINE_ISP:
+ name = "isp0";
+ break;
+ default:
+ name = "unknown";
+ break;
+ }
+ return name;
+}
+
+static enum isp_line_id vin_map_isp_line(enum vin_line_id line)
+{
+ enum isp_line_id line_id;
+
+ if (line > VIN_LINE_WR && line < VIN_LINE_MAX)
+ line_id = STF_ISP_LINE_SRC;
+ else
+ line_id = STF_ISP_LINE_INVALID;
+
+ return line_id;
+}
+
+enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def)
+{
+ enum isp_pad_id pad_id;
+
+ if (line == VIN_LINE_WR)
+ pad_id = STF_ISP_PAD_SINK;
+ else if ((line > VIN_LINE_WR) && (line < VIN_LINE_MAX))
+ pad_id = (enum isp_pad_id)vin_map_isp_line(line);
+ else
+ pad_id = def;
+
+ return pad_id;
+}
+
+int stf_vin_subdev_init(struct stfcamss *stfcamss)
+{
+ struct device *dev = stfcamss->dev;
+ struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
+ int i, ret = 0;
+
+ vin_dev->stfcamss = stfcamss;
+
+ vin_dev->isr_ops = devm_kzalloc(dev, sizeof(*vin_dev->isr_ops),
+ GFP_KERNEL);
+ if (!vin_dev->isr_ops)
+ return -ENOMEM;
+ vin_dev->isr_ops->isr_buffer_done = vin_buffer_done;
+ vin_dev->isr_ops->isr_change_buffer = vin_change_buffer;
+
+ atomic_set(&vin_dev->ref_count, 0);
+
+ ret = devm_request_irq(dev,
+ stfcamss->irq[STF_IRQ_VINWR],
+ stf_vin_wr_irq_handler,
+ 0, "vin_axiwr_irq", vin_dev);
+ if (ret) {
+ dev_err(dev, "Failed to request irq\n");
+ goto out;
+ }
+
+ ret = devm_request_irq(dev,
+ stfcamss->irq[STF_IRQ_ISP],
+ stf_vin_isp_irq_handler,
+ 0, "vin_isp_irq", vin_dev);
+ if (ret) {
+ dev_err(dev, "Failed to request isp irq\n");
+ goto out;
+ }
+
+ ret = devm_request_irq(dev,
+ stfcamss->irq[STF_IRQ_ISPCSIL],
+ stf_vin_isp_irq_csiline_handler,
+ 0, "vin_isp_irq_csiline", vin_dev);
+ if (ret) {
+ dev_err(dev, "failed to request isp irq csiline\n");
+ goto out;
+ }
+
+ for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++) {
+ struct dummy_buffer *dummy_buffer = &vin_dev->dummy_buffer[i];
+
+ mutex_init(&dummy_buffer->stream_lock);
+ dummy_buffer->nums =
+ i == 0 ? VIN_DUMMY_BUFFER_NUMS : ISP_DUMMY_BUFFER_NUMS;
+ dummy_buffer->stream_count = 0;
+ dummy_buffer->buffer =
+ devm_kzalloc(dev,
+ dummy_buffer->nums * sizeof(struct vin_dummy_buffer),
+ GFP_KERNEL);
+ atomic_set(&dummy_buffer->frame_skip, 0);
+ }
+
+ for (i = VIN_LINE_WR; i < STF_ISP_LINE_MAX + 1; i++) {
+ struct vin_line *l = &vin_dev->line[i];
+
+ l->video_out.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ l->video_out.stfcamss = stfcamss;
+ l->id = i;
+ l->formats = vin_formats_table[i].fmts;
+ l->nformats = vin_formats_table[i].nfmts;
+ spin_lock_init(&l->output_lock);
+
+ mutex_init(&l->stream_lock);
+ l->stream_count = 0;
+ }
+
+ return 0;
+out:
+ return ret;
+}
+
+static enum link vin_get_link(struct media_entity *entity)
+{
+ struct v4l2_subdev *subdev;
+ struct media_pad *pad;
+ bool isp = false;
+
+ while (1) {
+ pad = &entity->pads[0];
+ if (!(pad->flags & MEDIA_PAD_FL_SINK))
+ return LINK_ERROR;
+
+ pad = media_pad_remote_pad_first(pad);
+ if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
+ return LINK_ERROR;
+
+ entity = pad->entity;
+ subdev = media_entity_to_v4l2_subdev(entity);
+
+ if (!strncmp(subdev->name, STF_CSI_NAME,
+ strlen(STF_CSI_NAME))) {
+ if (isp)
+ return LINK_CSI_TO_ISP;
+ else
+ return LINK_CSI_TO_WR;
+ } else if (!strncmp(subdev->name, STF_DVP_NAME,
+ strlen(STF_DVP_NAME))) {
+ if (isp)
+ return LINK_DVP_TO_ISP;
+ else
+ return LINK_DVP_TO_WR;
+ } else if (!strncmp(subdev->name, STF_ISP_NAME,
+ strlen(STF_ISP_NAME))) {
+ isp = true;
+ } else {
+ return LINK_ERROR;
+ }
+ }
+}
+
+static int vin_enable_output(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+ unsigned long flags;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ output->state = VIN_OUTPUT_IDLE;
+
+ output->buf[0] = vin_buf_get_pending(output);
+
+ if (!output->buf[0] && output->buf[1]) {
+ output->buf[0] = output->buf[1];
+ output->buf[1] = NULL;
+ }
+
+ if (output->buf[0])
+ output->state = VIN_OUTPUT_SINGLE;
+
+ output->sequence = 0;
+
+ vin_output_init_addrs(line);
+ spin_unlock_irqrestore(&line->output_lock, flags);
+ return 0;
+}
+
+static int vin_disable_output(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+ unsigned long flags;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ output->state = VIN_OUTPUT_OFF;
+
+ spin_unlock_irqrestore(&line->output_lock, flags);
+ return 0;
+}
+
+static u32 vin_line_to_dummy_module(struct vin_line *line)
+{
+ u32 dummy_module = 0;
+
+ switch (line->id) {
+ case VIN_LINE_WR:
+ dummy_module = STF_DUMMY_VIN;
+ break;
+ case VIN_LINE_ISP:
+ dummy_module = STF_DUMMY_ISP;
+ break;
+ default:
+ dummy_module = STF_DUMMY_VIN;
+ break;
+ }
+
+ return dummy_module;
+}
+
+static int vin_alloc_dummy_buffer(struct stf_vin_dev *vin_dev,
+ struct v4l2_mbus_framefmt *fmt,
+ int dummy_module)
+{
+ struct device *dev = vin_dev->stfcamss->dev;
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[dummy_module];
+ struct vin_dummy_buffer *buffer = NULL;
+ int ret = 0, i;
+ u32 aligns;
+
+ for (i = 0; i < dummy_buffer->nums; i++) {
+ buffer = &vin_dev->dummy_buffer[dummy_module].buffer[i];
+ buffer->width = fmt->width;
+ buffer->height = fmt->height;
+ buffer->mcode = fmt->code;
+ if (i == STF_VIN_PAD_SINK) {
+ aligns = ALIGN(fmt->width * 4,
+ STFCAMSS_FRAME_WIDTH_ALIGN_8);
+ buffer->buffer_size = PAGE_ALIGN(aligns * fmt->height);
+ } else if (i == STF_ISP_PAD_SRC) {
+ aligns = ALIGN(fmt->width,
+ STFCAMSS_FRAME_WIDTH_ALIGN_8);
+ buffer->buffer_size =
+ PAGE_ALIGN(aligns * fmt->height * 3 / 2);
+ } else {
+ continue;
+ }
+
+ buffer->vaddr = dma_alloc_coherent(dev,
+ buffer->buffer_size,
+ &buffer->paddr[0],
+ GFP_KERNEL | __GFP_NOWARN);
+
+ if (buffer->vaddr) {
+ if (i == STF_ISP_PAD_SRC)
+ buffer->paddr[1] =
+ (dma_addr_t)(buffer->paddr[0] + aligns * fmt->height);
+ else
+ dev_dbg(dev, "signal plane\n");
+ }
+ }
+
+ return ret;
+}
+
+static void vin_free_dummy_buffer(struct stf_vin_dev *vin_dev, int dummy_module)
+{
+ struct device *dev = vin_dev->stfcamss->dev;
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[dummy_module];
+ struct vin_dummy_buffer *buffer = NULL;
+ int i;
+
+ for (i = 0; i < dummy_buffer->nums; i++) {
+ buffer = &dummy_buffer->buffer[i];
+ if (buffer->vaddr)
+ dma_free_coherent(dev, buffer->buffer_size,
+ buffer->vaddr, buffer->paddr[0]);
+ memset(buffer, 0, sizeof(struct vin_dummy_buffer));
+ }
+}
+
+static void vin_set_dummy_buffer(struct vin_line *line, u32 pad)
+{
+ struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
+ int dummy_module = vin_line_to_dummy_module(line);
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[dummy_module];
+ struct vin_dummy_buffer *buffer = NULL;
+
+ switch (pad) {
+ case STF_VIN_PAD_SINK:
+ if (line->id == VIN_LINE_WR) {
+ buffer = &dummy_buffer->buffer[STF_VIN_PAD_SINK];
+ stf_vin_wr_set_ping_addr(vin_dev, buffer->paddr[0]);
+ stf_vin_wr_set_pong_addr(vin_dev, buffer->paddr[0]);
+ } else {
+ buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
+ stf_vin_isp_set_yuv_addr(vin_dev,
+ buffer->paddr[0],
+ buffer->paddr[1]);
+ }
+ break;
+ case STF_ISP_PAD_SRC:
+ buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
+ stf_vin_isp_set_yuv_addr(vin_dev,
+ buffer->paddr[0],
+ buffer->paddr[1]);
+ break;
+ default:
+ break;
+ }
+}
+
+static int vin_set_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct vin_line *line = v4l2_get_subdevdata(sd);
+ struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
+ int dummy_module = vin_line_to_dummy_module(line);
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[dummy_module];
+ struct v4l2_mbus_framefmt *fmt;
+ enum link link;
+
+ fmt = __vin_get_format(line, NULL,
+ STF_VIN_PAD_SINK, V4L2_SUBDEV_FORMAT_ACTIVE);
+ mutex_lock(&dummy_buffer->stream_lock);
+ if (enable) {
+ if (dummy_buffer->stream_count == 0) {
+ vin_alloc_dummy_buffer(vin_dev, fmt, dummy_module);
+ vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
+ atomic_set(&dummy_buffer->frame_skip,
+ VIN_FRAME_DROP_MIN_VAL);
+ }
+ dummy_buffer->stream_count++;
+ } else {
+ if (dummy_buffer->stream_count == 1) {
+ vin_free_dummy_buffer(vin_dev, dummy_module);
+ /* set buffer addr to zero */
+ vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
+ } else {
+ vin_set_dummy_buffer(line,
+ stf_vin_map_isp_pad(line->id, STF_ISP_PAD_SINK));
+ }
+
+ dummy_buffer->stream_count--;
+ }
+ mutex_unlock(&dummy_buffer->stream_lock);
+
+ mutex_lock(&line->stream_lock);
+ link = vin_get_link(&sd->entity);
+ if (link == LINK_ERROR)
+ goto exit;
+
+ if (enable) {
+ if (line->stream_count == 0) {
+ stf_vin_stream_set(vin_dev, link);
+ if (line->id == VIN_LINE_WR) {
+ stf_vin_wr_irq_enable(vin_dev, 1);
+ stf_vin_wr_stream_set(vin_dev);
+ }
+ }
+ line->stream_count++;
+ } else {
+ if (line->stream_count == 1 && line->id == VIN_LINE_WR)
+ stf_vin_wr_irq_enable(vin_dev, 0);
+ line->stream_count--;
+ }
+exit:
+ mutex_unlock(&line->stream_lock);
+
+ if (enable)
+ vin_enable_output(line);
+ else
+ vin_disable_output(line);
+
+ return 0;
+}
+
+static struct v4l2_mbus_framefmt *
+__vin_get_format(struct vin_line *line,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ enum v4l2_subdev_format_whence which)
+{
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&line->subdev, state, pad);
+ return &line->fmt[pad];
+}
+
+static void vin_try_format(struct vin_line *line,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ struct v4l2_mbus_framefmt *fmt,
+ enum v4l2_subdev_format_whence which)
+{
+ unsigned int i;
+
+ switch (pad) {
+ case STF_VIN_PAD_SINK:
+ /* Set format on sink pad */
+ for (i = 0; i < line->nformats; i++)
+ if (fmt->code == line->formats[i].code)
+ break;
+
+ /* If not found, use UYVY as default */
+ if (i >= line->nformats)
+ fmt->code = line->formats[0].code;
+
+ fmt->width = clamp_t(u32, fmt->width,
+ STFCAMSS_FRAME_MIN_WIDTH,
+ STFCAMSS_FRAME_MAX_WIDTH);
+ fmt->height = clamp_t(u32, fmt->height,
+ STFCAMSS_FRAME_MIN_HEIGHT,
+ STFCAMSS_FRAME_MAX_HEIGHT);
+
+ fmt->field = V4L2_FIELD_NONE;
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+ fmt->flags = 0;
+ break;
+
+ case STF_VIN_PAD_SRC:
+ /* Set and return a format same as sink pad */
+ *fmt = *__vin_get_format(line, state, STF_VIN_PAD_SINK, which);
+ break;
+ }
+
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+}
+
+static int vin_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct vin_line *line = v4l2_get_subdevdata(sd);
+
+ if (code->index >= line->nformats)
+ return -EINVAL;
+ if (code->pad == STF_VIN_PAD_SINK) {
+ code->code = line->formats[code->index].code;
+ } else {
+ struct v4l2_mbus_framefmt *sink_fmt;
+
+ sink_fmt = __vin_get_format(line, state, STF_VIN_PAD_SINK,
+ code->which);
+
+ code->code = sink_fmt->code;
+ if (!code->code)
+ return -EINVAL;
+ }
+ code->flags = 0;
+
+ return 0;
+}
+
+static int vin_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ struct vin_line *line = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt format;
+
+ if (fse->index != 0)
+ return -EINVAL;
+
+ format.code = fse->code;
+ format.width = 1;
+ format.height = 1;
+ vin_try_format(line, state, fse->pad, &format, fse->which);
+ fse->min_width = format.width;
+ fse->min_height = format.height;
+
+ if (format.code != fse->code)
+ return -EINVAL;
+
+ format.code = fse->code;
+ format.width = -1;
+ format.height = -1;
+ vin_try_format(line, state, fse->pad, &format, fse->which);
+ fse->max_width = format.width;
+ fse->max_height = format.height;
+
+ return 0;
+}
+
+static int vin_get_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct vin_line *line = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *format;
+
+ format = __vin_get_format(line, state, fmt->pad, fmt->which);
+ if (!format)
+ return -EINVAL;
+
+ fmt->format = *format;
+
+ return 0;
+}
+
+static int vin_set_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct vin_line *line = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *format;
+
+ format = __vin_get_format(line, state, fmt->pad, fmt->which);
+ if (!format)
+ return -EINVAL;
+
+ mutex_lock(&line->stream_lock);
+ if (line->stream_count) {
+ fmt->format = *format;
+ mutex_unlock(&line->stream_lock);
+ goto out;
+ } else {
+ vin_try_format(line, state, fmt->pad, &fmt->format, fmt->which);
+ *format = fmt->format;
+ }
+ mutex_unlock(&line->stream_lock);
+
+ if (fmt->pad == STF_VIN_PAD_SINK) {
+ /* Propagate the format from sink to source */
+ format = __vin_get_format(line, state, STF_VIN_PAD_SRC,
+ fmt->which);
+
+ *format = fmt->format;
+ vin_try_format(line, state, STF_VIN_PAD_SRC, format,
+ fmt->which);
+ }
+
+out:
+ return 0;
+}
+
+static int vin_init_formats(struct v4l2_subdev *sd,
+ struct v4l2_subdev_fh *fh)
+{
+ struct v4l2_subdev_format format = {
+ .pad = STF_VIN_PAD_SINK,
+ .which = fh ?
+ V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
+ .format = {
+ .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
+ .width = 1920,
+ .height = 1080
+ }
+ };
+
+ return vin_set_format(sd, fh ? fh->state : NULL, &format);
+}
+
+static void vin_output_init_addrs(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+ struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
+ dma_addr_t ping_addr;
+ dma_addr_t y_addr, uv_addr;
+
+ output->active_buf = 0;
+
+ if (output->buf[0]) {
+ ping_addr = output->buf[0]->addr[0];
+ y_addr = output->buf[0]->addr[0];
+ uv_addr = output->buf[0]->addr[1];
+ } else {
+ return;
+ }
+
+ switch (vin_map_isp_line(line->id)) {
+ case STF_ISP_LINE_SRC:
+ stf_vin_isp_set_yuv_addr(vin_dev, y_addr, uv_addr);
+ break;
+ default:
+ if (line->id == VIN_LINE_WR) {
+ stf_vin_wr_set_ping_addr(vin_dev, ping_addr);
+ stf_vin_wr_set_pong_addr(vin_dev, ping_addr);
+ }
+ break;
+ }
+}
+
+static void vin_init_outputs(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+
+ output->state = VIN_OUTPUT_OFF;
+ output->buf[0] = NULL;
+ output->buf[1] = NULL;
+ output->active_buf = 0;
+ INIT_LIST_HEAD(&output->pending_bufs);
+ INIT_LIST_HEAD(&output->ready_bufs);
+}
+
+static void vin_buf_add_ready(struct vin_output *output,
+ struct stfcamss_buffer *buffer)
+{
+ INIT_LIST_HEAD(&buffer->queue);
+ list_add_tail(&buffer->queue, &output->ready_bufs);
+}
+
+static struct stfcamss_buffer *vin_buf_get_ready(struct vin_output *output)
+{
+ struct stfcamss_buffer *buffer = NULL;
+
+ if (!list_empty(&output->ready_bufs)) {
+ buffer = list_first_entry(&output->ready_bufs,
+ struct stfcamss_buffer,
+ queue);
+ list_del(&buffer->queue);
+ }
+
+ return buffer;
+}
+
+static void vin_buf_add_pending(struct vin_output *output,
+ struct stfcamss_buffer *buffer)
+{
+ INIT_LIST_HEAD(&buffer->queue);
+ list_add_tail(&buffer->queue, &output->pending_bufs);
+}
+
+static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output)
+{
+ struct stfcamss_buffer *buffer = NULL;
+
+ if (!list_empty(&output->pending_bufs)) {
+ buffer = list_first_entry(&output->pending_bufs,
+ struct stfcamss_buffer,
+ queue);
+ list_del(&buffer->queue);
+ }
+
+ return buffer;
+}
+
+static void vin_buf_update_on_last(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+
+ switch (output->state) {
+ case VIN_OUTPUT_CONTINUOUS:
+ output->state = VIN_OUTPUT_SINGLE;
+ output->active_buf = !output->active_buf;
+ break;
+ case VIN_OUTPUT_SINGLE:
+ output->state = VIN_OUTPUT_STOPPING;
+ break;
+ default:
+ break;
+ }
+}
+
+static void vin_buf_update_on_next(struct vin_line *line)
+{
+ struct vin_output *output = &line->output;
+
+ switch (output->state) {
+ case VIN_OUTPUT_CONTINUOUS:
+ output->active_buf = !output->active_buf;
+ break;
+ case VIN_OUTPUT_SINGLE:
+ default:
+ break;
+ }
+}
+
+static void vin_buf_update_on_new(struct vin_line *line,
+ struct vin_output *output,
+ struct stfcamss_buffer *new_buf)
+{
+ switch (output->state) {
+ case VIN_OUTPUT_SINGLE:
+ vin_buf_add_pending(output, new_buf);
+ break;
+ case VIN_OUTPUT_IDLE:
+ if (!output->buf[0]) {
+ output->buf[0] = new_buf;
+ vin_output_init_addrs(line);
+ output->state = VIN_OUTPUT_SINGLE;
+ } else {
+ vin_buf_add_pending(output, new_buf);
+ }
+ break;
+ case VIN_OUTPUT_STOPPING:
+ if (output->last_buffer) {
+ output->buf[output->active_buf] = output->last_buffer;
+ output->last_buffer = NULL;
+ }
+
+ output->state = VIN_OUTPUT_SINGLE;
+ vin_buf_add_pending(output, new_buf);
+ break;
+ case VIN_OUTPUT_CONTINUOUS:
+ default:
+ vin_buf_add_pending(output, new_buf);
+ break;
+ }
+}
+
+static void vin_buf_flush(struct vin_output *output,
+ enum vb2_buffer_state state)
+{
+ struct stfcamss_buffer *buf;
+ struct stfcamss_buffer *t;
+
+ list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+ list_del(&buf->queue);
+ }
+ list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+ list_del(&buf->queue);
+ }
+}
+
+static void vin_buffer_done(struct vin_line *line)
+{
+ struct stfcamss_buffer *ready_buf;
+ struct vin_output *output = &line->output;
+ unsigned long flags;
+ u64 ts = ktime_get_ns();
+
+ if (output->state == VIN_OUTPUT_OFF ||
+ output->state == VIN_OUTPUT_RESERVED)
+ return;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ while ((ready_buf = vin_buf_get_ready(output))) {
+ ready_buf->vb.vb2_buf.timestamp = ts;
+ ready_buf->vb.sequence = output->sequence++;
+
+ vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+ }
+
+ spin_unlock_irqrestore(&line->output_lock, flags);
+}
+
+static void vin_change_buffer(struct vin_line *line)
+{
+ struct stfcamss_buffer *ready_buf;
+ struct vin_output *output = &line->output;
+ struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
+ dma_addr_t *new_addr;
+ unsigned long flags;
+ u32 active_index;
+
+ if (output->state == VIN_OUTPUT_OFF ||
+ output->state == VIN_OUTPUT_STOPPING ||
+ output->state == VIN_OUTPUT_RESERVED ||
+ output->state == VIN_OUTPUT_IDLE)
+ return;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ active_index = output->active_buf;
+
+ ready_buf = output->buf[active_index];
+ if (!ready_buf) {
+ dev_warn(vin_dev->stfcamss->dev, "Missing ready buf %d %d!\n",
+ active_index, output->state);
+ active_index = !active_index;
+ ready_buf = output->buf[active_index];
+ if (!ready_buf) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Missing ready buf2 %d %d!\n",
+ active_index, output->state);
+ goto out_unlock;
+ }
+ }
+
+ /* Get next buffer */
+ output->buf[active_index] = vin_buf_get_pending(output);
+ if (!output->buf[active_index]) {
+ /* No next buffer - set same address */
+ new_addr = ready_buf->addr;
+ vin_buf_update_on_last(line);
+ } else {
+ new_addr = output->buf[active_index]->addr;
+ vin_buf_update_on_next(line);
+ }
+
+ if (output->state == VIN_OUTPUT_STOPPING) {
+ output->last_buffer = ready_buf;
+ } else {
+ switch (vin_map_isp_line(line->id)) {
+ case STF_ISP_LINE_SRC:
+ stf_vin_isp_set_yuv_addr(vin_dev,
+ new_addr[0],
+ new_addr[1]);
+ break;
+ default:
+ if (line->id == VIN_LINE_WR) {
+ stf_vin_wr_set_ping_addr(vin_dev, new_addr[0]);
+ stf_vin_wr_set_pong_addr(vin_dev, new_addr[0]);
+ }
+ break;
+ }
+
+ vin_buf_add_ready(output, ready_buf);
+ }
+
+ spin_unlock_irqrestore(&line->output_lock, flags);
+ return;
+
+out_unlock:
+ spin_unlock_irqrestore(&line->output_lock, flags);
+}
+
+static int vin_queue_buffer(struct stfcamss_video *vid,
+ struct stfcamss_buffer *buf)
+{
+ struct vin_line *line = container_of(vid, struct vin_line, video_out);
+ struct vin_output *output;
+ unsigned long flags;
+
+ output = &line->output;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ vin_buf_update_on_new(line, output, buf);
+
+ spin_unlock_irqrestore(&line->output_lock, flags);
+
+ return 0;
+}
+
+static int vin_flush_buffers(struct stfcamss_video *vid,
+ enum vb2_buffer_state state)
+{
+ struct vin_line *line = container_of(vid, struct vin_line, video_out);
+ struct vin_output *output = &line->output;
+ unsigned long flags;
+
+ spin_lock_irqsave(&line->output_lock, flags);
+
+ vin_buf_flush(output, state);
+ if (output->buf[0])
+ vb2_buffer_done(&output->buf[0]->vb.vb2_buf, state);
+
+ if (output->buf[1])
+ vb2_buffer_done(&output->buf[1]->vb.vb2_buf, state);
+
+ if (output->last_buffer) {
+ vb2_buffer_done(&output->last_buffer->vb.vb2_buf, state);
+ output->last_buffer = NULL;
+ }
+ output->buf[0] = NULL;
+ output->buf[1] = NULL;
+
+ spin_unlock_irqrestore(&line->output_lock, flags);
+ return 0;
+}
+
+static int vin_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ if (flags & MEDIA_LNK_FL_ENABLED)
+ if (media_pad_remote_pad_first(local))
+ return -EBUSY;
+ return 0;
+}
+
+static const struct v4l2_subdev_video_ops vin_video_ops = {
+ .s_stream = vin_set_stream,
+};
+
+static const struct v4l2_subdev_pad_ops vin_pad_ops = {
+ .enum_mbus_code = vin_enum_mbus_code,
+ .enum_frame_size = vin_enum_frame_size,
+ .get_fmt = vin_get_format,
+ .set_fmt = vin_set_format,
+};
+
+static const struct v4l2_subdev_ops vin_v4l2_ops = {
+ .video = &vin_video_ops,
+ .pad = &vin_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops vin_v4l2_internal_ops = {
+ .open = vin_init_formats,
+};
+
+static const struct stfcamss_video_ops stfcamss_vin_video_ops = {
+ .queue_buffer = vin_queue_buffer,
+ .flush_buffers = vin_flush_buffers,
+};
+
+static const struct media_entity_operations vin_media_ops = {
+ .link_setup = vin_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev)
+{
+ struct v4l2_subdev *sd;
+ struct stfcamss_video *video_out;
+ struct media_pad *pads;
+ int ret;
+ int i;
+
+ for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
+ char name[32];
+ char *sub_name = vin_get_line_subdevname(i);
+
+ sd = &vin_dev->line[i].subdev;
+ pads = vin_dev->line[i].pads;
+ video_out = &vin_dev->line[i].video_out;
+ video_out->id = i;
+
+ vin_init_outputs(&vin_dev->line[i]);
+
+ v4l2_subdev_init(sd, &vin_v4l2_ops);
+ sd->internal_ops = &vin_v4l2_internal_ops;
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d_%s",
+ STF_VIN_NAME, 0, sub_name);
+ v4l2_set_subdevdata(sd, &vin_dev->line[i]);
+
+ ret = vin_init_formats(sd, NULL);
+ if (ret < 0) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Failed to init format: %d\n", ret);
+ goto err_init;
+ }
+
+ pads[STF_VIN_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ pads[STF_VIN_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
+
+ sd->entity.function =
+ MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ sd->entity.ops = &vin_media_ops;
+ ret = media_entity_pads_init(&sd->entity,
+ STF_VIN_PADS_NUM, pads);
+ if (ret < 0) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Failed to init media entity: %d\n",
+ ret);
+ goto err_init;
+ }
+
+ ret = v4l2_device_register_subdev(v4l2_dev, sd);
+ if (ret < 0) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Failed to register subdev: %d\n", ret);
+ goto err_reg_subdev;
+ }
+
+ video_out->ops = &stfcamss_vin_video_ops;
+ video_out->bpl_alignment = 16 * 8;
+
+ snprintf(name, ARRAY_SIZE(name), "%s_%s%d",
+ sd->name, "video", i);
+ ret = stf_video_register(video_out, v4l2_dev, name);
+ if (ret < 0) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Failed to register video node: %d\n", ret);
+ goto err_vid_reg;
+ }
+
+ ret = media_create_pad_link(
+ &sd->entity, STF_VIN_PAD_SRC,
+ &video_out->vdev.entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+ if (ret < 0) {
+ dev_err(vin_dev->stfcamss->dev,
+ "Failed to link %s->%s entities: %d\n",
+ sd->entity.name, video_out->vdev.entity.name,
+ ret);
+ goto err_create_link;
+ }
+ }
+
+ return 0;
+
+err_create_link:
+ stf_video_unregister(video_out);
+err_vid_reg:
+ v4l2_device_unregister_subdev(sd);
+err_reg_subdev:
+ media_entity_cleanup(&sd->entity);
+err_init:
+ for (i--; i >= 0; i--) {
+ sd = &vin_dev->line[i].subdev;
+ video_out = &vin_dev->line[i].video_out;
+
+ stf_video_unregister(video_out);
+ v4l2_device_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+ }
+ return ret;
+}
+
+int stf_vin_unregister(struct stf_vin_dev *vin_dev)
+{
+ struct v4l2_subdev *sd;
+ struct stfcamss_video *video_out;
+ int i;
+
+ for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++)
+ mutex_destroy(&vin_dev->dummy_buffer[i].stream_lock);
+
+ for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
+ sd = &vin_dev->line[i].subdev;
+ video_out = &vin_dev->line[i].video_out;
+
+ stf_video_unregister(video_out);
+ v4l2_device_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+ mutex_destroy(&vin_dev->line[i].stream_lock);
+ }
+ return 0;
+}
diff --git a/drivers/media/platform/starfive/camss/stf_vin.h b/drivers/media/platform/starfive/camss/stf_vin.h
new file mode 100644
index 000000000000..28572eb6abe4
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_vin.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * stf_vin.h
+ *
+ * StarFive Camera Subsystem - VIN Module
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_VIN_H
+#define STF_VIN_H
+
+#include <linux/interrupt.h>
+#include <linux/spinlock_types.h>
+#include <media/v4l2-subdev.h>
+
+#include "stf_video.h"
+
+#define SYSCONSAIF_SYSCFG(x) (x)
+
+/* syscon offset 0 */
+#define U0_VIN_CNFG_AXI_DVP_EN BIT(2)
+
+/* syscon offset 20 */
+#define U0_VIN_CHANNEL_SEL_MASK GENMASK(3, 0)
+#define U0_VIN_AXIWR0_EN BIT(4)
+#define CHANNEL(x) ((x) << 0)
+
+/* syscon offset 32 */
+#define U0_VIN_INTR_CLEAN BIT(0)
+#define U0_VIN_INTR_M BIT(1)
+#define U0_VIN_PIX_CNT_END_MASK GENMASK(12, 2)
+#define U0_VIN_PIX_CT_MASK GENMASK(14, 13)
+#define U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS GENMASK(16, 15)
+
+#define PIX_CNT_END(x) ((x) << 2)
+#define PIX_CT(x) ((x) << 13)
+#define PIXEL_HEIGH_BIT_SEL(x) ((x) << 15)
+
+/* syscon offset 36 */
+#define U0_VIN_CNFG_DVP_HS_POS BIT(1)
+#define U0_VIN_CNFG_DVP_SWAP_EN BIT(2)
+#define U0_VIN_CNFG_DVP_VS_POS BIT(3)
+#define U0_VIN_CNFG_GEN_EN_AXIRD BIT(4)
+#define U0_VIN_CNFG_ISP_DVP_EN0 BIT(5)
+#define U0_VIN_MIPI_BYTE_EN_ISP0(n) ((n) << 6)
+#define U0_VIN_MIPI_CHANNEL_SEL0(n) ((n) << 8)
+#define U0_VIN_P_I_MIPI_HAEDER_EN0(n) ((n) << 12)
+#define U0_VIN_PIX_NUM(n) ((n) << 13)
+#define U0_VIN_MIPI_BYTE_EN_ISP0_MASK GENMASK(7, 6)
+#define U0_VIN_MIPI_CHANNEL_SEL0_MASK GENMASK(11, 8)
+#define U0_VIN_P_I_MIPI_HAEDER_EN0_MASK BIT(12)
+#define U0_VIN_PIX_NUM_MASK GENMASK(16, 13)
+
+#define STF_VIN_PAD_SINK 0
+#define STF_VIN_PAD_SRC 1
+#define STF_VIN_PADS_NUM 2
+
+#define ISP_DUMMY_BUFFER_NUMS STF_ISP_PAD_MAX
+#define VIN_DUMMY_BUFFER_NUMS 1
+
+enum {
+ STF_DUMMY_VIN,
+ STF_DUMMY_ISP,
+ STF_DUMMY_MODULE_NUMS,
+};
+
+enum link {
+ LINK_ERROR = -1,
+ LINK_DVP_TO_WR,
+ LINK_DVP_TO_ISP,
+ LINK_CSI_TO_WR,
+ LINK_CSI_TO_ISP,
+};
+
+struct vin_format {
+ u32 code;
+ u8 bpp;
+};
+
+struct vin_format_table {
+ const struct vin_format *fmts;
+ int nfmts;
+};
+
+enum vin_output_state {
+ VIN_OUTPUT_OFF,
+ VIN_OUTPUT_RESERVED,
+ VIN_OUTPUT_SINGLE,
+ VIN_OUTPUT_CONTINUOUS,
+ VIN_OUTPUT_IDLE,
+ VIN_OUTPUT_STOPPING
+};
+
+struct vin_output {
+ int active_buf;
+ struct stfcamss_buffer *buf[2];
+ struct stfcamss_buffer *last_buffer;
+ struct list_head pending_bufs;
+ struct list_head ready_bufs;
+ enum vin_output_state state;
+ unsigned int sequence;
+ unsigned int frame_skip;
+};
+
+/* The vin output lines */
+enum vin_line_id {
+ VIN_LINE_NONE = -1,
+ VIN_LINE_WR = 0,
+ VIN_LINE_ISP,
+ VIN_LINE_MAX,
+};
+
+struct vin_line {
+ enum vin_line_id id;
+ struct v4l2_subdev subdev;
+ struct media_pad pads[STF_VIN_PADS_NUM];
+ struct v4l2_mbus_framefmt fmt[STF_VIN_PADS_NUM];
+ struct stfcamss_video video_out;
+ struct mutex stream_lock; /* serialize stream control */
+ int stream_count;
+ struct vin_output output; /* pipeline and stream states */
+ spinlock_t output_lock;
+ const struct vin_format *formats;
+ unsigned int nformats;
+};
+
+struct vin_dummy_buffer {
+ dma_addr_t paddr[3];
+ void *vaddr;
+ u32 buffer_size;
+ u32 width;
+ u32 height;
+ u32 mcode;
+};
+
+struct dummy_buffer {
+ struct vin_dummy_buffer *buffer;
+ u32 nums;
+ struct mutex stream_lock; /* protects buffer data */
+ int stream_count;
+ atomic_t frame_skip;
+};
+
+struct vin_isr_ops {
+ void (*isr_buffer_done)(struct vin_line *line);
+ void (*isr_change_buffer)(struct vin_line *line);
+};
+
+struct stf_vin_dev {
+ struct stfcamss *stfcamss;
+ struct vin_line line[VIN_LINE_MAX];
+ struct dummy_buffer dummy_buffer[STF_DUMMY_MODULE_NUMS];
+ struct vin_isr_ops *isr_ops;
+ atomic_t ref_count;
+};
+
+int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev);
+int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link);
+void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable);
+void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
+void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
+void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
+ dma_addr_t y_addr, dma_addr_t uv_addr);
+irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv);
+irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv);
+irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv);
+int stf_vin_subdev_init(struct stfcamss *stfcamss);
+int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev);
+int stf_vin_unregister(struct stf_vin_dev *vin_dev);
+enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def);
+
+#endif /* STF_VIN_H */
diff --git a/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
new file mode 100644
index 000000000000..7bd3265128d0
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
@@ -0,0 +1,192 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_vin_hw_ops.c
+ *
+ * Register interface file for StarFive VIN module driver
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+#include "stf_camss.h"
+
+static void vin_intr_clear(struct stfcamss *stfcamss)
+{
+ stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_CLEAN);
+ stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_CLEAN);
+}
+
+irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv)
+{
+ struct stf_vin_dev *vin_dev = priv;
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[STF_DUMMY_VIN];
+
+ if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
+ vin_dev->isr_ops->isr_change_buffer(&vin_dev->line[VIN_LINE_WR]);
+ vin_dev->isr_ops->isr_buffer_done(&vin_dev->line[VIN_LINE_WR]);
+ }
+
+ vin_intr_clear(stfcamss);
+
+ return IRQ_HANDLED;
+}
+
+irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv)
+{
+ struct stf_vin_dev *vin_dev = priv;
+ u32 int_status;
+
+ int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
+
+ if (int_status & ISPC_INTS) {
+ if ((int_status & ISPC_ENUO))
+ vin_dev->isr_ops->isr_buffer_done(
+ &vin_dev->line[VIN_LINE_ISP]);
+
+ /* clear interrupt */
+ stf_isp_reg_write(vin_dev->stfcamss,
+ ISP_REG_ISP_CTRL_0,
+ (int_status & ~EN_INT_ALL) |
+ EN_INT_ISP_DONE |
+ EN_INT_CSI_DONE |
+ EN_INT_SC_DONE);
+ }
+
+ return IRQ_HANDLED;
+}
+
+irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv)
+{
+ struct stf_vin_dev *vin_dev = priv;
+ struct stf_isp_dev *isp_dev;
+ u32 int_status;
+
+ isp_dev = &vin_dev->stfcamss->isp_dev;
+
+ int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
+ if (int_status & ISPC_SCFEINT) {
+ struct dummy_buffer *dummy_buffer =
+ &vin_dev->dummy_buffer[STF_DUMMY_ISP];
+
+ if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
+ if ((int_status & ISPC_ENUO))
+ vin_dev->isr_ops->isr_change_buffer(
+ &vin_dev->line[VIN_LINE_ISP]);
+ }
+
+ stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_CSIINTS,
+ CSI_INTS_MASK, CSI_INTS(0x3));
+ stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_IESHD,
+ SHAD_UP_M | SHAD_UP_EN, 0x3);
+
+ /* clear interrupt */
+ stf_isp_reg_write(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0,
+ (int_status & ~EN_INT_ALL) | EN_INT_LINE_INT);
+ }
+
+ return IRQ_HANDLED;
+}
+
+int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev)
+{
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+
+ stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(20),
+ U0_VIN_AXIWR0_EN);
+
+ return 0;
+}
+
+int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link)
+{
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+ u32 val;
+
+ switch (link) {
+ case LINK_CSI_TO_WR:
+ val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(20));
+ val &= ~U0_VIN_CHANNEL_SEL_MASK;
+ val |= CHANNEL(0);
+ stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(20), val);
+
+ val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(28));
+ val &= ~U0_VIN_PIX_CT_MASK;
+ val |= PIX_CT(1);
+
+ val &= ~U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS;
+ val |= PIXEL_HEIGH_BIT_SEL(0);
+
+ val &= ~U0_VIN_PIX_CNT_END_MASK;
+ val |= PIX_CNT_END(IMAGE_MAX_WIDTH / 4 - 1);
+
+ stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(28), val);
+ break;
+ case LINK_CSI_TO_ISP:
+ val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(36));
+ val &= ~U0_VIN_MIPI_BYTE_EN_ISP0_MASK;
+ val |= U0_VIN_MIPI_BYTE_EN_ISP0(0);
+
+ val &= ~U0_VIN_MIPI_CHANNEL_SEL0_MASK;
+ val |= U0_VIN_MIPI_CHANNEL_SEL0(0);
+
+ val &= ~U0_VIN_PIX_NUM_MASK;
+ val |= U0_VIN_PIX_NUM(0);
+
+ val &= ~U0_VIN_P_I_MIPI_HAEDER_EN0_MASK;
+ val |= U0_VIN_P_I_MIPI_HAEDER_EN0(1);
+
+ stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(36), val);
+ break;
+ case LINK_DVP_TO_WR:
+ case LINK_DVP_TO_ISP:
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable)
+{
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+
+ if (enable) {
+ stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_M);
+ } else {
+ /* clear vin interrupt */
+ stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_CLEAN);
+ stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_CLEAN);
+ stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
+ U0_VIN_INTR_M);
+ }
+}
+
+void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
+{
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+
+ /* set the start address */
+ stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(32), (long)addr);
+}
+
+void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
+{
+ struct stfcamss *stfcamss = vin_dev->stfcamss;
+
+ /* set the start address */
+ stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(24), (long)addr);
+}
+
+void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
+ dma_addr_t y_addr, dma_addr_t uv_addr)
+{
+ stf_isp_reg_write(vin_dev->stfcamss,
+ ISP_REG_Y_PLANE_START_ADDR, y_addr);
+ stf_isp_reg_write(vin_dev->stfcamss,
+ ISP_REG_UV_PLANE_START_ADDR, uv_addr);
+}
--
2.34.1
Add ISP driver for StarFive Camera Subsystem.
Signed-off-by: Jack Zhu <[email protected]>
---
.../media/platform/starfive/camss/Makefile | 2 +
.../media/platform/starfive/camss/stf_camss.c | 76 ++-
.../media/platform/starfive/camss/stf_camss.h | 3 +
.../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
.../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
.../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
6 files changed, 1544 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
index eb457917a914..cdf57e8c9546 100644
--- a/drivers/media/platform/starfive/camss/Makefile
+++ b/drivers/media/platform/starfive/camss/Makefile
@@ -5,6 +5,8 @@
starfive-camss-objs += \
stf_camss.o \
+ stf_isp.o \
+ stf_isp_hw_ops.o \
stf_video.o
obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
index dc2b5dba7bd4..6f56b45f57db 100644
--- a/drivers/media/platform/starfive/camss/stf_camss.c
+++ b/drivers/media/platform/starfive/camss/stf_camss.c
@@ -115,12 +115,65 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
return ret;
}
+/*
+ * stfcamss_init_subdevices - Initialize subdev structures and resources
+ * @stfcamss: STFCAMSS device
+ *
+ * Return 0 on success or a negative error code on failure
+ */
+static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
+{
+ int ret;
+
+ ret = stf_isp_subdev_init(stfcamss);
+ if (ret < 0) {
+ dev_err(stfcamss->dev, "Failed to init isp subdev: %d\n", ret);
+ return ret;
+ }
+
+ return ret;
+}
+
+static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
+{
+ int ret;
+ struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
+
+ ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
+ if (ret < 0) {
+ dev_err(stfcamss->dev,
+ "Failed to register stf isp%d entity: %d\n", 0, ret);
+ return ret;
+ }
+
+ return ret;
+}
+
+static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
+{
+ stf_isp_unregister(&stfcamss->isp_dev);
+}
+
static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
struct v4l2_subdev *subdev,
struct v4l2_async_subdev *asd)
{
+ struct stfcamss *stfcamss =
+ container_of(async, struct stfcamss, notifier);
+ struct stfcamss_async_subdev *csd =
+ container_of(asd, struct stfcamss_async_subdev, asd);
+ enum stf_port_num port = csd->port;
+ struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
struct media_pad *pad[STF_PADS_NUM];
- unsigned int i, pad_num = 0;
+ unsigned int i, pad_num;
+
+ if (port == STF_PORT_CSI2RX) {
+ pad[0] = &isp_dev->pads[STF_PAD_SINK];
+ pad_num = 1;
+ } else if (port == STF_PORT_DVP) {
+ dev_err(stfcamss->dev, "Not support DVP sensor\n");
+ return -EPERM;
+ }
for (i = 0; i < pad_num; ++i) {
int ret;
@@ -223,12 +276,18 @@ static int stfcamss_probe(struct platform_device *pdev)
goto err_cleanup_notifier;
}
+ ret = stfcamss_init_subdevices(stfcamss);
+ if (ret < 0) {
+ dev_err(dev, "Failed to init subdevice: %d\n", ret);
+ goto err_cleanup_notifier;
+ }
+
stfcamss_mc_init(pdev, stfcamss);
ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
if (ret < 0) {
dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
- goto err_cleanup_notifier;
+ goto err_cleanup_media_device;
}
ret = media_device_register(&stfcamss->media_dev);
@@ -237,6 +296,12 @@ static int stfcamss_probe(struct platform_device *pdev)
goto err_unregister_device;
}
+ ret = stfcamss_register_subdevices(stfcamss);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register subdevice: %d\n", ret);
+ goto err_unregister_media_dev;
+ }
+
pm_runtime_enable(dev);
stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
@@ -244,15 +309,19 @@ static int stfcamss_probe(struct platform_device *pdev)
if (ret) {
dev_err(dev, "Failed to register async subdev nodes: %d\n",
ret);
- goto err_unregister_media_dev;
+ goto err_unregister_subdevs;
}
return 0;
+err_unregister_subdevs:
+ stfcamss_unregister_subdevices(stfcamss);
err_unregister_media_dev:
media_device_unregister(&stfcamss->media_dev);
err_unregister_device:
v4l2_device_unregister(&stfcamss->v4l2_dev);
+err_cleanup_media_device:
+ media_device_cleanup(&stfcamss->media_dev);
err_cleanup_notifier:
v4l2_async_nf_cleanup(&stfcamss->notifier);
return ret;
@@ -268,6 +337,7 @@ static int stfcamss_remove(struct platform_device *pdev)
{
struct stfcamss *stfcamss = platform_get_drvdata(pdev);
+ stfcamss_unregister_subdevices(stfcamss);
v4l2_device_unregister(&stfcamss->v4l2_dev);
media_device_cleanup(&stfcamss->media_dev);
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
index 15c4f34b9054..9482081288fa 100644
--- a/drivers/media/platform/starfive/camss/stf_camss.h
+++ b/drivers/media/platform/starfive/camss/stf_camss.h
@@ -18,6 +18,8 @@
#include <media/v4l2-async.h>
#include <media/v4l2-device.h>
+#include "stf_isp.h"
+
#define STF_DVP_NAME "stf_dvp"
#define STF_CSI_NAME "cdns_csi2rx"
#define STF_ISP_NAME "stf_isp"
@@ -65,6 +67,7 @@ struct stfcamss {
struct media_device media_dev;
struct media_pipeline pipe;
struct device *dev;
+ struct stf_isp_dev isp_dev;
struct v4l2_async_notifier notifier;
void __iomem *syscon_base;
void __iomem *isp_base;
diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
new file mode 100644
index 000000000000..933a583b398c
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_isp.c
@@ -0,0 +1,519 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_isp.c
+ *
+ * StarFive Camera Subsystem - ISP Module
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+#include <linux/firmware.h>
+#include <media/v4l2-event.h>
+
+#include "stf_camss.h"
+
+#define SINK_FORMATS_INDEX 0
+#define UO_FORMATS_INDEX 1
+
+static int isp_set_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel);
+
+static const struct isp_format isp_formats_sink[] = {
+ { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
+ { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
+ { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
+ { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
+};
+
+static const struct isp_format isp_formats_uo[] = {
+ { MEDIA_BUS_FMT_Y12_1X12, 8 },
+};
+
+static const struct isp_format_table isp_formats_st7110[] = {
+ { isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
+ { isp_formats_uo, ARRAY_SIZE(isp_formats_uo) },
+};
+
+int stf_isp_subdev_init(struct stfcamss *stfcamss)
+{
+ struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
+
+ isp_dev->stfcamss = stfcamss;
+ isp_dev->formats = isp_formats_st7110;
+ isp_dev->nformats = ARRAY_SIZE(isp_formats_st7110);
+
+ mutex_init(&isp_dev->stream_lock);
+ return 0;
+}
+
+static struct v4l2_mbus_framefmt *
+__isp_get_format(struct stf_isp_dev *isp_dev,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ enum v4l2_subdev_format_whence which)
+{
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&isp_dev->subdev, state, pad);
+
+ return &isp_dev->fmt[pad];
+}
+
+static int isp_set_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ int ret = 0;
+ struct v4l2_mbus_framefmt *fmt;
+ struct v4l2_event src_ch = { 0 };
+
+ fmt = __isp_get_format(isp_dev, NULL, STF_ISP_PAD_SINK,
+ V4L2_SUBDEV_FORMAT_ACTIVE);
+ mutex_lock(&isp_dev->stream_lock);
+ if (enable) {
+ if (isp_dev->stream_count == 0) {
+ stf_isp_clk_enable(isp_dev);
+ stf_isp_reset(isp_dev);
+ stf_isp_init_cfg(isp_dev);
+ stf_isp_settings(isp_dev, isp_dev->rect, fmt->code);
+ stf_isp_stream_set(isp_dev);
+ }
+ isp_dev->stream_count++;
+ } else {
+ if (isp_dev->stream_count == 0)
+ goto exit;
+
+ if (isp_dev->stream_count == 1)
+ stf_isp_clk_disable(isp_dev);
+
+ isp_dev->stream_count--;
+ }
+ src_ch.type = V4L2_EVENT_SOURCE_CHANGE,
+ src_ch.u.src_change.changes = isp_dev->stream_count,
+
+ v4l2_subdev_notify_event(sd, &src_ch);
+exit:
+ mutex_unlock(&isp_dev->stream_lock);
+
+ return ret;
+}
+
+static void isp_try_format(struct stf_isp_dev *isp_dev,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ struct v4l2_mbus_framefmt *fmt,
+ enum v4l2_subdev_format_whence which)
+{
+ const struct isp_format_table *formats;
+ struct stf_isp_crop *rect;
+ unsigned int i;
+
+ if (pad == STF_ISP_PAD_SINK) {
+ /* Set format on sink pad */
+ formats = &isp_dev->formats[SINK_FORMATS_INDEX];
+ rect = &isp_dev->rect[SINK_FORMATS_INDEX];
+ } else if (pad == STF_ISP_PAD_SRC) {
+ formats = &isp_dev->formats[UO_FORMATS_INDEX];
+ rect = &isp_dev->rect[UO_FORMATS_INDEX];
+ }
+
+ fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
+ STFCAMSS_FRAME_MAX_WIDTH);
+ fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
+ STFCAMSS_FRAME_MAX_HEIGHT);
+ fmt->height &= ~0x1;
+ fmt->field = V4L2_FIELD_NONE;
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+ fmt->flags = 0;
+
+ for (i = 0; i < formats->nfmts; i++) {
+ if (fmt->code == formats->fmts[i].code)
+ break;
+ }
+
+ if (i >= formats->nfmts) {
+ fmt->code = formats->fmts[0].code;
+ rect->bpp = formats->fmts[0].bpp;
+ } else {
+ rect->bpp = formats->fmts[i].bpp;
+ }
+}
+
+static int isp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ const struct isp_format_table *formats;
+
+ if (code->index >= isp_dev->nformats)
+ return -EINVAL;
+ if (code->pad == STF_ISP_PAD_SINK) {
+ formats = &isp_dev->formats[SINK_FORMATS_INDEX];
+ code->code = formats->fmts[code->index].code;
+ } else {
+ struct v4l2_mbus_framefmt *sink_fmt;
+
+ sink_fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK,
+ code->which);
+
+ code->code = sink_fmt->code;
+ if (!code->code)
+ return -EINVAL;
+ }
+ code->flags = 0;
+
+ return 0;
+}
+
+static int isp_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt format;
+
+ if (fse->index != 0)
+ return -EINVAL;
+
+ format.code = fse->code;
+ format.width = 1;
+ format.height = 1;
+ isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
+ fse->min_width = format.width;
+ fse->min_height = format.height;
+
+ if (format.code != fse->code)
+ return -EINVAL;
+
+ format.code = fse->code;
+ format.width = -1;
+ format.height = -1;
+ isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
+ fse->max_width = format.width;
+ fse->max_height = format.height;
+
+ return 0;
+}
+
+static int isp_get_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *format;
+
+ format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
+ if (!format)
+ return -EINVAL;
+
+ fmt->format = *format;
+
+ return 0;
+}
+
+static int isp_set_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *format;
+
+ format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
+ if (!format)
+ return -EINVAL;
+
+ mutex_lock(&isp_dev->stream_lock);
+
+ isp_try_format(isp_dev, state, fmt->pad, &fmt->format, fmt->which);
+ *format = fmt->format;
+
+ mutex_unlock(&isp_dev->stream_lock);
+
+ /* Propagate to in crop */
+ if (fmt->pad == STF_ISP_PAD_SINK) {
+ struct v4l2_subdev_selection sel = { 0 };
+ int ret;
+
+ /* Reset sink pad compose selection */
+ sel.which = fmt->which;
+ sel.pad = STF_ISP_PAD_SINK;
+ sel.target = V4L2_SEL_TGT_CROP;
+ sel.r.width = fmt->format.width;
+ sel.r.height = fmt->format.height;
+ ret = isp_set_selection(sd, state, &sel);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static struct v4l2_rect *
+__isp_get_crop(struct stf_isp_dev *isp_dev,
+ struct v4l2_subdev_state *state,
+ unsigned int pad,
+ enum v4l2_subdev_format_whence which)
+{
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_crop(&isp_dev->subdev, state,
+ STF_ISP_PAD_SINK);
+
+ return &isp_dev->rect[pad].rect;
+}
+
+static void isp_try_crop(struct stf_isp_dev *isp_dev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_rect *rect,
+ enum v4l2_subdev_format_whence which)
+{
+ struct v4l2_mbus_framefmt *fmt;
+
+ fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK, which);
+
+ if (rect->width > fmt->width)
+ rect->width = fmt->width;
+
+ if (rect->width + rect->left > fmt->width)
+ rect->left = fmt->width - rect->width;
+
+ if (rect->height > fmt->height)
+ rect->height = fmt->height;
+
+ if (rect->height + rect->top > fmt->height)
+ rect->top = fmt->height - rect->height;
+
+ if (rect->width < STFCAMSS_FRAME_MIN_WIDTH) {
+ rect->left = 0;
+ rect->width = STFCAMSS_FRAME_MAX_WIDTH;
+ }
+
+ if (rect->height < STFCAMSS_FRAME_MIN_HEIGHT) {
+ rect->top = 0;
+ rect->height = STFCAMSS_FRAME_MAX_HEIGHT;
+ }
+ rect->height &= ~0x1;
+}
+
+static int isp_get_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format fmt = { 0 };
+ struct v4l2_rect *rect;
+ int ret;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ if (sel->pad == STF_ISP_PAD_SINK) {
+ fmt.pad = sel->pad;
+ fmt.which = sel->which;
+ ret = isp_get_format(sd, state, &fmt);
+ if (ret < 0)
+ return ret;
+
+ sel->r.left = 0;
+ sel->r.top = 0;
+ sel->r.width = fmt.format.width;
+ sel->r.height = fmt.format.height;
+ } else if (sel->pad == STF_ISP_PAD_SRC) {
+ rect = __isp_get_crop(isp_dev, state,
+ sel->pad, sel->which);
+ sel->r = *rect;
+ }
+ break;
+
+ case V4L2_SEL_TGT_CROP:
+ rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
+ if (!rect)
+ return -EINVAL;
+
+ sel->r = *rect;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int isp_set_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
+ struct v4l2_rect *rect;
+ int ret = 0;
+
+ if (sel->target == V4L2_SEL_TGT_CROP &&
+ sel->pad == STF_ISP_PAD_SINK) {
+ struct v4l2_subdev_selection crop = { 0 };
+
+ rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
+ if (!rect)
+ return -EINVAL;
+
+ mutex_lock(&isp_dev->stream_lock);
+ isp_try_crop(isp_dev, state, &sel->r, sel->which);
+ *rect = sel->r;
+ mutex_unlock(&isp_dev->stream_lock);
+
+ /* Reset source crop selection */
+ crop.which = sel->which;
+ crop.pad = STF_ISP_PAD_SRC;
+ crop.target = V4L2_SEL_TGT_CROP;
+ crop.r = *rect;
+ ret = isp_set_selection(sd, state, &crop);
+ } else if (sel->target == V4L2_SEL_TGT_CROP &&
+ sel->pad == STF_ISP_PAD_SRC) {
+ struct v4l2_subdev_format fmt = { 0 };
+
+ rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
+ if (!rect)
+ return -EINVAL;
+
+ mutex_lock(&isp_dev->stream_lock);
+ isp_try_crop(isp_dev, state, &sel->r, sel->which);
+ *rect = sel->r;
+ mutex_unlock(&isp_dev->stream_lock);
+
+ /* Reset source pad format width and height */
+ fmt.which = sel->which;
+ fmt.pad = STF_ISP_PAD_SRC;
+ fmt.format.width = rect->width;
+ fmt.format.height = rect->height;
+ ret = isp_set_format(sd, state, &fmt);
+ if (ret < 0)
+ return ret;
+ }
+
+ dev_dbg(isp_dev->stfcamss->dev, "pad: %d sel(%d,%d)/%dx%d\n",
+ sel->pad, sel->r.left, sel->r.top, sel->r.width, sel->r.height);
+
+ return 0;
+}
+
+static int isp_init_formats(struct v4l2_subdev *sd,
+ struct v4l2_subdev_fh *fh)
+{
+ struct v4l2_subdev_format format = {
+ .pad = STF_ISP_PAD_SINK,
+ .which =
+ fh ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
+ .format = {
+ .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
+ .width = 1920,
+ .height = 1080
+ }
+ };
+
+ return isp_set_format(sd, fh ? fh->state : NULL, &format);
+}
+
+static int isp_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ if (flags & MEDIA_LNK_FL_ENABLED)
+ if (media_pad_remote_pad_first(local))
+ return -EBUSY;
+ return 0;
+}
+
+static int stf_isp_subscribe_event(struct v4l2_subdev *sd,
+ struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_src_change_event_subdev_subscribe(sd, fh, sub);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct v4l2_subdev_core_ops isp_core_ops = {
+ .subscribe_event = stf_isp_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static const struct v4l2_subdev_video_ops isp_video_ops = {
+ .s_stream = isp_set_stream,
+};
+
+static const struct v4l2_subdev_pad_ops isp_pad_ops = {
+ .enum_mbus_code = isp_enum_mbus_code,
+ .enum_frame_size = isp_enum_frame_size,
+ .get_fmt = isp_get_format,
+ .set_fmt = isp_set_format,
+ .get_selection = isp_get_selection,
+ .set_selection = isp_set_selection,
+};
+
+static const struct v4l2_subdev_ops isp_v4l2_ops = {
+ .core = &isp_core_ops,
+ .video = &isp_video_ops,
+ .pad = &isp_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops isp_v4l2_internal_ops = {
+ .open = isp_init_formats,
+};
+
+static const struct media_entity_operations isp_media_ops = {
+ .link_setup = isp_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
+{
+ struct v4l2_subdev *sd = &isp_dev->subdev;
+ struct media_pad *pads = isp_dev->pads;
+ int ret;
+
+ v4l2_subdev_init(sd, &isp_v4l2_ops);
+ sd->internal_ops = &isp_v4l2_internal_ops;
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d", STF_ISP_NAME, 0);
+ v4l2_set_subdevdata(sd, isp_dev);
+
+ ret = isp_init_formats(sd, NULL);
+ if (ret < 0) {
+ dev_err(isp_dev->stfcamss->dev, "Failed to init format: %d\n",
+ ret);
+ return ret;
+ }
+
+ pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
+
+ sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ sd->entity.ops = &isp_media_ops;
+ ret = media_entity_pads_init(&sd->entity, STF_ISP_PAD_MAX, pads);
+ if (ret < 0) {
+ dev_err(isp_dev->stfcamss->dev,
+ "Failed to init media entity: %d\n", ret);
+ return ret;
+ }
+
+ ret = v4l2_device_register_subdev(v4l2_dev, sd);
+ if (ret < 0) {
+ dev_err(isp_dev->stfcamss->dev,
+ "Failed to register subdev: %d\n", ret);
+ goto err_sreg;
+ }
+
+ return 0;
+
+err_sreg:
+ media_entity_cleanup(&sd->entity);
+ return ret;
+}
+
+int stf_isp_unregister(struct stf_isp_dev *isp_dev)
+{
+ v4l2_device_unregister_subdev(&isp_dev->subdev);
+ media_entity_cleanup(&isp_dev->subdev.entity);
+ mutex_destroy(&isp_dev->stream_lock);
+ return 0;
+}
diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
new file mode 100644
index 000000000000..1e5c98482350
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_isp.h
@@ -0,0 +1,479 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * stf_isp.h
+ *
+ * StarFive Camera Subsystem - ISP Module
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_ISP_H
+#define STF_ISP_H
+
+#include <media/v4l2-subdev.h>
+
+#include "stf_video.h"
+
+#define ISP_RAW_DATA_BITS 12
+#define SCALER_RATIO_MAX 1
+#define STF_ISP_REG_OFFSET_MAX 0x0fff
+#define STF_ISP_REG_DELAY_MAX 100
+
+/* isp registers */
+#define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
+#define CSI_SCD_ERR BIT(6)
+#define CSI_ITU656_ERR BIT(4)
+#define CSI_ITU656_F BIT(3)
+#define CSI_SCD_DONE BIT(2)
+#define CSI_BUSY_S BIT(1)
+#define CSI_EN_S BIT(0)
+
+#define ISP_REG_CSIINTS 0x008
+#define CSI_INTS(n) ((n) << 16)
+#define CSI_SHA_M(n) ((n) << 0)
+#define CSI_INTS_MASK GENMASK(17, 16)
+
+#define ISP_REG_CSI_MODULE_CFG 0x010
+#define CSI_DUMP_EN BIT(19)
+#define CSI_VS_EN BIT(18)
+#define CSI_SC_EN BIT(17)
+#define CSI_OBA_EN BIT(16)
+#define CSI_AWB_EN BIT(7)
+#define CSI_LCCF_EN BIT(6)
+#define CSI_OECFHM_EN BIT(5)
+#define CSI_OECF_EN BIT(4)
+#define CSI_LCBQ_EN BIT(3)
+#define CSI_OBC_EN BIT(2)
+#define CSI_DEC_EN BIT(1)
+#define CSI_DC_EN BIT(0)
+
+#define ISP_REG_SENSOR 0x014
+#define DVP_SYNC_POL(n) ((n) << 2)
+#define ITU656_EN(n) ((n) << 1)
+#define IMAGER_SEL(n) ((n) << 0)
+
+#define ISP_REG_RAW_FORMAT_CFG 0x018
+#define SMY13(n) ((n) << 14)
+#define SMY12(n) ((n) << 12)
+#define SMY11(n) ((n) << 10)
+#define SMY10(n) ((n) << 8)
+#define SMY3(n) ((n) << 6)
+#define SMY2(n) ((n) << 4)
+#define SMY1(n) ((n) << 2)
+#define SMY0(n) ((n) << 0)
+
+#define ISP_REG_PIC_CAPTURE_START_CFG 0x01c
+#define VSTART_CAP(n) ((n) << 16)
+#define HSTART_CAP(n) ((n) << 0)
+
+#define ISP_REG_PIC_CAPTURE_END_CFG 0x020
+#define VEND_CAP(n) ((n) << 16)
+#define HEND_CAP(n) ((n) << 0)
+
+#define ISP_REG_DUMP_CFG_0 0x024
+#define ISP_REG_DUMP_CFG_1 0x028
+#define DUMP_ID(n) ((n) << 24)
+#define DUMP_SHT(n) ((n) << 20)
+#define DUMP_BURST_LEN(n) ((n) << 16)
+#define DUMP_SD(n) ((n) << 0)
+#define DUMP_BURST_LEN_MASK GENMASK(17, 16)
+#define DUMP_SD_MASK GENMASK(15, 0)
+
+#define ISP_REG_DEC_CFG 0x030
+#define DEC_V_KEEP(n) ((n) << 24)
+#define DEC_V_PERIOD(n) ((n) << 16)
+#define DEC_H_KEEP(n) ((n) << 8)
+#define DEC_H_PERIOD(n) ((n) << 0)
+
+#define ISP_REG_OBC_CFG 0x034
+#define OBC_W_H(y) ((y) << 4)
+#define OBC_W_W(x) ((x) << 0)
+
+#define ISP_REG_DC_CFG_1 0x044
+#define DC_AXI_ID(n) ((n) << 0)
+
+#define ISP_REG_LCCF_CFG_0 0x050
+#define Y_DISTANCE(y) ((y) << 16)
+#define X_DISTANCE(x) ((x) << 0)
+
+#define ISP_REG_LCCF_CFG_1 0x058
+#define LCCF_MAX_DIS(n) ((n) << 0)
+
+#define ISP_REG_LCBQ_CFG_0 0x074
+#define H_LCBQ(y) ((y) << 12)
+#define W_LCBQ(x) ((x) << 8)
+
+#define ISP_REG_LCBQ_CFG_1 0x07c
+#define Y_COOR(y) ((y) << 16)
+#define X_COOR(x) ((x) << 0)
+
+#define ISP_REG_LCCF_CFG_2 0x0e0
+#define ISP_REG_LCCF_CFG_3 0x0e4
+#define ISP_REG_LCCF_CFG_4 0x0e8
+#define ISP_REG_LCCF_CFG_5 0x0ec
+#define LCCF_F2_PAR(n) ((n) << 16)
+#define LCCF_F1_PAR(n) ((n) << 0)
+
+#define ISP_REG_OECF_X0_CFG0 0x100
+#define ISP_REG_OECF_X0_CFG1 0x104
+#define ISP_REG_OECF_X0_CFG2 0x108
+#define ISP_REG_OECF_X0_CFG3 0x10c
+#define ISP_REG_OECF_X0_CFG4 0x110
+#define ISP_REG_OECF_X0_CFG5 0x114
+#define ISP_REG_OECF_X0_CFG6 0x118
+#define ISP_REG_OECF_X0_CFG7 0x11c
+
+#define ISP_REG_OECF_Y3_CFG0 0x1e0
+#define ISP_REG_OECF_Y3_CFG1 0x1e4
+#define ISP_REG_OECF_Y3_CFG2 0x1e8
+#define ISP_REG_OECF_Y3_CFG3 0x1ec
+#define ISP_REG_OECF_Y3_CFG4 0x1f0
+#define ISP_REG_OECF_Y3_CFG5 0x1f4
+#define ISP_REG_OECF_Y3_CFG6 0x1f8
+#define ISP_REG_OECF_Y3_CFG7 0x1fc
+
+#define ISP_REG_OECF_S0_CFG0 0x200
+#define ISP_REG_OECF_S3_CFG7 0x27c
+#define OCEF_PAR_H(n) ((n) << 16)
+#define OCEF_PAR_L(n) ((n) << 0)
+
+#define ISP_REG_AWB_X0_CFG_0 0x280
+#define ISP_REG_AWB_X0_CFG_1 0x284
+#define ISP_REG_AWB_X1_CFG_0 0x288
+#define ISP_REG_AWB_X1_CFG_1 0x28c
+#define ISP_REG_AWB_X2_CFG_0 0x290
+#define ISP_REG_AWB_X2_CFG_1 0x294
+#define ISP_REG_AWB_X3_CFG_0 0x298
+#define ISP_REG_AWB_X3_CFG_1 0x29c
+#define AWB_X_SYMBOL_H(n) ((n) << 16)
+#define AWB_X_SYMBOL_L(n) ((n) << 0)
+
+#define ISP_REG_AWB_Y0_CFG_0 0x2a0
+#define ISP_REG_AWB_Y0_CFG_1 0x2a4
+#define ISP_REG_AWB_Y1_CFG_0 0x2a8
+#define ISP_REG_AWB_Y1_CFG_1 0x2ac
+#define ISP_REG_AWB_Y2_CFG_0 0x2b0
+#define ISP_REG_AWB_Y2_CFG_1 0x2b4
+#define ISP_REG_AWB_Y3_CFG_0 0x2b8
+#define ISP_REG_AWB_Y3_CFG_1 0x2bc
+#define AWB_Y_SYMBOL_H(n) ((n) << 16)
+#define AWB_Y_SYMBOL_L(n) ((n) << 0)
+
+#define ISP_REG_AWB_S0_CFG_0 0x2c0
+#define ISP_REG_AWB_S0_CFG_1 0x2c4
+#define ISP_REG_AWB_S1_CFG_0 0x2c8
+#define ISP_REG_AWB_S1_CFG_1 0x2cc
+#define ISP_REG_AWB_S2_CFG_0 0x2d0
+#define ISP_REG_AWB_S2_CFG_1 0x2d4
+#define ISP_REG_AWB_S3_CFG_0 0x2d8
+#define ISP_REG_AWB_S3_CFG_1 0x2dc
+#define AWB_S_SYMBOL_H(n) ((n) << 16)
+#define AWB_S_SYMBOL_L(n) ((n) << 0)
+
+#define ISP_REG_OBCG_CFG_0 0x2e0
+#define ISP_REG_OBCG_CFG_1 0x2e4
+#define ISP_REG_OBCG_CFG_2 0x2e8
+#define ISP_REG_OBCG_CFG_3 0x2ec
+#define ISP_REG_OBCO_CFG_0 0x2f0
+#define ISP_REG_OBCO_CFG_1 0x2f4
+#define ISP_REG_OBCO_CFG_2 0x2f8
+#define ISP_REG_OBCO_CFG_3 0x2fc
+#define GAIN_D_POINT(x) ((x) << 24)
+#define GAIN_C_POINT(x) ((x) << 16)
+#define GAIN_B_POINT(x) ((x) << 8)
+#define GAIN_A_POINT(x) ((x) << 0)
+#define OFFSET_D_POINT(x) ((x) << 24)
+#define OFFSET_C_POINT(x) ((x) << 16)
+#define OFFSET_B_POINT(x) ((x) << 8)
+#define OFFSET_A_POINT(x) ((x) << 0)
+
+#define ISP_REG_ISP_CTRL_0 0xa00
+#define ISPC_SCFEINT BIT(27)
+#define ISPC_VSFWINT BIT(26)
+#define ISPC_VSINT BIT(25)
+#define ISPC_INTS BIT(24)
+#define ISPC_ENUO BIT(20)
+#define ISPC_ENLS BIT(17)
+#define ISPC_ENSS1 BIT(12)
+#define ISPC_ENSS0 BIT(11)
+#define ISPC_RST BIT(1)
+#define ISPC_EN BIT(0)
+#define ISPC_RST_MASK BIT(1)
+
+#define ISP_REG_ISP_CTRL_1 0xa08
+#define CTRL_SAT(n) ((n) << 28)
+#define CTRL_DBC BIT(22)
+#define CTRL_CTC BIT(21)
+#define CTRL_YHIST BIT(20)
+#define CTRL_YCURVE BIT(19)
+#define CTRL_CTM BIT(18)
+#define CTRL_BIYUV BIT(17)
+#define CTRL_SCE BIT(8)
+#define CTRL_EE BIT(7)
+#define CTRL_CCE BIT(5)
+#define CTRL_RGE BIT(4)
+#define CTRL_CME BIT(3)
+#define CTRL_AE BIT(2)
+#define CTRL_CE BIT(1)
+#define CTRL_SAT_MASK GENMASK(31, 28)
+
+#define ISP_REG_PIPELINE_XY_SIZE 0xa0c
+#define H_ACT_CAP(n) ((n) << 16)
+#define W_ACT_CAP(n) ((n) << 0)
+
+#define ISP_REG_ICTC 0xa10
+#define GF_MODE(n) ((n) << 30)
+#define MAXGT(n) ((n) << 16)
+#define MINGT(n) ((n) << 0)
+
+#define ISP_REG_IDBC 0xa14
+#define BADGT(n) ((n) << 16)
+#define BADXT(n) ((n) << 0)
+
+#define ISP_REG_ICFAM 0xa1c
+#define CROSS_COV(n) ((n) << 4)
+#define HV_W(n) ((n) << 0)
+
+#define ISP_REG_CS_GAIN 0xa30
+#define CMAD(n) ((n) << 16)
+#define CMAB(n) ((n) << 0)
+
+#define ISP_REG_CS_THRESHOLD 0xa34
+#define CMD(n) ((n) << 16)
+#define CMB(n) ((n) << 0)
+
+#define ISP_REG_CS_OFFSET 0xa38
+#define VOFF(n) ((n) << 16)
+#define UOFF(n) ((n) << 0)
+
+#define ISP_REG_CS_HUE_F 0xa3c
+#define SIN(n) ((n) << 16)
+#define COS(n) ((n) << 0)
+
+#define ISP_REG_CS_SCALE 0xa40
+#define CMSF(n) ((n) << 0)
+
+#define ISP_REG_IESHD 0xa50
+#define SHAD_UP_M BIT(1)
+#define SHAD_UP_EN BIT(0)
+
+#define ISP_REG_YADJ0 0xa54
+#define YOIR(n) ((n) << 16)
+#define YIMIN(n) ((n) << 0)
+
+#define ISP_REG_YADJ1 0xa58
+#define YOMAX(n) ((n) << 16)
+#define YOMIN(n) ((n) << 0)
+
+#define ISP_REG_Y_PLANE_START_ADDR 0xa80
+#define ISP_REG_UV_PLANE_START_ADDR 0xa84
+
+#define ISP_REG_STRIDE 0xa88
+#define IMG_STR(n) ((n) << 0)
+
+#define ISP_REG_ITIIWSR 0xb20
+#define ITI_HSIZE(n) ((n) << 16)
+#define ITI_WSIZE(n) ((n) << 0)
+
+#define ISP_REG_ITIDWLSR 0xb24
+#define ITI_WSTRIDE(n) ((n) << 0)
+
+#define ISP_REG_ITIPDFR 0xb38
+#define ITI_PACKAGE_FMT(n) ((n) << 0)
+
+#define ISP_REG_ITIDRLSR 0xb3C
+#define ITI_STRIDE_L(n) ((n) << 0)
+
+#define ISP_REG_DNYUV_YSWR0 0xc00
+#define ISP_REG_DNYUV_YSWR1 0xc04
+#define ISP_REG_DNYUV_CSWR0 0xc08
+#define ISP_REG_DNYUV_CSWR1 0xc0c
+#define YUVSW5(n) ((n) << 20)
+#define YUVSW4(n) ((n) << 16)
+#define YUVSW3(n) ((n) << 12)
+#define YUVSW2(n) ((n) << 8)
+#define YUVSW1(n) ((n) << 4)
+#define YUVSW0(n) ((n) << 0)
+
+#define ISP_REG_DNYUV_YDR0 0xc10
+#define ISP_REG_DNYUV_YDR1 0xc14
+#define ISP_REG_DNYUV_YDR2 0xc18
+#define ISP_REG_DNYUV_CDR0 0xc1c
+#define ISP_REG_DNYUV_CDR1 0xc20
+#define ISP_REG_DNYUV_CDR2 0xc24
+#define CURVE_D_H(n) ((n) << 16)
+#define CURVE_D_L(n) ((n) << 0)
+
+#define ISP_REG_ICAMD_0 0xc40
+#define DNRM_F(n) ((n) << 16)
+#define ISP_REG_ICAMD_12 0xc70
+#define ISP_REG_ICAMD_20 0xc90
+#define ISP_REG_ICAMD_24 0xca0
+#define ISP_REG_ICAMD_25 0xca4
+#define CCM_M_DAT(n) ((n) << 0)
+
+#define ISP_REG_GAMMA_VAL0 0xe00
+#define ISP_REG_GAMMA_VAL1 0xe04
+#define ISP_REG_GAMMA_VAL2 0xe08
+#define ISP_REG_GAMMA_VAL3 0xe0c
+#define ISP_REG_GAMMA_VAL4 0xe10
+#define ISP_REG_GAMMA_VAL5 0xe14
+#define ISP_REG_GAMMA_VAL6 0xe18
+#define ISP_REG_GAMMA_VAL7 0xe1c
+#define ISP_REG_GAMMA_VAL8 0xe20
+#define ISP_REG_GAMMA_VAL9 0xe24
+#define ISP_REG_GAMMA_VAL10 0xe28
+#define ISP_REG_GAMMA_VAL11 0xe2c
+#define ISP_REG_GAMMA_VAL12 0xe30
+#define ISP_REG_GAMMA_VAL13 0xe34
+#define ISP_REG_GAMMA_VAL14 0xe38
+#define GAMMA_S_VAL(n) ((n) << 16)
+#define GAMMA_VAL(n) ((n) << 0)
+
+#define ISP_REG_R2Y_0 0xe40
+#define ISP_REG_R2Y_1 0xe44
+#define ISP_REG_R2Y_2 0xe48
+#define ISP_REG_R2Y_3 0xe4c
+#define ISP_REG_R2Y_4 0xe50
+#define ISP_REG_R2Y_5 0xe54
+#define ISP_REG_R2Y_6 0xe58
+#define ISP_REG_R2Y_7 0xe5c
+#define ISP_REG_R2Y_8 0xe60
+#define CSC_M(n) ((n) << 0)
+
+#define ISP_REG_SHARPEN0 0xe80
+#define ISP_REG_SHARPEN1 0xe84
+#define ISP_REG_SHARPEN2 0xe88
+#define ISP_REG_SHARPEN3 0xe8c
+#define ISP_REG_SHARPEN4 0xe90
+#define ISP_REG_SHARPEN5 0xe94
+#define ISP_REG_SHARPEN6 0xe98
+#define ISP_REG_SHARPEN7 0xe9c
+#define ISP_REG_SHARPEN8 0xea0
+#define ISP_REG_SHARPEN9 0xea4
+#define ISP_REG_SHARPEN10 0xea8
+#define ISP_REG_SHARPEN11 0xeac
+#define ISP_REG_SHARPEN12 0xeb0
+#define ISP_REG_SHARPEN13 0xeb4
+#define ISP_REG_SHARPEN14 0xeb8
+#define S_DELTA(n) ((n) << 16)
+#define S_WEIGHT(n) ((n) << 8)
+
+#define ISP_REG_SHARPEN_FS0 0xebc
+#define ISP_REG_SHARPEN_FS1 0xec0
+#define ISP_REG_SHARPEN_FS2 0xec4
+#define ISP_REG_SHARPEN_FS3 0xec8
+#define ISP_REG_SHARPEN_FS4 0xecc
+#define ISP_REG_SHARPEN_FS5 0xed0
+#define S_FACTOR(n) ((n) << 24)
+#define S_SLOPE(n) ((n) << 0)
+
+#define ISP_REG_SHARPEN_WN 0xed4
+#define PDIRF(n) ((n) << 28)
+#define NDIRF(n) ((n) << 24)
+#define WSUM(n) ((n) << 0)
+
+#define ISP_REG_IUVS1 0xed8
+#define UVDIFF2(n) ((n) << 16)
+#define UVDIFF1(n) ((n) << 0)
+
+#define ISP_REG_IUVS2 0xedc
+#define UVF(n) ((n) << 24)
+#define UVSLOPE(n) ((n) << 0)
+
+#define ISP_REG_IUVCKS1 0xee0
+#define UVCKDIFF2(n) ((n) << 16)
+#define UVCKDIFF1(n) ((n) << 0)
+
+#define ISP_REG_IUVCKS2 0xee4
+#define UVCKSLOPE(n) ((n) << 0)
+
+#define ISP_REG_ISHRPET 0xee8
+#define TH(n) ((n) << 8)
+#define EN(n) ((n) << 0)
+
+#define ISP_REG_YCURVE_0 0xf00
+#define ISP_REG_YCURVE_63 0xffc
+#define L_PARAM(n) ((n) << 0)
+
+#define IMAGE_MAX_WIDTH 1920
+#define IMAGE_MAX_HEIGH 1080
+
+/* The output line of ISP */
+enum isp_line_id {
+ STF_ISP_LINE_INVALID = -1,
+ STF_ISP_LINE_SRC = 1,
+ STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
+};
+
+/* pad id for media framework */
+enum isp_pad_id {
+ STF_ISP_PAD_SINK = 0,
+ STF_ISP_PAD_SRC,
+ STF_ISP_PAD_MAX
+};
+
+enum {
+ EN_INT_NONE = 0,
+ EN_INT_ISP_DONE = (0x1 << 24),
+ EN_INT_CSI_DONE = (0x1 << 25),
+ EN_INT_SC_DONE = (0x1 << 26),
+ EN_INT_LINE_INT = (0x1 << 27),
+ EN_INT_ALL = (0xF << 24),
+};
+
+enum {
+ INTERFACE_DVP = 0,
+ INTERFACE_CSI,
+};
+
+struct isp_format {
+ u32 code;
+ u8 bpp;
+};
+
+struct isp_format_table {
+ const struct isp_format *fmts;
+ int nfmts;
+};
+
+struct regval_t {
+ u32 addr;
+ u32 val;
+ u32 delay_ms;
+};
+
+struct reg_table {
+ const struct regval_t *regval;
+ int regval_num;
+};
+
+struct stf_isp_crop {
+ struct v4l2_rect rect;
+ u32 bpp;
+};
+
+struct stf_isp_dev {
+ struct stfcamss *stfcamss;
+ struct v4l2_subdev subdev;
+ struct media_pad pads[STF_ISP_PAD_MAX];
+ struct v4l2_mbus_framefmt fmt[STF_ISP_PAD_MAX];
+ struct stf_isp_crop rect[STF_ISP_PAD_MAX];
+ const struct isp_format_table *formats;
+ unsigned int nformats;
+ struct mutex stream_lock; /* serialize stream control */
+ int stream_count;
+};
+
+int stf_isp_clk_enable(struct stf_isp_dev *isp_dev);
+int stf_isp_clk_disable(struct stf_isp_dev *isp_dev);
+int stf_isp_reset(struct stf_isp_dev *isp_dev);
+void stf_isp_init_cfg(struct stf_isp_dev *isp_dev);
+void stf_isp_settings(struct stf_isp_dev *isp_dev,
+ struct stf_isp_crop *crop_array, u32 mcode);
+void stf_isp_stream_set(struct stf_isp_dev *isp_dev);
+int stf_isp_subdev_init(struct stfcamss *stfcamss);
+int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev);
+int stf_isp_unregister(struct stf_isp_dev *isp_dev);
+
+#endif /* STF_ISP_H */
diff --git a/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
new file mode 100644
index 000000000000..2088b6bd0d61
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
@@ -0,0 +1,468 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_isp_hw_ops.c
+ *
+ * Register interface file for StarFive ISP driver
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ *
+ */
+
+#include "stf_camss.h"
+
+static void stf_isp_config_obc(struct stfcamss *stfcamss)
+{
+ u32 reg_val, reg_add;
+
+ stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG, OBC_W_H(11) | OBC_W_W(11));
+
+ reg_val = GAIN_D_POINT(0x40) | GAIN_C_POINT(0x40) |
+ GAIN_B_POINT(0x40) | GAIN_A_POINT(0x40);
+ for (reg_add = ISP_REG_OBCG_CFG_0; reg_add <= ISP_REG_OBCG_CFG_3;) {
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ }
+
+ reg_val = OFFSET_D_POINT(0) | OFFSET_C_POINT(0) |
+ OFFSET_B_POINT(0) | OFFSET_A_POINT(0);
+ for (reg_add = ISP_REG_OBCO_CFG_0; reg_add <= ISP_REG_OBCO_CFG_3;) {
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ }
+}
+
+static void stf_isp_config_oecf(struct stfcamss *stfcamss)
+{
+ u32 reg_add, par_val;
+ u16 par_h, par_l;
+
+ par_h = 0x10; par_l = 0;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG0; reg_add <= ISP_REG_OECF_Y3_CFG0;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x40; par_l = 0x20;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG1; reg_add <= ISP_REG_OECF_Y3_CFG1;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x80; par_l = 0x60;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG2; reg_add <= ISP_REG_OECF_Y3_CFG2;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0xc0; par_l = 0xa0;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG3; reg_add <= ISP_REG_OECF_Y3_CFG3;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x100; par_l = 0xe0;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG4; reg_add <= ISP_REG_OECF_Y3_CFG4;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x200; par_l = 0x180;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG5; reg_add <= ISP_REG_OECF_Y3_CFG5;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x300; par_l = 0x280;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG6; reg_add <= ISP_REG_OECF_Y3_CFG6;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x3fe; par_l = 0x380;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_X0_CFG7; reg_add <= ISP_REG_OECF_Y3_CFG7;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 0x20;
+ }
+
+ par_h = 0x80; par_l = 0x80;
+ par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
+ for (reg_add = ISP_REG_OECF_S0_CFG0; reg_add <= ISP_REG_OECF_S3_CFG7;) {
+ stf_isp_reg_write(stfcamss, reg_add, par_val);
+ reg_add += 4;
+ }
+}
+
+static void stf_isp_config_lccf(struct stfcamss *stfcamss)
+{
+ u32 reg_add;
+
+ stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
+ Y_DISTANCE(0x21C) | X_DISTANCE(0x3C0));
+ stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1, LCCF_MAX_DIS(0xb));
+
+ for (reg_add = ISP_REG_LCCF_CFG_2; reg_add <= ISP_REG_LCCF_CFG_5;) {
+ stf_isp_reg_write(stfcamss, reg_add,
+ LCCF_F2_PAR(0x0) | LCCF_F1_PAR(0x0));
+ reg_add += 4;
+ }
+}
+
+static void stf_isp_config_awb(struct stfcamss *stfcamss)
+{
+ u32 reg_val, reg_add;
+ u16 symbol_h, symbol_l;
+
+ symbol_h = 0x0; symbol_l = 0x0;
+ reg_val = AWB_X_SYMBOL_H(symbol_h) | AWB_X_SYMBOL_L(symbol_l);
+
+ for (reg_add = ISP_REG_AWB_X0_CFG_0; reg_add <= ISP_REG_AWB_X3_CFG_1;) {
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ }
+
+ symbol_h = 0x0, symbol_l = 0x0;
+ reg_val = AWB_Y_SYMBOL_H(symbol_h) | AWB_Y_SYMBOL_L(symbol_l);
+
+ for (reg_add = ISP_REG_AWB_Y0_CFG_0; reg_add <= ISP_REG_AWB_Y3_CFG_1;) {
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ }
+
+ symbol_h = 0x80, symbol_l = 0x80;
+ reg_val = AWB_S_SYMBOL_H(symbol_h) | AWB_S_SYMBOL_L(symbol_l);
+
+ for (reg_add = ISP_REG_AWB_S0_CFG_0; reg_add <= ISP_REG_AWB_S3_CFG_1;) {
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ }
+}
+
+static void stf_isp_config_grgb(struct stfcamss *stfcamss)
+{
+ stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
+ GF_MODE(1) | MAXGT(0x140) | MINGT(0x40));
+ stf_isp_reg_write(stfcamss, ISP_REG_IDBC, BADGT(0x200) | BADXT(0x200));
+}
+
+static void stf_isp_config_cfa(struct stfcamss *stfcamss)
+{
+ stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG,
+ SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) | SMY3(2) |
+ SMY2(3) | SMY1(2) | SMY0(3));
+ stf_isp_reg_write(stfcamss, ISP_REG_ICFAM, CROSS_COV(3) | HV_W(2));
+}
+
+static void stf_isp_config_ccm(struct stfcamss *stfcamss)
+{
+ u32 reg_add;
+
+ stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6) | CCM_M_DAT(0));
+
+ for (reg_add = ISP_REG_ICAMD_12; reg_add <= ISP_REG_ICAMD_20;) {
+ stf_isp_reg_write(stfcamss, reg_add, CCM_M_DAT(0x80));
+ reg_add += 0x10;
+ }
+
+ stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_24, CCM_M_DAT(0x700));
+ stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_25, CCM_M_DAT(0x200));
+}
+
+static void stf_isp_config_gamma(struct stfcamss *stfcamss)
+{
+ u32 reg_val, reg_add;
+ u16 gamma_slope_v, gamma_v;
+
+ gamma_slope_v = 0x2400; gamma_v = 0x0;
+ reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
+ stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL0, reg_val);
+
+ gamma_slope_v = 0x800; gamma_v = 0x20;
+ for (reg_add = ISP_REG_GAMMA_VAL1; reg_add <= ISP_REG_GAMMA_VAL7;) {
+ reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ gamma_v += 0x20;
+ }
+
+ gamma_v = 0x100;
+ for (reg_add = ISP_REG_GAMMA_VAL8; reg_add <= ISP_REG_GAMMA_VAL13;) {
+ reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
+ stf_isp_reg_write(stfcamss, reg_add, reg_val);
+ reg_add += 4;
+ gamma_v += 0x80;
+ }
+
+ gamma_v = 0x3fe;
+ reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
+ stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL14, reg_val);
+}
+
+static void stf_isp_config_r2y(struct stfcamss *stfcamss)
+{
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_0, CSC_M(0x4C));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_1, CSC_M(0x97));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_2, CSC_M(0x1d));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_3, CSC_M(0x1d5));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_4, CSC_M(0x1ac));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_5, CSC_M(0x80));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_6, CSC_M(0x80));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_7, CSC_M(0x194));
+ stf_isp_reg_write(stfcamss, ISP_REG_R2Y_8, CSC_M(0x1ec));
+}
+
+static void stf_isp_config_y_curve(struct stfcamss *stfcamss)
+{
+ u32 reg_add;
+ u16 y_curve;
+
+ y_curve = 0x0;
+ for (reg_add = ISP_REG_YCURVE_0; reg_add <= ISP_REG_YCURVE_63;) {
+ stf_isp_reg_write(stfcamss, reg_add, y_curve);
+ reg_add += 4;
+ y_curve += 0x10;
+ }
+}
+
+static void stf_isp_config_sharpen(struct stfcamss *sc)
+{
+ u32 reg_add;
+
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN0, S_DELTA(0x7) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN1, S_DELTA(0x18) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN2, S_DELTA(0x80) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN3, S_DELTA(0x100) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN4, S_DELTA(0x10) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN5, S_DELTA(0x60) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN6, S_DELTA(0x100) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN7, S_DELTA(0x190) | S_WEIGHT(0xf));
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN8, S_DELTA(0x0) | S_WEIGHT(0xf));
+
+ for (reg_add = ISP_REG_SHARPEN9; reg_add <= ISP_REG_SHARPEN14;) {
+ stf_isp_reg_write(sc, reg_add, S_WEIGHT(0xf));
+ reg_add += 4;
+ }
+
+ for (reg_add = ISP_REG_SHARPEN_FS0; reg_add <= ISP_REG_SHARPEN_FS5;) {
+ stf_isp_reg_write(sc, reg_add, S_FACTOR(0x10) | S_SLOPE(0x0));
+ reg_add += 4;
+ }
+
+ stf_isp_reg_write(sc, ISP_REG_SHARPEN_WN,
+ PDIRF(0x8) | NDIRF(0x8) | WSUM(0xd7c));
+ stf_isp_reg_write(sc, ISP_REG_IUVS1, UVDIFF2(0xC0) | UVDIFF1(0x40));
+ stf_isp_reg_write(sc, ISP_REG_IUVS2, UVF(0xff) | UVSLOPE(0x0));
+ stf_isp_reg_write(sc, ISP_REG_IUVCKS1,
+ UVCKDIFF2(0xa0) | UVCKDIFF1(0x40));
+}
+
+static void stf_isp_config_dnyuv(struct stfcamss *stfcamss)
+{
+ u32 reg_val;
+
+ reg_val = YUVSW5(7) | YUVSW4(7) | YUVSW3(7) | YUVSW2(7) |
+ YUVSW1(7) | YUVSW0(7);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0, reg_val);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0, reg_val);
+
+ reg_val = YUVSW3(7) | YUVSW2(7) | YUVSW1(7) | YUVSW0(7);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1, reg_val);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1, reg_val);
+
+ reg_val = CURVE_D_H(0x60) | CURVE_D_L(0x40);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0, reg_val);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0, reg_val);
+
+ reg_val = CURVE_D_H(0xd8) | CURVE_D_L(0x90);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1, reg_val);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1, reg_val);
+
+ reg_val = CURVE_D_H(0x1e6) | CURVE_D_L(0x144);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2, reg_val);
+ stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2, reg_val);
+}
+
+static void stf_isp_config_sat(struct stfcamss *stfcamss)
+{
+ stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN, CMAD(0x0) | CMAB(0x100));
+ stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD, CMD(0x1f) | CMB(0x1));
+ stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET, VOFF(0x0) | UOFF(0x0));
+ stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F, SIN(0x0) | COS(0x100));
+ stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, CMSF(0x8));
+ stf_isp_reg_write(stfcamss, ISP_REG_YADJ0, YOIR(0x401) | YIMIN(0x1));
+ stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
+}
+
+int stf_isp_clk_enable(struct stf_isp_dev *isp_dev)
+{
+ struct stfcamss *stfcamss = isp_dev->stfcamss;
+
+ clk_prepare_enable(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
+ reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
+ reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
+
+ return 0;
+}
+
+int stf_isp_clk_disable(struct stf_isp_dev *isp_dev)
+{
+ struct stfcamss *stfcamss = isp_dev->stfcamss;
+
+ reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
+ reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
+ clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
+
+ return 0;
+}
+
+int stf_isp_reset(struct stf_isp_dev *isp_dev)
+{
+ stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
+ ISPC_RST_MASK, ISPC_RST);
+ stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
+ ISPC_RST_MASK, 0);
+
+ return 0;
+}
+
+void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
+{
+ stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
+ stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
+ DEC_V_KEEP(0x0) |
+ DEC_V_PERIOD(0x0) |
+ DEC_H_KEEP(0x0) |
+ DEC_H_PERIOD(0x0));
+
+ stf_isp_config_obc(isp_dev->stfcamss);
+ stf_isp_config_oecf(isp_dev->stfcamss);
+ stf_isp_config_lccf(isp_dev->stfcamss);
+ stf_isp_config_awb(isp_dev->stfcamss);
+ stf_isp_config_grgb(isp_dev->stfcamss);
+ stf_isp_config_cfa(isp_dev->stfcamss);
+ stf_isp_config_ccm(isp_dev->stfcamss);
+ stf_isp_config_gamma(isp_dev->stfcamss);
+ stf_isp_config_r2y(isp_dev->stfcamss);
+ stf_isp_config_y_curve(isp_dev->stfcamss);
+ stf_isp_config_sharpen(isp_dev->stfcamss);
+ stf_isp_config_dnyuv(isp_dev->stfcamss);
+ stf_isp_config_sat(isp_dev->stfcamss);
+
+ stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
+ CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
+ CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
+ stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
+ CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
+ CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
+ CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
+}
+
+static void stf_isp_config_crop(struct stfcamss *stfcamss,
+ struct stf_isp_crop *crop)
+{
+ struct v4l2_rect *rect = &crop[STF_ISP_PAD_SRC].rect;
+ u32 bpp = crop[STF_ISP_PAD_SRC].bpp;
+ u32 val;
+
+ val = VSTART_CAP(rect->top) | HSTART_CAP(rect->left);
+ stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_START_CFG, val);
+
+ val = VEND_CAP(rect->height + rect->top - 1) |
+ HEND_CAP(rect->width + rect->left - 1);
+ stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_END_CFG, val);
+
+ val = H_ACT_CAP(rect->height) | W_ACT_CAP(rect->width);
+ stf_isp_reg_write(stfcamss, ISP_REG_PIPELINE_XY_SIZE, val);
+
+ val = ALIGN(rect->width * bpp / 8, STFCAMSS_FRAME_WIDTH_ALIGN_8);
+ stf_isp_reg_write(stfcamss, ISP_REG_STRIDE, val);
+}
+
+static void stf_isp_config_raw_fmt(struct stfcamss *stfcamss, u32 mcode)
+{
+ u32 val, val1;
+
+ switch (mcode) {
+ case MEDIA_BUS_FMT_SRGGB10_1X10:
+ case MEDIA_BUS_FMT_SRGGB8_1X8:
+ /* 3 2 3 2 1 0 1 0 B Gb B Gb Gr R Gr R */
+ val = SMY13(3) | SMY12(2) | SMY11(3) | SMY10(2) |
+ SMY3(1) | SMY2(0) | SMY1(1) | SMY0(0);
+ val1 = CTRL_SAT(0x0);
+ break;
+ case MEDIA_BUS_FMT_SGRBG10_1X10:
+ case MEDIA_BUS_FMT_SGRBG8_1X8:
+ /* 2 3 2 3 0 1 0 1, Gb B Gb B R Gr R Gr */
+ val = SMY13(2) | SMY12(3) | SMY11(2) | SMY10(3) |
+ SMY3(0) | SMY2(1) | SMY1(0) | SMY0(1);
+ val1 = CTRL_SAT(0x2);
+ break;
+ case MEDIA_BUS_FMT_SGBRG10_1X10:
+ case MEDIA_BUS_FMT_SGBRG8_1X8:
+ /* 1 0 1 0 3 2 3 2, Gr R Gr R B Gb B Gb */
+ val = SMY13(1) | SMY12(0) | SMY11(1) | SMY10(0) |
+ SMY3(3) | SMY2(2) | SMY1(3) | SMY0(2);
+ val1 = CTRL_SAT(0x3);
+ break;
+ case MEDIA_BUS_FMT_SBGGR10_1X10:
+ case MEDIA_BUS_FMT_SBGGR8_1X8:
+ /* 0 1 0 1 2 3 2 3 R Gr R Gr Gb B Gb B */
+ val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
+ SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
+ val1 = CTRL_SAT(0x1);
+ break;
+ default:
+ val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
+ SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
+ val1 = CTRL_SAT(0x1);
+ break;
+ }
+ stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG, val);
+ stf_isp_reg_set_bit(stfcamss, ISP_REG_ISP_CTRL_1, CTRL_SAT_MASK, val1);
+}
+
+void stf_isp_settings(struct stf_isp_dev *isp_dev,
+ struct stf_isp_crop *crop, u32 mcode)
+{
+ struct stfcamss *stfcamss = isp_dev->stfcamss;
+
+ stf_isp_config_crop(stfcamss, crop);
+ stf_isp_config_raw_fmt(stfcamss, mcode);
+
+ stf_isp_reg_set_bit(stfcamss, ISP_REG_DUMP_CFG_1,
+ DUMP_BURST_LEN_MASK | DUMP_SD_MASK,
+ DUMP_BURST_LEN(3));
+
+ stf_isp_reg_write(stfcamss, ISP_REG_ITIIWSR,
+ ITI_HSIZE(IMAGE_MAX_HEIGH) |
+ ITI_WSIZE(IMAGE_MAX_WIDTH));
+ stf_isp_reg_write(stfcamss, ISP_REG_ITIDWLSR, ITI_WSTRIDE(0x960));
+ stf_isp_reg_write(stfcamss, ISP_REG_ITIDRLSR, ITI_STRIDE_L(0x960));
+ stf_isp_reg_write(stfcamss, ISP_REG_SENSOR, 0x1);
+}
+
+void stf_isp_stream_set(struct stf_isp_dev *isp_dev)
+{
+ struct stfcamss *stfcamss = isp_dev->stfcamss;
+
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
+ ISPC_ENUO | ISPC_ENLS | ISPC_RST, 10);
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
+ ISPC_ENUO | ISPC_ENLS, 10);
+ stf_isp_reg_write(stfcamss, ISP_REG_IESHD, SHAD_UP_M);
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
+ ISPC_ENUO | ISPC_ENLS | ISPC_EN, 10);
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
+ CSI_INTS(1) | CSI_SHA_M(4), 10);
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
+ CSI_INTS(2) | CSI_SHA_M(4), 10);
+ stf_isp_reg_write_delay(stfcamss, ISP_REG_CSI_INPUT_EN_AND_STATUS,
+ CSI_EN_S, 10);
+}
--
2.34.1
Add starfive_camss.rst file that documents the Starfive Camera
Subsystem driver which is used for handing image sensor data.
Signed-off-by: Jack Zhu <[email protected]>
---
.../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
.../media/starfive_camss_graph.dot | 16 ++++++
.../admin-guide/media/v4l-drivers.rst | 1 +
MAINTAINERS | 1 +
4 files changed, 75 insertions(+)
create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
new file mode 100644
index 000000000000..a6378849384f
--- /dev/null
+++ b/Documentation/admin-guide/media/starfive_camss.rst
@@ -0,0 +1,57 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. include:: <isonum.txt>
+
+================================
+Starfive Camera Subsystem driver
+================================
+
+Introduction
+------------
+
+This file documents the driver for the Starfive Camera Subsystem found on
+Starfive JH7110 SoC. The driver is located under drivers/media/platform/
+starfive.
+
+The driver implements V4L2, Media controller and v4l2_subdev interfaces.
+Camera sensor using V4L2 subdev interface in the kernel is supported.
+
+The driver has been successfully used on the Gstreamer 1.18.5 with
+v4l2src plugin.
+
+
+Starfive Camera Subsystem hardware
+----------------------------------
+
+The Starfive Camera Subsystem hardware consists of:
+
+- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
+- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
+ based camera sensor data stream.
+- ISP: handles the image data streams from the MIPI CSIRx Controller.
+- VIN(Video In): a top-level module, is responsible for controlling power
+ and clocks to other modules, dumps the input data to memory or transfers the
+ input data to ISP.
+
+
+Topology
+--------
+
+The media controller pipeline graph is as follows:
+
+.. _starfive_camss_graph:
+
+.. kernel-figure:: starfive_camss_graph.dot
+ :alt: starfive_camss_graph.dot
+ :align: center
+
+The driver has 2 video devices:
+
+- stf_vin0_wr_video0: capture device for images directly from the VIN module.
+- stf_vin0_isp0_video1: capture device for images without scaling.
+
+The driver has 3 subdevices:
+
+- stf_isp0: is responsible for all the isp operations.
+- stf_vin0_wr: used to dump RAW images to memory.
+- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
diff --git a/Documentation/admin-guide/media/starfive_camss_graph.dot b/Documentation/admin-guide/media/starfive_camss_graph.dot
new file mode 100644
index 000000000000..275661b720dc
--- /dev/null
+++ b/Documentation/admin-guide/media/starfive_camss_graph.dot
@@ -0,0 +1,16 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port1 -> n0000000d:port0 [style=dashed]
+ n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000004:port1 -> n00000007 [style=bold]
+ n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n0000000d:port1 -> n00000010 [style=bold]
+ n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000018:port1 -> n00000004:port0 [style=dashed]
+ n00000018:port1 -> n00000001:port0 [style=dashed]
+ n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000022:port0 -> n00000018:port0 [style=bold]
+}
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index 1c41f87c3917..2274fd29c1d7 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -27,6 +27,7 @@ Video4Linux (V4L) driver-specific documentation
si470x
si4713
si476x
+ starfive_camss
vimc
visl
vivid
diff --git a/MAINTAINERS b/MAINTAINERS
index 9ef5d0e5dff0..71291dc58671 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -20075,6 +20075,7 @@ M: Jack Zhu <[email protected]>
M: Changhuang Liang <[email protected]>
L: [email protected]
S: Maintained
+F: Documentation/admin-guide/media/starfive_camss.rst
F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
STARFIVE DEVICETREES
--
2.34.1
Add video driver for StarFive Camera Subsystem.
Signed-off-by: Jack Zhu <[email protected]>
---
.../media/platform/starfive/camss/Makefile | 4 +-
.../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
.../media/platform/starfive/camss/stf_video.h | 92 +++
3 files changed, 819 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
index d56ddd078a71..eb457917a914 100644
--- a/drivers/media/platform/starfive/camss/Makefile
+++ b/drivers/media/platform/starfive/camss/Makefile
@@ -3,6 +3,8 @@
# Makefile for StarFive Camera Subsystem driver
#
-starfive-camss-objs += stf_camss.o
+starfive-camss-objs += \
+ stf_camss.o \
+ stf_video.o
obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
new file mode 100644
index 000000000000..2e6472fe51c6
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_video.c
@@ -0,0 +1,724 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf_video.c
+ *
+ * StarFive Camera Subsystem - V4L2 device node
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#include <linux/pm_runtime.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "stf_camss.h"
+#include "stf_video.h"
+
+static const struct stfcamss_format_info formats_pix_wr[] = {
+ {
+ .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .pixelformat = V4L2_PIX_FMT_SRGGB10,
+ .planes = 1,
+ .vsub = { 1 },
+ .bpp = 10,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .pixelformat = V4L2_PIX_FMT_SGRBG10,
+ .planes = 1,
+ .vsub = { 1 },
+ .bpp = 10,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .pixelformat = V4L2_PIX_FMT_SGBRG10,
+ .planes = 1,
+ .vsub = { 1 },
+ .bpp = 10,
+ },
+ {
+ .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .pixelformat = V4L2_PIX_FMT_SBGGR10,
+ .planes = 1,
+ .vsub = { 1 },
+ .bpp = 10,
+ },
+};
+
+static const struct stfcamss_format_info formats_pix_isp[] = {
+ {
+ .code = MEDIA_BUS_FMT_Y12_1X12,
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .planes = 2,
+ .vsub = { 1, 2 },
+ .bpp = 8,
+ },
+};
+
+/* -----------------------------------------------------------------------------
+ * Helper functions
+ */
+
+static int video_find_format(u32 code, u32 pixelformat,
+ struct stfcamss_video *video)
+{
+ unsigned int i;
+
+ for (i = 0; i < video->nformats; ++i) {
+ if (video->formats[i].code == code &&
+ video->formats[i].pixelformat == pixelformat)
+ return i;
+ }
+
+ for (i = 0; i < video->nformats; ++i)
+ if (video->formats[i].code == code)
+ return i;
+
+ for (i = 0; i < video->nformats; ++i)
+ if (video->formats[i].pixelformat == pixelformat)
+ return i;
+
+ return -EINVAL;
+}
+
+static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
+{
+ struct v4l2_pix_format *pix;
+ const struct stfcamss_format_info *fi;
+ u32 width, height;
+ u32 bpl;
+ unsigned int i;
+
+ pix = &f->fmt.pix;
+
+ for (i = 0; i < video->nformats; i++)
+ if (pix->pixelformat == video->formats[i].pixelformat)
+ break;
+
+ if (i == video->nformats)
+ i = 0; /* default format */
+
+ fi = &video->formats[i];
+ width = pix->width;
+ height = pix->height;
+
+ memset(pix, 0, sizeof(*pix));
+
+ pix->pixelformat = fi->pixelformat;
+ pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
+ STFCAMSS_FRAME_MAX_WIDTH);
+ pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
+ STFCAMSS_FRAME_MAX_HEIGHT);
+ bpl = pix->width * fi->bpp / 8;
+ bpl = ALIGN(bpl, video->bpl_alignment);
+ pix->bytesperline = bpl;
+
+ for (i = 0; i < fi->planes; ++i)
+ pix->sizeimage += bpl * pix->height / fi->vsub[i];
+
+ pix->field = V4L2_FIELD_NONE;
+ pix->colorspace = V4L2_COLORSPACE_SRGB;
+ pix->flags = 0;
+ pix->ycbcr_enc =
+ V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
+ pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
+ pix->colorspace,
+ pix->ycbcr_enc);
+ pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
+
+ return 0;
+}
+
+static int stf_video_init_format(struct stfcamss_video *video)
+{
+ int ret;
+ struct v4l2_format format = {
+ .type = video->type,
+ .fmt.pix = {
+ .width = 1920,
+ .height = 1080,
+ .pixelformat = V4L2_PIX_FMT_RGB565,
+ },
+ };
+
+ ret = __video_try_fmt(video, &format);
+
+ if (ret < 0)
+ return ret;
+
+ video->active_fmt = format;
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * Video queue operations
+ */
+
+static int video_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct stfcamss_video *video = vb2_get_drv_priv(q);
+ const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
+
+ if (*num_planes) {
+ if (*num_planes != 1)
+ return -EINVAL;
+
+ if (sizes[0] < format->sizeimage)
+ return -EINVAL;
+ }
+
+ *num_planes = 1;
+ sizes[0] = format->sizeimage;
+ if (!sizes[0])
+ dev_err(video->stfcamss->dev,
+ "%s: error size is zero!!!\n", __func__);
+
+ dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
+ *num_planes, sizes[0]);
+
+ return 0;
+}
+
+static int video_buf_init(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
+ struct stfcamss_buffer *buffer =
+ container_of(vbuf, struct stfcamss_buffer, vb);
+ const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
+ dma_addr_t *paddr;
+
+ paddr = vb2_plane_cookie(vb, 0);
+ buffer->addr[0] = *paddr;
+
+ if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
+ fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
+ fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
+ fmt->pixelformat == V4L2_PIX_FMT_NV61)
+ buffer->addr[1] =
+ buffer->addr[0] + fmt->bytesperline * fmt->height;
+
+ return 0;
+}
+
+static int video_buf_prepare(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
+ const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
+
+ if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
+ dev_err(video->stfcamss->dev,
+ "sizeimage = %d, plane size = %d\n",
+ fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
+ return -EINVAL;
+ }
+ vb2_set_plane_payload(vb, 0, fmt->sizeimage);
+
+ vbuf->field = V4L2_FIELD_NONE;
+
+ return 0;
+}
+
+static void video_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
+ struct stfcamss_buffer *buffer =
+ container_of(vbuf, struct stfcamss_buffer, vb);
+
+ video->ops->queue_buffer(video, buffer);
+}
+
+/*
+ * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
+ * @mbus: v4l2_mbus_framefmt format (input)
+ * @pix: v4l2_pix_format_mplane format (output)
+ * @f: a pointer to formats array element to be used for the conversion
+ * @alignment: bytesperline alignment value
+ *
+ * Fill the output pix structure with information from the input mbus format.
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
+ struct v4l2_pix_format *pix,
+ const struct stfcamss_format_info *f,
+ unsigned int alignment)
+{
+ u32 bytesperline;
+ unsigned int i;
+
+ memset(pix, 0, sizeof(*pix));
+ v4l2_fill_pix_format(pix, mbus);
+ pix->pixelformat = f->pixelformat;
+ bytesperline = pix->width * f->bpp / 8;
+ bytesperline = ALIGN(bytesperline, alignment);
+ pix->bytesperline = bytesperline;
+
+ for (i = 0; i < f->planes; ++i)
+ pix->sizeimage += bytesperline * pix->height / f->vsub[i];
+
+ return 0;
+}
+
+static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
+ u32 *pad)
+{
+ struct media_pad *remote;
+
+ remote = media_pad_remote_pad_first(&video->pad);
+
+ if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
+ return NULL;
+
+ if (pad)
+ *pad = remote->index;
+
+ return media_entity_to_v4l2_subdev(remote->entity);
+}
+
+static int video_get_subdev_format(struct stfcamss_video *video,
+ struct v4l2_format *format)
+{
+ struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
+ struct v4l2_subdev_format fmt;
+ struct v4l2_subdev *subdev;
+ u32 pixelformat;
+ u32 pad;
+ int ret;
+
+ subdev = video_remote_subdev(video, &pad);
+ if (!subdev)
+ return -EPIPE;
+
+ fmt.pad = pad;
+ fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+
+ ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
+ if (ret)
+ return ret;
+
+ pixelformat = pix->pixelformat;
+ ret = video_find_format(fmt.format.code, pixelformat, video);
+ if (ret < 0)
+ return ret;
+
+ format->type = video->type;
+
+ return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
+ &video->formats[ret], video->bpl_alignment);
+}
+
+static int video_check_format(struct stfcamss_video *video)
+{
+ struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
+ struct v4l2_format format;
+ struct v4l2_pix_format *sd_pix = &format.fmt.pix;
+ int ret;
+
+ sd_pix->pixelformat = pix->pixelformat;
+ ret = video_get_subdev_format(video, &format);
+ if (ret < 0)
+ return ret;
+
+ if (pix->pixelformat != sd_pix->pixelformat ||
+ pix->height != sd_pix->height ||
+ pix->width != sd_pix->width ||
+ pix->field != format.fmt.pix.field) {
+ dev_err(video->stfcamss->dev,
+ "not match:\n"
+ "pixelformat: 0x%x <-> 0x%x\n"
+ "height: %d <-> %d\n"
+ "field: %d <-> %d\n",
+ pix->pixelformat, sd_pix->pixelformat,
+ pix->height, sd_pix->height,
+ pix->field, format.fmt.pix.field);
+ return -EPIPE;
+ }
+
+ return 0;
+}
+
+static int video_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct stfcamss_video *video = vb2_get_drv_priv(q);
+ struct video_device *vdev = &video->vdev;
+ struct media_entity *entity;
+ struct media_pad *pad;
+ struct v4l2_subdev *subdev;
+ int ret;
+
+ ret = video_device_pipeline_start(vdev, &video->stfcamss->pipe);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev,
+ "Failed to media_pipeline_start: %d\n", ret);
+ return ret;
+ }
+
+ ret = video_check_format(video);
+ if (ret < 0)
+ goto error;
+
+ ret = pm_runtime_resume_and_get(video->stfcamss->dev);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev, "power up failed %d\n", ret);
+ goto error;
+ }
+
+ entity = &vdev->entity;
+ while (1) {
+ pad = &entity->pads[0];
+ if (!(pad->flags & MEDIA_PAD_FL_SINK))
+ break;
+
+ pad = media_pad_remote_pad_first(pad);
+ if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
+ break;
+
+ entity = pad->entity;
+ subdev = media_entity_to_v4l2_subdev(entity);
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, 1);
+ if (ret < 0 && ret != -ENOIOCTLCMD)
+ goto err_pm_put;
+ }
+ return 0;
+
+err_pm_put:
+ pm_runtime_put(video->stfcamss->dev);
+error:
+ video_device_pipeline_stop(vdev);
+ video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
+ return ret;
+}
+
+static void video_stop_streaming(struct vb2_queue *q)
+{
+ struct stfcamss_video *video = vb2_get_drv_priv(q);
+ struct video_device *vdev = &video->vdev;
+ struct media_entity *entity;
+ struct media_pad *pad;
+ struct v4l2_subdev *subdev;
+ int ret;
+
+ entity = &vdev->entity;
+ while (1) {
+ pad = &entity->pads[0];
+ if (!(pad->flags & MEDIA_PAD_FL_SINK))
+ break;
+
+ pad = media_pad_remote_pad_first(pad);
+ if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
+ break;
+
+ entity = pad->entity;
+ subdev = media_entity_to_v4l2_subdev(entity);
+
+ v4l2_subdev_call(subdev, video, s_stream, 0);
+ }
+
+ ret = pm_runtime_put(video->stfcamss->dev);
+ if (ret < 0)
+ dev_err(video->stfcamss->dev, "power down failed:%d\n", ret);
+
+ video_device_pipeline_stop(vdev);
+ video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
+}
+
+static const struct vb2_ops stf_video_vb2_q_ops = {
+ .queue_setup = video_queue_setup,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .buf_init = video_buf_init,
+ .buf_prepare = video_buf_prepare,
+ .buf_queue = video_buf_queue,
+ .start_streaming = video_start_streaming,
+ .stop_streaming = video_stop_streaming,
+};
+
+/* -----------------------------------------------------------------------------
+ * V4L2 ioctls
+ */
+
+static int video_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, "stf camss", sizeof(cap->driver));
+ strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
+
+ return 0;
+}
+
+static int video_get_pfmt_by_index(struct stfcamss_video *video, int ndx)
+{
+ int i, j, k;
+
+ /* find index "i" of "k"th unique pixelformat in formats array */
+ k = -1;
+ for (i = 0; i < video->nformats; i++) {
+ for (j = 0; j < i; j++) {
+ if (video->formats[i].pixelformat ==
+ video->formats[j].pixelformat)
+ break;
+ }
+
+ if (j == i)
+ k++;
+
+ if (k == ndx)
+ return i;
+ }
+
+ return -EINVAL;
+}
+
+static int video_get_pfmt_by_mcode(struct stfcamss_video *video, u32 mcode)
+{
+ int i;
+
+ for (i = 0; i < video->nformats; i++) {
+ if (video->formats[i].code == mcode)
+ return i;
+ }
+
+ return -EINVAL;
+}
+
+static int video_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct stfcamss_video *video = video_drvdata(file);
+ int i;
+
+ if (f->type != video->type)
+ return -EINVAL;
+ if (f->index >= video->nformats)
+ return -EINVAL;
+
+ if (f->mbus_code) {
+ /* Each entry in formats[] table has unique mbus_code */
+ if (f->index > 0)
+ return -EINVAL;
+
+ i = video_get_pfmt_by_mcode(video, f->mbus_code);
+ } else {
+ i = video_get_pfmt_by_index(video, f->index);
+ }
+
+ if (i < 0)
+ return -EINVAL;
+
+ f->pixelformat = video->formats[i].pixelformat;
+
+ return 0;
+}
+
+static int video_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct stfcamss_video *video = video_drvdata(file);
+ int i;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ for (i = 0; i < video->nformats; i++) {
+ if (video->formats[i].pixelformat == fsize->pixel_format)
+ break;
+ }
+
+ if (i == video->nformats)
+ return -EINVAL;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+ fsize->stepwise.min_width = STFCAMSS_FRAME_MIN_WIDTH;
+ fsize->stepwise.max_width = STFCAMSS_FRAME_MAX_WIDTH;
+ fsize->stepwise.min_height = STFCAMSS_FRAME_MIN_HEIGHT;
+ fsize->stepwise.max_height = STFCAMSS_FRAME_MAX_HEIGHT;
+ fsize->stepwise.step_width = 1;
+ fsize->stepwise.step_height = 1;
+
+ return 0;
+}
+
+static int video_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct stfcamss_video *video = video_drvdata(file);
+
+ *f = video->active_fmt;
+
+ return 0;
+}
+
+static int video_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct stfcamss_video *video = video_drvdata(file);
+ int ret;
+
+ if (vb2_is_busy(&video->vb2_q))
+ return -EBUSY;
+
+ ret = __video_try_fmt(video, f);
+ if (ret < 0)
+ return ret;
+
+ video->active_fmt = *f;
+
+ return 0;
+}
+
+static int video_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct stfcamss_video *video = video_drvdata(file);
+
+ return __video_try_fmt(video, f);
+}
+
+static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
+ .vidioc_querycap = video_querycap,
+ .vidioc_enum_fmt_vid_cap = video_enum_fmt,
+ .vidioc_enum_fmt_vid_out = video_enum_fmt,
+ .vidioc_enum_framesizes = video_enum_framesizes,
+ .vidioc_g_fmt_vid_cap = video_g_fmt,
+ .vidioc_s_fmt_vid_cap = video_s_fmt,
+ .vidioc_try_fmt_vid_cap = video_try_fmt,
+ .vidioc_g_fmt_vid_out = video_g_fmt,
+ .vidioc_s_fmt_vid_out = video_s_fmt,
+ .vidioc_try_fmt_vid_out = video_try_fmt,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+};
+
+/* -----------------------------------------------------------------------------
+ * V4L2 file operations
+ */
+
+static const struct v4l2_file_operations stf_vid_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+ .read = vb2_fop_read,
+};
+
+/* -----------------------------------------------------------------------------
+ * STFCAMSS video core
+ */
+
+static void stf_video_release(struct video_device *vdev)
+{
+ struct stfcamss_video *video = video_get_drvdata(vdev);
+
+ media_entity_cleanup(&vdev->entity);
+
+ mutex_destroy(&video->q_lock);
+ mutex_destroy(&video->lock);
+}
+
+int stf_video_register(struct stfcamss_video *video,
+ struct v4l2_device *v4l2_dev, const char *name)
+{
+ struct video_device *vdev;
+ struct vb2_queue *q;
+ struct media_pad *pad = &video->pad;
+ int ret;
+
+ vdev = &video->vdev;
+
+ mutex_init(&video->q_lock);
+
+ q = &video->vb2_q;
+ q->drv_priv = video;
+ q->mem_ops = &vb2_dma_contig_memops;
+ q->ops = &stf_video_vb2_q_ops;
+ q->type = video->type;
+ q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ q->buf_struct_size = sizeof(struct stfcamss_buffer);
+ q->dev = video->stfcamss->dev;
+ q->lock = &video->q_lock;
+ q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
+ ret = vb2_queue_init(q);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev,
+ "Failed to init vb2 queue: %d\n", ret);
+ goto err_vb2_init;
+ }
+
+ pad->flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&vdev->entity, 1, pad);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev,
+ "Failed to init video entity: %d\n", ret);
+ goto err_vb2_init;
+ }
+
+ mutex_init(&video->lock);
+
+ if (video->id == STF_V_LINE_WR) {
+ video->formats = formats_pix_wr;
+ video->nformats = ARRAY_SIZE(formats_pix_wr);
+ video->bpl_alignment = 8;
+ } else {
+ video->formats = formats_pix_isp;
+ video->nformats = ARRAY_SIZE(formats_pix_isp);
+ video->bpl_alignment = 1;
+ }
+
+ ret = stf_video_init_format(video);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev,
+ "Failed to init format: %d\n", ret);
+ goto err_vid_init_format;
+ }
+
+ vdev->fops = &stf_vid_fops;
+ vdev->ioctl_ops = &stf_vid_ioctl_ops;
+ vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
+ vdev->vfl_dir = VFL_DIR_RX;
+ vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
+ vdev->release = stf_video_release;
+ vdev->v4l2_dev = v4l2_dev;
+ vdev->queue = &video->vb2_q;
+ vdev->lock = &video->lock;
+ strscpy(vdev->name, name, sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
+ if (ret < 0) {
+ dev_err(video->stfcamss->dev,
+ "Failed to register video device: %d\n", ret);
+ goto err_vid_reg;
+ }
+
+ video_set_drvdata(vdev, video);
+ return 0;
+
+err_vid_reg:
+err_vid_init_format:
+ media_entity_cleanup(&vdev->entity);
+ mutex_destroy(&video->lock);
+err_vb2_init:
+ mutex_destroy(&video->q_lock);
+ return ret;
+}
+
+void stf_video_unregister(struct stfcamss_video *video)
+{
+ vb2_video_unregister_device(&video->vdev);
+}
diff --git a/drivers/media/platform/starfive/camss/stf_video.h b/drivers/media/platform/starfive/camss/stf_video.h
new file mode 100644
index 000000000000..60323c23a40c
--- /dev/null
+++ b/drivers/media/platform/starfive/camss/stf_video.h
@@ -0,0 +1,92 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * stf_video.h
+ *
+ * StarFive Camera Subsystem - V4L2 device node
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_VIDEO_H
+#define STF_VIDEO_H
+
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-fh.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+
+#define STFCAMSS_FRAME_MIN_WIDTH 64
+#define STFCAMSS_FRAME_MAX_WIDTH 1920
+#define STFCAMSS_FRAME_MIN_HEIGHT 64
+#define STFCAMSS_FRAME_MAX_HEIGHT 1080
+#define STFCAMSS_FRAME_WIDTH_ALIGN_8 8
+#define STFCAMSS_FRAME_WIDTH_ALIGN_128 128
+#define STFCAMSS_MIN_BUFFERS 2
+
+#define STFCAMSS_MAX_ENTITY_NAME_LEN 27
+
+enum stf_v_line_id {
+ STF_V_LINE_WR = 0,
+ STF_V_LINE_ISP,
+ STF_V_LINE_MAX,
+};
+
+struct stfcamss_buffer {
+ struct vb2_v4l2_buffer vb;
+ dma_addr_t addr[3];
+ struct list_head queue;
+};
+
+struct fract {
+ u8 numerator;
+ u8 denominator;
+};
+
+/*
+ * struct stfcamss_format_info - ISP media bus format information
+ * @code: V4L2 media bus format code
+ * @pixelformat: V4L2 pixel format FCC identifier
+ * @planes: Number of planes
+ * @vsub: Vertical subsampling (for each plane)
+ * @bpp: Bits per pixel when stored in memory (for each plane)
+ */
+struct stfcamss_format_info {
+ u32 code;
+ u32 pixelformat;
+ u8 planes;
+ u8 vsub[3];
+ u8 bpp;
+};
+
+struct stfcamss_video {
+ struct stfcamss *stfcamss;
+ u8 id;
+ struct vb2_queue vb2_q;
+ struct video_device vdev;
+ struct media_pad pad;
+ struct v4l2_format active_fmt;
+ enum v4l2_buf_type type;
+ const struct stfcamss_video_ops *ops;
+ struct mutex lock; /* serialize device access */
+ struct mutex q_lock; /* protects the queue */
+ unsigned int bpl_alignment;
+ const struct stfcamss_format_info *formats;
+ unsigned int nformats;
+};
+
+struct stfcamss_video_ops {
+ int (*queue_buffer)(struct stfcamss_video *vid,
+ struct stfcamss_buffer *buf);
+ int (*flush_buffers)(struct stfcamss_video *vid,
+ enum vb2_buffer_state state);
+};
+
+int stf_video_register(struct stfcamss_video *video,
+ struct v4l2_device *v4l2_dev, const char *name);
+
+void stf_video_unregister(struct stfcamss_video *video);
+
+#endif /* STF_VIDEO_H */
--
2.34.1
Hi Jack,
kernel test robot noticed the following build warnings:
[auto build test WARNING on media-tree/master]
[also build test WARNING on robh/for-next linus/master v6.4-rc7 next-20230621]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Jack-Zhu/media-dt-bindings-Add-JH7110-Camera-Subsystem/20230619-193105
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230619112838.19797-6-jack.zhu%40starfivetech.com
patch subject: [PATCH v7 5/6] media: starfive: camss: Add ISP driver
config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20230622/[email protected]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230622/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> drivers/media/platform/starfive/camss/stf_camss.c:173:13: warning: variable 'pad_num' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
} else if (port == STF_PORT_DVP) {
^~~~~~~~~~~~~~~~~~~~
drivers/media/platform/starfive/camss/stf_camss.c:178:18: note: uninitialized use occurs here
for (i = 0; i < pad_num; ++i) {
^~~~~~~
drivers/media/platform/starfive/camss/stf_camss.c:173:9: note: remove the 'if' if its condition is always true
} else if (port == STF_PORT_DVP) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/media/platform/starfive/camss/stf_camss.c:168:25: note: initialize the variable 'pad_num' to silence this warning
unsigned int i, pad_num;
^
= 0
1 warning generated.
--
>> drivers/media/platform/starfive/camss/stf_isp.c:113:13: warning: variable 'formats' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
} else if (pad == STF_ISP_PAD_SRC) {
^~~~~~~~~~~~~~~~~~~~~~
drivers/media/platform/starfive/camss/stf_isp.c:127:18: note: uninitialized use occurs here
for (i = 0; i < formats->nfmts; i++) {
^~~~~~~
drivers/media/platform/starfive/camss/stf_isp.c:113:9: note: remove the 'if' if its condition is always true
} else if (pad == STF_ISP_PAD_SRC) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/media/platform/starfive/camss/stf_isp.c:105:40: note: initialize the variable 'formats' to silence this warning
const struct isp_format_table *formats;
^
= NULL
1 warning generated.
vim +173 drivers/media/platform/starfive/camss/stf_camss.c
156
157 static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
158 struct v4l2_subdev *subdev,
159 struct v4l2_async_subdev *asd)
160 {
161 struct stfcamss *stfcamss =
162 container_of(async, struct stfcamss, notifier);
163 struct stfcamss_async_subdev *csd =
164 container_of(asd, struct stfcamss_async_subdev, asd);
165 enum stf_port_num port = csd->port;
166 struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
167 struct media_pad *pad[STF_PADS_NUM];
168 unsigned int i, pad_num;
169
170 if (port == STF_PORT_CSI2RX) {
171 pad[0] = &isp_dev->pads[STF_PAD_SINK];
172 pad_num = 1;
> 173 } else if (port == STF_PORT_DVP) {
174 dev_err(stfcamss->dev, "Not support DVP sensor\n");
175 return -EPERM;
176 }
177
178 for (i = 0; i < pad_num; ++i) {
179 int ret;
180
181 ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
182 if (ret < 0)
183 return ret;
184 }
185
186 return 0;
187 }
188
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On 2023/6/19 19:28, Jack Zhu wrote:
> Hi,
>
> This series is the v7 series that attempts to support the Camera Subsystem
> found on StarFive JH7110 SoC.
>
Hi everyone,
Could you please help to review and give your comments?
Thanks a lot!
> The following are the media graph for the device and the v4l2-compliance
> output.
>
> ===========================================================================
> [the media graph]:
>
> digraph board {
> rankdir=TB
> n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000001:port1 -> n0000000d:port0 [style=dashed]
> n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000004:port1 -> n00000007 [style=bold]
> n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n0000000d:port1 -> n00000010 [style=bold]
> n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000018:port1 -> n00000004:port0 [style=dashed]
> n00000018:port1 -> n00000001:port0 [style=dashed]
> n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000022:port0 -> n00000018:port0 [style=bold]
> }
>
> [the device topology]:
>
> Media controller API version 6.4.0
>
> Media device information
> ------------------------
> driver starfive-camss
> model Starfive Camera Subsystem
> serial
> bus info platform:19840000.camss
> hw revision 0x0
> driver version 6.4.0
>
> Device topology
> - entity 1: stf_isp0 (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev0
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/1920x1080]
> <- "cdns_csi2rx.19800000.csi-bridge":1 []
> pad1: Source
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/1920x1080]
> -> "stf_vin0_isp0":0 []
>
> - entity 4: stf_vin0_wr (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev1
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
> <- "cdns_csi2rx.19800000.csi-bridge":1 []
> pad1: Source
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
> -> "stf_vin0_wr_video0":0 [ENABLED,IMMUTABLE]
>
> - entity 7: stf_vin0_wr_video0 (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video0
> pad0: Sink
> <- "stf_vin0_wr":1 [ENABLED,IMMUTABLE]
>
> - entity 13: stf_vin0_isp0 (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev2
> pad0: Sink
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
> <- "stf_isp0":1 []
> pad1: Source
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
> -> "stf_vin0_isp0_video1":0 [ENABLED,IMMUTABLE]
>
> - entity 16: stf_vin0_isp0_video1 (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video1
> pad0: Sink
> <- "stf_vin0_isp0":1 [ENABLED,IMMUTABLE]
>
> - entity 24: cdns_csi2rx.19800000.csi-bridge (5 pads, 3 links)
> type V4L2 subdev subtype Unknown flags 0
> pad0: Sink
> <- "imx219 6-0010":0 [ENABLED,IMMUTABLE]
> pad1: Source
> -> "stf_vin0_wr":0 []
> -> "stf_isp0":0 []
> pad2: Source
> pad3: Source
> pad4: Source
>
> - entity 34: imx219 6-0010 (1 pad, 1 link)
> type V4L2 subdev subtype Sensor flags 0
> device node name /dev/v4l-subdev3
> pad0: Source
> [fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
> crop.bounds:(8,8)/3280x2464
> crop:(8,8)/3280x2464]
> -> "cdns_csi2rx.19800000.csi-bridge":0 [ENABLED,IMMUTABLE]
>
> ===========================================================================
> [the v4l2-compliance output]:
>
> v4l2-compliance -s -d /dev/video1
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
>
> Compliance test for stf camss device /dev/video1:
>
> Driver Info:
> Driver name : stf camss
> Card type : Starfive Camera Subsystem
> Bus info : platform:19840000.camss
> Driver version : 6.4.0
> Capabilities : 0x85200001
> Video Capture
> Read/Write
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x05200001
> Video Capture
> Read/Write
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : starfive-camss
> Model : Starfive Camera Subsystem
> Serial :
> Bus info : platform:19840000.camss
> Media version : 6.4.0
> Hardware revision: 0x00000000 (0)
> Driver version : 6.4.0
> Interface Info:
> ID : 0x03000012
> Type : V4L Video
> Entity Info:
> ID : 0x00000010 (16)
> Name : stf_vin0_isp0_video1
> Function : V4L2 I/O
> Pad 0x01000011 : 0: Sink
> Link 0x02000014: from remote pad 0x100000f of entity 'stf_vin0_isp0' (Video Pixel Formatter): Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
> test invalid ioctls: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Test input 0:
>
> Streaming ioctls:
> test read/write: OK
> test blocking wait: OK
> test MMAP (no poll): OK
> test MMAP (select): OK
> test MMAP (epoll): OK
> test USERPTR (no poll): OK (Not Supported)
> test USERPTR (select): OK (Not Supported)
> test DMABUF: Cannot test, specify --expbuf-device
>
> Total for stf camss device /dev/video1: 53, Succeeded: 53, Failed: 0, Warnings: 0
>
> ===========================================================================
>
> Changes in v7:
> - HAS_DMA is used instead of DMA_CMA in Kconfig.
> - Dropped some non-essential member variables.
> - Used v4l2_async_nf_add_fwnode_remote() to simplify the relevant code.
> - Modified some Local variable types in the function.
> - Used v4l2_create_fwnode_links_to_pad() to simplify the relevant code.
> - Added error handling for clk_prepare_enable().
> - Simplified stfcamss_format_info struct and modified the relevant code.
> - Dropped enum_input, g_input and s_input.
> - Unified v4l2_ioctl_ops struct.
> - Used v4l2_fh_open()/vb2_fop_release to replace deprecated APIs.
> - Added a camss directory under the starfive directory and modified the
> patch title.
>
> v6 link: https://lore.kernel.org/all/[email protected]/
>
> Changes in v6:
> - Added 'bus-type' in bindings example.
> - Corrected spelling errors.
> - As reviewed by Bryan, used 'nclks' and 'nrsts' variables.
> - Added lccf config for ISP.
>
> v5 link: https://lore.kernel.org/all/[email protected]/
>
> Changes in v5:
> - Rebased on v6.4-rc1.
> - Added new patch.
> - Modified ISP driver.
>
> v4 link: https://lore.kernel.org/all/[email protected]/
>
> Previous cover letter from v4:
>
> This patch series adds support for the StarFive Camera Subsystem
> found on StarFive JH7110 SoC.
>
> The driver implements V4L2, Media controller and V4L2 subdev interfaces.
> Camera sensor using V4L2 subdev interface in the kernel is supported.
>
> The driver is tested on VisionFive V2 board with IMX219 camera sensor.
> GStreamer 1.18.5 with v4l2src plugin is supported.
>
> Changes since v3:
> Patch 1:
> - Modified port@0 and port@1 properties.
> - Extended the port@0 example with appropriate properties.
> - Added 'port@0' for 'required'
> Patch 2:
> - Modified spelling errors.
> Patch 3:
> - Merged patch 5 into the patch with an explanation for compatible in
> commit msg.
> Patch 6:
> - Asserted pixel_rst[i] reset in the loop after the err_disable_pixclk
> label.
> - Modified Code Style for getting sys_rst and p_rst.
> - Renamed clk_name to name and modified the relevant code.
> Patch 9:
> - Added static for stfcamss_get_mem_res function.
> - Added static for isp_close function.
> - Fixed implicit conversion warning for stf_vin_map_isp_pad function.
> - Dropped unused variables.
>
> v3: https://lore.kernel.org/all/[email protected]/
>
> Changes since v2:
> - Rebased on v6.3-rc1.
> Patch 1:
> - Modified spelling errors.
> - Added port@0.
> - Modified '$ref' of port.
> - Added 'ports' to 'required'.
> - Dropped 'stfcamss' label in example.
> - Added port@0 in example.
> - Added MAINTAINERS file.
> Patch 2:
> - Split this patch into three new patches.
> - Modified compatible property.
> - Replaced clock names with the existing names.
> - Modified 'bus-type' and 'clock-lanes'
> - Added port@2 - port@4
> - Dropped 'csi2rx' label in example.
> Patch 3:
> - Updated rst and dot file as three pipelines were deleted.
> Patch 4:
> - Split this patch into three new patches.
> - Dropped .s_power() and .get_fmt().
> - Dropped CSI-2 DT support.
> - Dropped v4l2_device_register_subdev_nodes().
> - Used assigned-clock-rates in DT to set clk value.
> - Modified 'compatible' field.
> Patch 5:
> - Deleted three pipelines.
> - Modified 'stfcamss_clocks'/'stfcamss_resets' struct.
> - Dropped stfcamss_find_sensor() function.
> - Removed redundant code from stfcamss_of_parse_endpoint_node().
> - Modified spelling errors.
> - Rewrote stfcamss_reg_media_subdev_node() function.
> - Modified stfcamss_subdev_notifier_bound().
> - Modified stfcamss_probe() function.
> - Dropped stfcamss_suspend() and stfcamss_resume().
> - Dropped dev_info() in stfcamss_remove() function.
> - Added 'stf_' prefix for enum subdev_type.
> - Moved all includes to the top in stf_camss.h file.
> - Dropped unused fields in stfcamss struct.
> - Replaced Custom logging macros with regular macros.
> - Rewrote register read and write functions.
> - Used lowercase for all hex constants.
> - Used macro to name registers.
> - Dropped unused ioctl and stf_isp_ioctl.h file.
>
> v2: https://lore.kernel.org/all/[email protected]/
>
> Changes since v1:
> - Deleted starfive,jh7110-mipi-csi2.yaml.
> - Converted cdns,csi2rx.txt to cdns,csi2rx.yaml and added ‘resets’
> properties.
> - Added ‘cdns,csi2rx.yaml’ in ‘CADENCE MIPI-CSI2 BRIDGES’ entry.
> - The following contents were modified in starfive,jh7110-camss.yaml:
> dropped quotes from ’id’ and ‘schema’; dropped ‘|’ for ‘description’;
> corrected the wrong or redundant words: ‘a ISP’, ‘PD ISP’;
> dropped ‘minItems’ for ‘reg’, ‘clocks’, ‘resets’ and ‘interrupts’;
> dropped the '_clk' and 'rst_' prefix about the 'clock-names' and
> 'reset-names';
> changed ‘endpoint@1’ to ‘endpoint’; updated examples;
> - Updated Subject for some patches.
> - Merged patch 6, 7, 8, 9, 10, 11 into one patch.
>
> Jack Zhu (6):
> media: dt-bindings: Add JH7110 Camera Subsystem
> media: admin-guide: Add starfive_camss.rst for Starfive Camera
> Subsystem
> media: starfive: camss: Add basic driver
> media: starfive: camss: Add video driver
> media: starfive: camss: Add ISP driver
> media: starfive: camss: Add VIN driver
>
> .../admin-guide/media/starfive_camss.rst | 57 +
> .../media/starfive_camss_graph.dot | 16 +
> .../admin-guide/media/v4l-drivers.rst | 1 +
> .../bindings/media/starfive,jh7110-camss.yaml | 180 +++
> MAINTAINERS | 9 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/starfive/Kconfig | 5 +
> drivers/media/platform/starfive/Makefile | 2 +
> drivers/media/platform/starfive/camss/Kconfig | 16 +
> .../media/platform/starfive/camss/Makefile | 14 +
> .../media/platform/starfive/camss/stf_camss.c | 444 +++++++
> .../media/platform/starfive/camss/stf_camss.h | 151 +++
> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++
> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++
> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++
> .../media/platform/starfive/camss/stf_video.c | 724 +++++++++++
> .../media/platform/starfive/camss/stf_video.h | 92 ++
> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
> .../media/platform/starfive/camss/stf_vin.h | 173 +++
> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
> 21 files changed, 4613 insertions(+)
> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
> create mode 100644 Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> create mode 100644 drivers/media/platform/starfive/Kconfig
> create mode 100644 drivers/media/platform/starfive/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
> create mode 100644 drivers/media/platform/starfive/camss/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>
--
Regards,
Jack Zhu
Hi Laurent, Bryan,
Could you please help to review? Thank you for your time!
On 2023/6/19 19:28, Jack Zhu wrote:
> Hi,
>
> This series is the v7 series that attempts to support the Camera Subsystem
> found on StarFive JH7110 SoC.
>
> The following are the media graph for the device and the v4l2-compliance
> output.
>
> ===========================================================================
> [the media graph]:
>
> digraph board {
> rankdir=TB
> n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000001:port1 -> n0000000d:port0 [style=dashed]
> n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000004:port1 -> n00000007 [style=bold]
> n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> n0000000d:port1 -> n00000010 [style=bold]
> n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000018:port1 -> n00000004:port0 [style=dashed]
> n00000018:port1 -> n00000001:port0 [style=dashed]
> n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> n00000022:port0 -> n00000018:port0 [style=bold]
> }
>
> [the device topology]:
>
> Media controller API version 6.4.0
>
> Media device information
> ------------------------
> driver starfive-camss
> model Starfive Camera Subsystem
> serial
> bus info platform:19840000.camss
> hw revision 0x0
> driver version 6.4.0
>
> Device topology
> - entity 1: stf_isp0 (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev0
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/1920x1080]
> <- "cdns_csi2rx.19800000.csi-bridge":1 []
> pad1: Source
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/1920x1080]
> -> "stf_vin0_isp0":0 []
>
> - entity 4: stf_vin0_wr (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev1
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
> <- "cdns_csi2rx.19800000.csi-bridge":1 []
> pad1: Source
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
> -> "stf_vin0_wr_video0":0 [ENABLED,IMMUTABLE]
>
> - entity 7: stf_vin0_wr_video0 (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video0
> pad0: Sink
> <- "stf_vin0_wr":1 [ENABLED,IMMUTABLE]
>
> - entity 13: stf_vin0_isp0 (2 pads, 2 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev2
> pad0: Sink
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
> <- "stf_isp0":1 []
> pad1: Source
> [fmt:Y12_1X12/1920x1080 field:none colorspace:srgb]
> -> "stf_vin0_isp0_video1":0 [ENABLED,IMMUTABLE]
>
> - entity 16: stf_vin0_isp0_video1 (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video1
> pad0: Sink
> <- "stf_vin0_isp0":1 [ENABLED,IMMUTABLE]
>
> - entity 24: cdns_csi2rx.19800000.csi-bridge (5 pads, 3 links)
> type V4L2 subdev subtype Unknown flags 0
> pad0: Sink
> <- "imx219 6-0010":0 [ENABLED,IMMUTABLE]
> pad1: Source
> -> "stf_vin0_wr":0 []
> -> "stf_isp0":0 []
> pad2: Source
> pad3: Source
> pad4: Source
>
> - entity 34: imx219 6-0010 (1 pad, 1 link)
> type V4L2 subdev subtype Sensor flags 0
> device node name /dev/v4l-subdev3
> pad0: Source
> [fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
> crop.bounds:(8,8)/3280x2464
> crop:(8,8)/3280x2464]
> -> "cdns_csi2rx.19800000.csi-bridge":0 [ENABLED,IMMUTABLE]
>
> ===========================================================================
> [the v4l2-compliance output]:
>
> v4l2-compliance -s -d /dev/video1
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
>
> Compliance test for stf camss device /dev/video1:
>
> Driver Info:
> Driver name : stf camss
> Card type : Starfive Camera Subsystem
> Bus info : platform:19840000.camss
> Driver version : 6.4.0
> Capabilities : 0x85200001
> Video Capture
> Read/Write
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x05200001
> Video Capture
> Read/Write
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : starfive-camss
> Model : Starfive Camera Subsystem
> Serial :
> Bus info : platform:19840000.camss
> Media version : 6.4.0
> Hardware revision: 0x00000000 (0)
> Driver version : 6.4.0
> Interface Info:
> ID : 0x03000012
> Type : V4L Video
> Entity Info:
> ID : 0x00000010 (16)
> Name : stf_vin0_isp0_video1
> Function : V4L2 I/O
> Pad 0x01000011 : 0: Sink
> Link 0x02000014: from remote pad 0x100000f of entity 'stf_vin0_isp0' (Video Pixel Formatter): Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
> test invalid ioctls: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Test input 0:
>
> Streaming ioctls:
> test read/write: OK
> test blocking wait: OK
> test MMAP (no poll): OK
> test MMAP (select): OK
> test MMAP (epoll): OK
> test USERPTR (no poll): OK (Not Supported)
> test USERPTR (select): OK (Not Supported)
> test DMABUF: Cannot test, specify --expbuf-device
>
> Total for stf camss device /dev/video1: 53, Succeeded: 53, Failed: 0, Warnings: 0
>
> ===========================================================================
>
> Changes in v7:
> - HAS_DMA is used instead of DMA_CMA in Kconfig.
> - Dropped some non-essential member variables.
> - Used v4l2_async_nf_add_fwnode_remote() to simplify the relevant code.
> - Modified some Local variable types in the function.
> - Used v4l2_create_fwnode_links_to_pad() to simplify the relevant code.
> - Added error handling for clk_prepare_enable().
> - Simplified stfcamss_format_info struct and modified the relevant code.
> - Dropped enum_input, g_input and s_input.
> - Unified v4l2_ioctl_ops struct.
> - Used v4l2_fh_open()/vb2_fop_release to replace deprecated APIs.
> - Added a camss directory under the starfive directory and modified the
> patch title.
>
> v6 link: https://lore.kernel.org/all/[email protected]/
>
> Changes in v6:
> - Added 'bus-type' in bindings example.
> - Corrected spelling errors.
> - As reviewed by Bryan, used 'nclks' and 'nrsts' variables.
> - Added lccf config for ISP.
>
> v5 link: https://lore.kernel.org/all/[email protected]/
>
> Changes in v5:
> - Rebased on v6.4-rc1.
> - Added new patch.
> - Modified ISP driver.
>
> v4 link: https://lore.kernel.org/all/[email protected]/
>
> Previous cover letter from v4:
>
> This patch series adds support for the StarFive Camera Subsystem
> found on StarFive JH7110 SoC.
>
> The driver implements V4L2, Media controller and V4L2 subdev interfaces.
> Camera sensor using V4L2 subdev interface in the kernel is supported.
>
> The driver is tested on VisionFive V2 board with IMX219 camera sensor.
> GStreamer 1.18.5 with v4l2src plugin is supported.
>
> Changes since v3:
> Patch 1:
> - Modified port@0 and port@1 properties.
> - Extended the port@0 example with appropriate properties.
> - Added 'port@0' for 'required'
> Patch 2:
> - Modified spelling errors.
> Patch 3:
> - Merged patch 5 into the patch with an explanation for compatible in
> commit msg.
> Patch 6:
> - Asserted pixel_rst[i] reset in the loop after the err_disable_pixclk
> label.
> - Modified Code Style for getting sys_rst and p_rst.
> - Renamed clk_name to name and modified the relevant code.
> Patch 9:
> - Added static for stfcamss_get_mem_res function.
> - Added static for isp_close function.
> - Fixed implicit conversion warning for stf_vin_map_isp_pad function.
> - Dropped unused variables.
>
> v3: https://lore.kernel.org/all/[email protected]/
>
> Changes since v2:
> - Rebased on v6.3-rc1.
> Patch 1:
> - Modified spelling errors.
> - Added port@0.
> - Modified '$ref' of port.
> - Added 'ports' to 'required'.
> - Dropped 'stfcamss' label in example.
> - Added port@0 in example.
> - Added MAINTAINERS file.
> Patch 2:
> - Split this patch into three new patches.
> - Modified compatible property.
> - Replaced clock names with the existing names.
> - Modified 'bus-type' and 'clock-lanes'
> - Added port@2 - port@4
> - Dropped 'csi2rx' label in example.
> Patch 3:
> - Updated rst and dot file as three pipelines were deleted.
> Patch 4:
> - Split this patch into three new patches.
> - Dropped .s_power() and .get_fmt().
> - Dropped CSI-2 DT support.
> - Dropped v4l2_device_register_subdev_nodes().
> - Used assigned-clock-rates in DT to set clk value.
> - Modified 'compatible' field.
> Patch 5:
> - Deleted three pipelines.
> - Modified 'stfcamss_clocks'/'stfcamss_resets' struct.
> - Dropped stfcamss_find_sensor() function.
> - Removed redundant code from stfcamss_of_parse_endpoint_node().
> - Modified spelling errors.
> - Rewrote stfcamss_reg_media_subdev_node() function.
> - Modified stfcamss_subdev_notifier_bound().
> - Modified stfcamss_probe() function.
> - Dropped stfcamss_suspend() and stfcamss_resume().
> - Dropped dev_info() in stfcamss_remove() function.
> - Added 'stf_' prefix for enum subdev_type.
> - Moved all includes to the top in stf_camss.h file.
> - Dropped unused fields in stfcamss struct.
> - Replaced Custom logging macros with regular macros.
> - Rewrote register read and write functions.
> - Used lowercase for all hex constants.
> - Used macro to name registers.
> - Dropped unused ioctl and stf_isp_ioctl.h file.
>
> v2: https://lore.kernel.org/all/[email protected]/
>
> Changes since v1:
> - Deleted starfive,jh7110-mipi-csi2.yaml.
> - Converted cdns,csi2rx.txt to cdns,csi2rx.yaml and added ‘resets’
> properties.
> - Added ‘cdns,csi2rx.yaml’ in ‘CADENCE MIPI-CSI2 BRIDGES’ entry.
> - The following contents were modified in starfive,jh7110-camss.yaml:
> dropped quotes from ’id’ and ‘schema’; dropped ‘|’ for ‘description’;
> corrected the wrong or redundant words: ‘a ISP’, ‘PD ISP’;
> dropped ‘minItems’ for ‘reg’, ‘clocks’, ‘resets’ and ‘interrupts’;
> dropped the '_clk' and 'rst_' prefix about the 'clock-names' and
> 'reset-names';
> changed ‘endpoint@1’ to ‘endpoint’; updated examples;
> - Updated Subject for some patches.
> - Merged patch 6, 7, 8, 9, 10, 11 into one patch.
>
> Jack Zhu (6):
> media: dt-bindings: Add JH7110 Camera Subsystem
> media: admin-guide: Add starfive_camss.rst for Starfive Camera
> Subsystem
> media: starfive: camss: Add basic driver
> media: starfive: camss: Add video driver
> media: starfive: camss: Add ISP driver
> media: starfive: camss: Add VIN driver
>
> .../admin-guide/media/starfive_camss.rst | 57 +
> .../media/starfive_camss_graph.dot | 16 +
> .../admin-guide/media/v4l-drivers.rst | 1 +
> .../bindings/media/starfive,jh7110-camss.yaml | 180 +++
> MAINTAINERS | 9 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/starfive/Kconfig | 5 +
> drivers/media/platform/starfive/Makefile | 2 +
> drivers/media/platform/starfive/camss/Kconfig | 16 +
> .../media/platform/starfive/camss/Makefile | 14 +
> .../media/platform/starfive/camss/stf_camss.c | 444 +++++++
> .../media/platform/starfive/camss/stf_camss.h | 151 +++
> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++
> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++
> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++
> .../media/platform/starfive/camss/stf_video.c | 724 +++++++++++
> .../media/platform/starfive/camss/stf_video.h | 92 ++
> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
> .../media/platform/starfive/camss/stf_vin.h | 173 +++
> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
> 21 files changed, 4613 insertions(+)
> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
> create mode 100644 Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> create mode 100644 drivers/media/platform/starfive/Kconfig
> create mode 100644 drivers/media/platform/starfive/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
> create mode 100644 drivers/media/platform/starfive/camss/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>
--
Regards,
Jack Zhu
Hi Jack,
On 19/06/2023 13:28, Jack Zhu wrote:
> Add ISP driver for StarFive Camera Subsystem.
>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../media/platform/starfive/camss/Makefile | 2 +
> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
> .../media/platform/starfive/camss/stf_camss.h | 3 +
> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
> 6 files changed, 1544 insertions(+), 3 deletions(-)
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
Just a quick note:
When I compile this I get the following warnings:
drivers/media/platform/starfive/camss/stf_camss.c: In function 'stfcamss_subdev_notifier_bound':
drivers/media/platform/starfive/camss/stf_camss.c:217:23: warning: 'pad[0]' may be used uninitialized [-Wmaybe-uninitialized]
217 | ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/media/platform/starfive/camss/stf_camss.c:202:27: note: 'pad[0]' was declared here
202 | struct media_pad *pad[STF_PADS_NUM];
| ^~~
drivers/media/platform/starfive/camss/stf_isp.c: In function 'isp_try_format.isra':
drivers/media/platform/starfive/camss/stf_isp.c:105:40: warning: 'formats' may be used uninitialized [-Wmaybe-uninitialized]
105 | const struct isp_format_table *formats;
| ^~~~~~~
drivers/media/platform/starfive/camss/stf_isp.c:105:40: note: 'formats' was declared here
105 | const struct isp_format_table *formats;
| ^~~~~~~
drivers/media/platform/starfive/camss/stf_isp.c:106:30: warning: 'rect' may be used uninitialized [-Wmaybe-uninitialized]
106 | struct stf_isp_crop *rect;
| ^~~~
drivers/media/platform/starfive/camss/stf_isp.c:106:30: note: 'rect' was declared here
106 | struct stf_isp_crop *rect;
| ^~~~
That's when I build with: make W=1 KCFLAGS=-Wmaybe-uninitialized
Regards,
Hans
>
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> index eb457917a914..cdf57e8c9546 100644
> --- a/drivers/media/platform/starfive/camss/Makefile
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -5,6 +5,8 @@
>
> starfive-camss-objs += \
> stf_camss.o \
> + stf_isp.o \
> + stf_isp_hw_ops.o \
> stf_video.o
>
> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> index dc2b5dba7bd4..6f56b45f57db 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.c
> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> @@ -115,12 +115,65 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
> return ret;
> }
>
> +/*
> + * stfcamss_init_subdevices - Initialize subdev structures and resources
> + * @stfcamss: STFCAMSS device
> + *
> + * Return 0 on success or a negative error code on failure
> + */
> +static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
> +{
> + int ret;
> +
> + ret = stf_isp_subdev_init(stfcamss);
> + if (ret < 0) {
> + dev_err(stfcamss->dev, "Failed to init isp subdev: %d\n", ret);
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
> +{
> + int ret;
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> +
> + ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
> + if (ret < 0) {
> + dev_err(stfcamss->dev,
> + "Failed to register stf isp%d entity: %d\n", 0, ret);
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
> +{
> + stf_isp_unregister(&stfcamss->isp_dev);
> +}
> +
> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> struct v4l2_subdev *subdev,
> struct v4l2_async_subdev *asd)
> {
> + struct stfcamss *stfcamss =
> + container_of(async, struct stfcamss, notifier);
> + struct stfcamss_async_subdev *csd =
> + container_of(asd, struct stfcamss_async_subdev, asd);
> + enum stf_port_num port = csd->port;
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> struct media_pad *pad[STF_PADS_NUM];
> - unsigned int i, pad_num = 0;
> + unsigned int i, pad_num;
> +
> + if (port == STF_PORT_CSI2RX) {
> + pad[0] = &isp_dev->pads[STF_PAD_SINK];
> + pad_num = 1;
> + } else if (port == STF_PORT_DVP) {
> + dev_err(stfcamss->dev, "Not support DVP sensor\n");
> + return -EPERM;
> + }
>
> for (i = 0; i < pad_num; ++i) {
> int ret;
> @@ -223,12 +276,18 @@ static int stfcamss_probe(struct platform_device *pdev)
> goto err_cleanup_notifier;
> }
>
> + ret = stfcamss_init_subdevices(stfcamss);
> + if (ret < 0) {
> + dev_err(dev, "Failed to init subdevice: %d\n", ret);
> + goto err_cleanup_notifier;
> + }
> +
> stfcamss_mc_init(pdev, stfcamss);
>
> ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
> if (ret < 0) {
> dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
> - goto err_cleanup_notifier;
> + goto err_cleanup_media_device;
> }
>
> ret = media_device_register(&stfcamss->media_dev);
> @@ -237,6 +296,12 @@ static int stfcamss_probe(struct platform_device *pdev)
> goto err_unregister_device;
> }
>
> + ret = stfcamss_register_subdevices(stfcamss);
> + if (ret < 0) {
> + dev_err(dev, "Failed to register subdevice: %d\n", ret);
> + goto err_unregister_media_dev;
> + }
> +
> pm_runtime_enable(dev);
>
> stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
> @@ -244,15 +309,19 @@ static int stfcamss_probe(struct platform_device *pdev)
> if (ret) {
> dev_err(dev, "Failed to register async subdev nodes: %d\n",
> ret);
> - goto err_unregister_media_dev;
> + goto err_unregister_subdevs;
> }
>
> return 0;
>
> +err_unregister_subdevs:
> + stfcamss_unregister_subdevices(stfcamss);
> err_unregister_media_dev:
> media_device_unregister(&stfcamss->media_dev);
> err_unregister_device:
> v4l2_device_unregister(&stfcamss->v4l2_dev);
> +err_cleanup_media_device:
> + media_device_cleanup(&stfcamss->media_dev);
> err_cleanup_notifier:
> v4l2_async_nf_cleanup(&stfcamss->notifier);
> return ret;
> @@ -268,6 +337,7 @@ static int stfcamss_remove(struct platform_device *pdev)
> {
> struct stfcamss *stfcamss = platform_get_drvdata(pdev);
>
> + stfcamss_unregister_subdevices(stfcamss);
> v4l2_device_unregister(&stfcamss->v4l2_dev);
> media_device_cleanup(&stfcamss->media_dev);
> pm_runtime_disable(&pdev->dev);
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
> index 15c4f34b9054..9482081288fa 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.h
> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
> @@ -18,6 +18,8 @@
> #include <media/v4l2-async.h>
> #include <media/v4l2-device.h>
>
> +#include "stf_isp.h"
> +
> #define STF_DVP_NAME "stf_dvp"
> #define STF_CSI_NAME "cdns_csi2rx"
> #define STF_ISP_NAME "stf_isp"
> @@ -65,6 +67,7 @@ struct stfcamss {
> struct media_device media_dev;
> struct media_pipeline pipe;
> struct device *dev;
> + struct stf_isp_dev isp_dev;
> struct v4l2_async_notifier notifier;
> void __iomem *syscon_base;
> void __iomem *isp_base;
> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
> new file mode 100644
> index 000000000000..933a583b398c
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
> @@ -0,0 +1,519 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_isp.c
> + *
> + * StarFive Camera Subsystem - ISP Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +#include <linux/firmware.h>
> +#include <media/v4l2-event.h>
> +
> +#include "stf_camss.h"
> +
> +#define SINK_FORMATS_INDEX 0
> +#define UO_FORMATS_INDEX 1
> +
> +static int isp_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel);
> +
> +static const struct isp_format isp_formats_sink[] = {
> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
> +};
> +
> +static const struct isp_format isp_formats_uo[] = {
> + { MEDIA_BUS_FMT_Y12_1X12, 8 },
> +};
> +
> +static const struct isp_format_table isp_formats_st7110[] = {
> + { isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
> + { isp_formats_uo, ARRAY_SIZE(isp_formats_uo) },
> +};
> +
> +int stf_isp_subdev_init(struct stfcamss *stfcamss)
> +{
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> +
> + isp_dev->stfcamss = stfcamss;
> + isp_dev->formats = isp_formats_st7110;
> + isp_dev->nformats = ARRAY_SIZE(isp_formats_st7110);
> +
> + mutex_init(&isp_dev->stream_lock);
> + return 0;
> +}
> +
> +static struct v4l2_mbus_framefmt *
> +__isp_get_format(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> + return v4l2_subdev_get_try_format(&isp_dev->subdev, state, pad);
> +
> + return &isp_dev->fmt[pad];
> +}
> +
> +static int isp_set_stream(struct v4l2_subdev *sd, int enable)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + int ret = 0;
> + struct v4l2_mbus_framefmt *fmt;
> + struct v4l2_event src_ch = { 0 };
> +
> + fmt = __isp_get_format(isp_dev, NULL, STF_ISP_PAD_SINK,
> + V4L2_SUBDEV_FORMAT_ACTIVE);
> + mutex_lock(&isp_dev->stream_lock);
> + if (enable) {
> + if (isp_dev->stream_count == 0) {
> + stf_isp_clk_enable(isp_dev);
> + stf_isp_reset(isp_dev);
> + stf_isp_init_cfg(isp_dev);
> + stf_isp_settings(isp_dev, isp_dev->rect, fmt->code);
> + stf_isp_stream_set(isp_dev);
> + }
> + isp_dev->stream_count++;
> + } else {
> + if (isp_dev->stream_count == 0)
> + goto exit;
> +
> + if (isp_dev->stream_count == 1)
> + stf_isp_clk_disable(isp_dev);
> +
> + isp_dev->stream_count--;
> + }
> + src_ch.type = V4L2_EVENT_SOURCE_CHANGE,
> + src_ch.u.src_change.changes = isp_dev->stream_count,
> +
> + v4l2_subdev_notify_event(sd, &src_ch);
> +exit:
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + return ret;
> +}
> +
> +static void isp_try_format(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + struct v4l2_mbus_framefmt *fmt,
> + enum v4l2_subdev_format_whence which)
> +{
> + const struct isp_format_table *formats;
> + struct stf_isp_crop *rect;
> + unsigned int i;
> +
> + if (pad == STF_ISP_PAD_SINK) {
> + /* Set format on sink pad */
> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
> + rect = &isp_dev->rect[SINK_FORMATS_INDEX];
> + } else if (pad == STF_ISP_PAD_SRC) {
> + formats = &isp_dev->formats[UO_FORMATS_INDEX];
> + rect = &isp_dev->rect[UO_FORMATS_INDEX];
> + }
> +
> + fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> + STFCAMSS_FRAME_MAX_WIDTH);
> + fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> + STFCAMSS_FRAME_MAX_HEIGHT);
> + fmt->height &= ~0x1;
> + fmt->field = V4L2_FIELD_NONE;
> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> + fmt->flags = 0;
> +
> + for (i = 0; i < formats->nfmts; i++) {
> + if (fmt->code == formats->fmts[i].code)
> + break;
> + }
> +
> + if (i >= formats->nfmts) {
> + fmt->code = formats->fmts[0].code;
> + rect->bpp = formats->fmts[0].bpp;
> + } else {
> + rect->bpp = formats->fmts[i].bpp;
> + }
> +}
> +
> +static int isp_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + const struct isp_format_table *formats;
> +
> + if (code->index >= isp_dev->nformats)
> + return -EINVAL;
> + if (code->pad == STF_ISP_PAD_SINK) {
> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
> + code->code = formats->fmts[code->index].code;
> + } else {
> + struct v4l2_mbus_framefmt *sink_fmt;
> +
> + sink_fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK,
> + code->which);
> +
> + code->code = sink_fmt->code;
> + if (!code->code)
> + return -EINVAL;
> + }
> + code->flags = 0;
> +
> + return 0;
> +}
> +
> +static int isp_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt format;
> +
> + if (fse->index != 0)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = 1;
> + format.height = 1;
> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
> + fse->min_width = format.width;
> + fse->min_height = format.height;
> +
> + if (format.code != fse->code)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = -1;
> + format.height = -1;
> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
> + fse->max_width = format.width;
> + fse->max_height = format.height;
> +
> + return 0;
> +}
> +
> +static int isp_get_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + fmt->format = *format;
> +
> + return 0;
> +}
> +
> +static int isp_set_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> +
> + isp_try_format(isp_dev, state, fmt->pad, &fmt->format, fmt->which);
> + *format = fmt->format;
> +
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Propagate to in crop */
> + if (fmt->pad == STF_ISP_PAD_SINK) {
> + struct v4l2_subdev_selection sel = { 0 };
> + int ret;
> +
> + /* Reset sink pad compose selection */
> + sel.which = fmt->which;
> + sel.pad = STF_ISP_PAD_SINK;
> + sel.target = V4L2_SEL_TGT_CROP;
> + sel.r.width = fmt->format.width;
> + sel.r.height = fmt->format.height;
> + ret = isp_set_selection(sd, state, &sel);
> + if (ret < 0)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static struct v4l2_rect *
> +__isp_get_crop(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> + return v4l2_subdev_get_try_crop(&isp_dev->subdev, state,
> + STF_ISP_PAD_SINK);
> +
> + return &isp_dev->rect[pad].rect;
> +}
> +
> +static void isp_try_crop(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_rect *rect,
> + enum v4l2_subdev_format_whence which)
> +{
> + struct v4l2_mbus_framefmt *fmt;
> +
> + fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK, which);
> +
> + if (rect->width > fmt->width)
> + rect->width = fmt->width;
> +
> + if (rect->width + rect->left > fmt->width)
> + rect->left = fmt->width - rect->width;
> +
> + if (rect->height > fmt->height)
> + rect->height = fmt->height;
> +
> + if (rect->height + rect->top > fmt->height)
> + rect->top = fmt->height - rect->height;
> +
> + if (rect->width < STFCAMSS_FRAME_MIN_WIDTH) {
> + rect->left = 0;
> + rect->width = STFCAMSS_FRAME_MAX_WIDTH;
> + }
> +
> + if (rect->height < STFCAMSS_FRAME_MIN_HEIGHT) {
> + rect->top = 0;
> + rect->height = STFCAMSS_FRAME_MAX_HEIGHT;
> + }
> + rect->height &= ~0x1;
> +}
> +
> +static int isp_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_subdev_format fmt = { 0 };
> + struct v4l2_rect *rect;
> + int ret;
> +
> + switch (sel->target) {
> + case V4L2_SEL_TGT_CROP_BOUNDS:
> + if (sel->pad == STF_ISP_PAD_SINK) {
> + fmt.pad = sel->pad;
> + fmt.which = sel->which;
> + ret = isp_get_format(sd, state, &fmt);
> + if (ret < 0)
> + return ret;
> +
> + sel->r.left = 0;
> + sel->r.top = 0;
> + sel->r.width = fmt.format.width;
> + sel->r.height = fmt.format.height;
> + } else if (sel->pad == STF_ISP_PAD_SRC) {
> + rect = __isp_get_crop(isp_dev, state,
> + sel->pad, sel->which);
> + sel->r = *rect;
> + }
> + break;
> +
> + case V4L2_SEL_TGT_CROP:
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + sel->r = *rect;
> + break;
> +
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int isp_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_rect *rect;
> + int ret = 0;
> +
> + if (sel->target == V4L2_SEL_TGT_CROP &&
> + sel->pad == STF_ISP_PAD_SINK) {
> + struct v4l2_subdev_selection crop = { 0 };
> +
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
> + *rect = sel->r;
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Reset source crop selection */
> + crop.which = sel->which;
> + crop.pad = STF_ISP_PAD_SRC;
> + crop.target = V4L2_SEL_TGT_CROP;
> + crop.r = *rect;
> + ret = isp_set_selection(sd, state, &crop);
> + } else if (sel->target == V4L2_SEL_TGT_CROP &&
> + sel->pad == STF_ISP_PAD_SRC) {
> + struct v4l2_subdev_format fmt = { 0 };
> +
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
> + *rect = sel->r;
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Reset source pad format width and height */
> + fmt.which = sel->which;
> + fmt.pad = STF_ISP_PAD_SRC;
> + fmt.format.width = rect->width;
> + fmt.format.height = rect->height;
> + ret = isp_set_format(sd, state, &fmt);
> + if (ret < 0)
> + return ret;
> + }
> +
> + dev_dbg(isp_dev->stfcamss->dev, "pad: %d sel(%d,%d)/%dx%d\n",
> + sel->pad, sel->r.left, sel->r.top, sel->r.width, sel->r.height);
> +
> + return 0;
> +}
> +
> +static int isp_init_formats(struct v4l2_subdev *sd,
> + struct v4l2_subdev_fh *fh)
> +{
> + struct v4l2_subdev_format format = {
> + .pad = STF_ISP_PAD_SINK,
> + .which =
> + fh ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
> + .format = {
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> + .width = 1920,
> + .height = 1080
> + }
> + };
> +
> + return isp_set_format(sd, fh ? fh->state : NULL, &format);
> +}
> +
> +static int isp_link_setup(struct media_entity *entity,
> + const struct media_pad *local,
> + const struct media_pad *remote, u32 flags)
> +{
> + if (flags & MEDIA_LNK_FL_ENABLED)
> + if (media_pad_remote_pad_first(local))
> + return -EBUSY;
> + return 0;
> +}
> +
> +static int stf_isp_subscribe_event(struct v4l2_subdev *sd,
> + struct v4l2_fh *fh,
> + struct v4l2_event_subscription *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return v4l2_src_change_event_subdev_subscribe(sd, fh, sub);
> + default:
> + return -EINVAL;
> + }
> +}
> +
> +static const struct v4l2_subdev_core_ops isp_core_ops = {
> + .subscribe_event = stf_isp_subscribe_event,
> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
> +
> +static const struct v4l2_subdev_video_ops isp_video_ops = {
> + .s_stream = isp_set_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops isp_pad_ops = {
> + .enum_mbus_code = isp_enum_mbus_code,
> + .enum_frame_size = isp_enum_frame_size,
> + .get_fmt = isp_get_format,
> + .set_fmt = isp_set_format,
> + .get_selection = isp_get_selection,
> + .set_selection = isp_set_selection,
> +};
> +
> +static const struct v4l2_subdev_ops isp_v4l2_ops = {
> + .core = &isp_core_ops,
> + .video = &isp_video_ops,
> + .pad = &isp_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops isp_v4l2_internal_ops = {
> + .open = isp_init_formats,
> +};
> +
> +static const struct media_entity_operations isp_media_ops = {
> + .link_setup = isp_link_setup,
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
> +{
> + struct v4l2_subdev *sd = &isp_dev->subdev;
> + struct media_pad *pads = isp_dev->pads;
> + int ret;
> +
> + v4l2_subdev_init(sd, &isp_v4l2_ops);
> + sd->internal_ops = &isp_v4l2_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d", STF_ISP_NAME, 0);
> + v4l2_set_subdevdata(sd, isp_dev);
> +
> + ret = isp_init_formats(sd, NULL);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev, "Failed to init format: %d\n",
> + ret);
> + return ret;
> + }
> +
> + pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +
> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> + sd->entity.ops = &isp_media_ops;
> + ret = media_entity_pads_init(&sd->entity, STF_ISP_PAD_MAX, pads);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev,
> + "Failed to init media entity: %d\n", ret);
> + return ret;
> + }
> +
> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev,
> + "Failed to register subdev: %d\n", ret);
> + goto err_sreg;
> + }
> +
> + return 0;
> +
> +err_sreg:
> + media_entity_cleanup(&sd->entity);
> + return ret;
> +}
> +
> +int stf_isp_unregister(struct stf_isp_dev *isp_dev)
> +{
> + v4l2_device_unregister_subdev(&isp_dev->subdev);
> + media_entity_cleanup(&isp_dev->subdev.entity);
> + mutex_destroy(&isp_dev->stream_lock);
> + return 0;
> +}
> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
> new file mode 100644
> index 000000000000..1e5c98482350
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
> @@ -0,0 +1,479 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_isp.h
> + *
> + * StarFive Camera Subsystem - ISP Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_ISP_H
> +#define STF_ISP_H
> +
> +#include <media/v4l2-subdev.h>
> +
> +#include "stf_video.h"
> +
> +#define ISP_RAW_DATA_BITS 12
> +#define SCALER_RATIO_MAX 1
> +#define STF_ISP_REG_OFFSET_MAX 0x0fff
> +#define STF_ISP_REG_DELAY_MAX 100
> +
> +/* isp registers */
> +#define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
> +#define CSI_SCD_ERR BIT(6)
> +#define CSI_ITU656_ERR BIT(4)
> +#define CSI_ITU656_F BIT(3)
> +#define CSI_SCD_DONE BIT(2)
> +#define CSI_BUSY_S BIT(1)
> +#define CSI_EN_S BIT(0)
> +
> +#define ISP_REG_CSIINTS 0x008
> +#define CSI_INTS(n) ((n) << 16)
> +#define CSI_SHA_M(n) ((n) << 0)
> +#define CSI_INTS_MASK GENMASK(17, 16)
> +
> +#define ISP_REG_CSI_MODULE_CFG 0x010
> +#define CSI_DUMP_EN BIT(19)
> +#define CSI_VS_EN BIT(18)
> +#define CSI_SC_EN BIT(17)
> +#define CSI_OBA_EN BIT(16)
> +#define CSI_AWB_EN BIT(7)
> +#define CSI_LCCF_EN BIT(6)
> +#define CSI_OECFHM_EN BIT(5)
> +#define CSI_OECF_EN BIT(4)
> +#define CSI_LCBQ_EN BIT(3)
> +#define CSI_OBC_EN BIT(2)
> +#define CSI_DEC_EN BIT(1)
> +#define CSI_DC_EN BIT(0)
> +
> +#define ISP_REG_SENSOR 0x014
> +#define DVP_SYNC_POL(n) ((n) << 2)
> +#define ITU656_EN(n) ((n) << 1)
> +#define IMAGER_SEL(n) ((n) << 0)
> +
> +#define ISP_REG_RAW_FORMAT_CFG 0x018
> +#define SMY13(n) ((n) << 14)
> +#define SMY12(n) ((n) << 12)
> +#define SMY11(n) ((n) << 10)
> +#define SMY10(n) ((n) << 8)
> +#define SMY3(n) ((n) << 6)
> +#define SMY2(n) ((n) << 4)
> +#define SMY1(n) ((n) << 2)
> +#define SMY0(n) ((n) << 0)
> +
> +#define ISP_REG_PIC_CAPTURE_START_CFG 0x01c
> +#define VSTART_CAP(n) ((n) << 16)
> +#define HSTART_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_PIC_CAPTURE_END_CFG 0x020
> +#define VEND_CAP(n) ((n) << 16)
> +#define HEND_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_DUMP_CFG_0 0x024
> +#define ISP_REG_DUMP_CFG_1 0x028
> +#define DUMP_ID(n) ((n) << 24)
> +#define DUMP_SHT(n) ((n) << 20)
> +#define DUMP_BURST_LEN(n) ((n) << 16)
> +#define DUMP_SD(n) ((n) << 0)
> +#define DUMP_BURST_LEN_MASK GENMASK(17, 16)
> +#define DUMP_SD_MASK GENMASK(15, 0)
> +
> +#define ISP_REG_DEC_CFG 0x030
> +#define DEC_V_KEEP(n) ((n) << 24)
> +#define DEC_V_PERIOD(n) ((n) << 16)
> +#define DEC_H_KEEP(n) ((n) << 8)
> +#define DEC_H_PERIOD(n) ((n) << 0)
> +
> +#define ISP_REG_OBC_CFG 0x034
> +#define OBC_W_H(y) ((y) << 4)
> +#define OBC_W_W(x) ((x) << 0)
> +
> +#define ISP_REG_DC_CFG_1 0x044
> +#define DC_AXI_ID(n) ((n) << 0)
> +
> +#define ISP_REG_LCCF_CFG_0 0x050
> +#define Y_DISTANCE(y) ((y) << 16)
> +#define X_DISTANCE(x) ((x) << 0)
> +
> +#define ISP_REG_LCCF_CFG_1 0x058
> +#define LCCF_MAX_DIS(n) ((n) << 0)
> +
> +#define ISP_REG_LCBQ_CFG_0 0x074
> +#define H_LCBQ(y) ((y) << 12)
> +#define W_LCBQ(x) ((x) << 8)
> +
> +#define ISP_REG_LCBQ_CFG_1 0x07c
> +#define Y_COOR(y) ((y) << 16)
> +#define X_COOR(x) ((x) << 0)
> +
> +#define ISP_REG_LCCF_CFG_2 0x0e0
> +#define ISP_REG_LCCF_CFG_3 0x0e4
> +#define ISP_REG_LCCF_CFG_4 0x0e8
> +#define ISP_REG_LCCF_CFG_5 0x0ec
> +#define LCCF_F2_PAR(n) ((n) << 16)
> +#define LCCF_F1_PAR(n) ((n) << 0)
> +
> +#define ISP_REG_OECF_X0_CFG0 0x100
> +#define ISP_REG_OECF_X0_CFG1 0x104
> +#define ISP_REG_OECF_X0_CFG2 0x108
> +#define ISP_REG_OECF_X0_CFG3 0x10c
> +#define ISP_REG_OECF_X0_CFG4 0x110
> +#define ISP_REG_OECF_X0_CFG5 0x114
> +#define ISP_REG_OECF_X0_CFG6 0x118
> +#define ISP_REG_OECF_X0_CFG7 0x11c
> +
> +#define ISP_REG_OECF_Y3_CFG0 0x1e0
> +#define ISP_REG_OECF_Y3_CFG1 0x1e4
> +#define ISP_REG_OECF_Y3_CFG2 0x1e8
> +#define ISP_REG_OECF_Y3_CFG3 0x1ec
> +#define ISP_REG_OECF_Y3_CFG4 0x1f0
> +#define ISP_REG_OECF_Y3_CFG5 0x1f4
> +#define ISP_REG_OECF_Y3_CFG6 0x1f8
> +#define ISP_REG_OECF_Y3_CFG7 0x1fc
> +
> +#define ISP_REG_OECF_S0_CFG0 0x200
> +#define ISP_REG_OECF_S3_CFG7 0x27c
> +#define OCEF_PAR_H(n) ((n) << 16)
> +#define OCEF_PAR_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_X0_CFG_0 0x280
> +#define ISP_REG_AWB_X0_CFG_1 0x284
> +#define ISP_REG_AWB_X1_CFG_0 0x288
> +#define ISP_REG_AWB_X1_CFG_1 0x28c
> +#define ISP_REG_AWB_X2_CFG_0 0x290
> +#define ISP_REG_AWB_X2_CFG_1 0x294
> +#define ISP_REG_AWB_X3_CFG_0 0x298
> +#define ISP_REG_AWB_X3_CFG_1 0x29c
> +#define AWB_X_SYMBOL_H(n) ((n) << 16)
> +#define AWB_X_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_Y0_CFG_0 0x2a0
> +#define ISP_REG_AWB_Y0_CFG_1 0x2a4
> +#define ISP_REG_AWB_Y1_CFG_0 0x2a8
> +#define ISP_REG_AWB_Y1_CFG_1 0x2ac
> +#define ISP_REG_AWB_Y2_CFG_0 0x2b0
> +#define ISP_REG_AWB_Y2_CFG_1 0x2b4
> +#define ISP_REG_AWB_Y3_CFG_0 0x2b8
> +#define ISP_REG_AWB_Y3_CFG_1 0x2bc
> +#define AWB_Y_SYMBOL_H(n) ((n) << 16)
> +#define AWB_Y_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_S0_CFG_0 0x2c0
> +#define ISP_REG_AWB_S0_CFG_1 0x2c4
> +#define ISP_REG_AWB_S1_CFG_0 0x2c8
> +#define ISP_REG_AWB_S1_CFG_1 0x2cc
> +#define ISP_REG_AWB_S2_CFG_0 0x2d0
> +#define ISP_REG_AWB_S2_CFG_1 0x2d4
> +#define ISP_REG_AWB_S3_CFG_0 0x2d8
> +#define ISP_REG_AWB_S3_CFG_1 0x2dc
> +#define AWB_S_SYMBOL_H(n) ((n) << 16)
> +#define AWB_S_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_OBCG_CFG_0 0x2e0
> +#define ISP_REG_OBCG_CFG_1 0x2e4
> +#define ISP_REG_OBCG_CFG_2 0x2e8
> +#define ISP_REG_OBCG_CFG_3 0x2ec
> +#define ISP_REG_OBCO_CFG_0 0x2f0
> +#define ISP_REG_OBCO_CFG_1 0x2f4
> +#define ISP_REG_OBCO_CFG_2 0x2f8
> +#define ISP_REG_OBCO_CFG_3 0x2fc
> +#define GAIN_D_POINT(x) ((x) << 24)
> +#define GAIN_C_POINT(x) ((x) << 16)
> +#define GAIN_B_POINT(x) ((x) << 8)
> +#define GAIN_A_POINT(x) ((x) << 0)
> +#define OFFSET_D_POINT(x) ((x) << 24)
> +#define OFFSET_C_POINT(x) ((x) << 16)
> +#define OFFSET_B_POINT(x) ((x) << 8)
> +#define OFFSET_A_POINT(x) ((x) << 0)
> +
> +#define ISP_REG_ISP_CTRL_0 0xa00
> +#define ISPC_SCFEINT BIT(27)
> +#define ISPC_VSFWINT BIT(26)
> +#define ISPC_VSINT BIT(25)
> +#define ISPC_INTS BIT(24)
> +#define ISPC_ENUO BIT(20)
> +#define ISPC_ENLS BIT(17)
> +#define ISPC_ENSS1 BIT(12)
> +#define ISPC_ENSS0 BIT(11)
> +#define ISPC_RST BIT(1)
> +#define ISPC_EN BIT(0)
> +#define ISPC_RST_MASK BIT(1)
> +
> +#define ISP_REG_ISP_CTRL_1 0xa08
> +#define CTRL_SAT(n) ((n) << 28)
> +#define CTRL_DBC BIT(22)
> +#define CTRL_CTC BIT(21)
> +#define CTRL_YHIST BIT(20)
> +#define CTRL_YCURVE BIT(19)
> +#define CTRL_CTM BIT(18)
> +#define CTRL_BIYUV BIT(17)
> +#define CTRL_SCE BIT(8)
> +#define CTRL_EE BIT(7)
> +#define CTRL_CCE BIT(5)
> +#define CTRL_RGE BIT(4)
> +#define CTRL_CME BIT(3)
> +#define CTRL_AE BIT(2)
> +#define CTRL_CE BIT(1)
> +#define CTRL_SAT_MASK GENMASK(31, 28)
> +
> +#define ISP_REG_PIPELINE_XY_SIZE 0xa0c
> +#define H_ACT_CAP(n) ((n) << 16)
> +#define W_ACT_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_ICTC 0xa10
> +#define GF_MODE(n) ((n) << 30)
> +#define MAXGT(n) ((n) << 16)
> +#define MINGT(n) ((n) << 0)
> +
> +#define ISP_REG_IDBC 0xa14
> +#define BADGT(n) ((n) << 16)
> +#define BADXT(n) ((n) << 0)
> +
> +#define ISP_REG_ICFAM 0xa1c
> +#define CROSS_COV(n) ((n) << 4)
> +#define HV_W(n) ((n) << 0)
> +
> +#define ISP_REG_CS_GAIN 0xa30
> +#define CMAD(n) ((n) << 16)
> +#define CMAB(n) ((n) << 0)
> +
> +#define ISP_REG_CS_THRESHOLD 0xa34
> +#define CMD(n) ((n) << 16)
> +#define CMB(n) ((n) << 0)
> +
> +#define ISP_REG_CS_OFFSET 0xa38
> +#define VOFF(n) ((n) << 16)
> +#define UOFF(n) ((n) << 0)
> +
> +#define ISP_REG_CS_HUE_F 0xa3c
> +#define SIN(n) ((n) << 16)
> +#define COS(n) ((n) << 0)
> +
> +#define ISP_REG_CS_SCALE 0xa40
> +#define CMSF(n) ((n) << 0)
> +
> +#define ISP_REG_IESHD 0xa50
> +#define SHAD_UP_M BIT(1)
> +#define SHAD_UP_EN BIT(0)
> +
> +#define ISP_REG_YADJ0 0xa54
> +#define YOIR(n) ((n) << 16)
> +#define YIMIN(n) ((n) << 0)
> +
> +#define ISP_REG_YADJ1 0xa58
> +#define YOMAX(n) ((n) << 16)
> +#define YOMIN(n) ((n) << 0)
> +
> +#define ISP_REG_Y_PLANE_START_ADDR 0xa80
> +#define ISP_REG_UV_PLANE_START_ADDR 0xa84
> +
> +#define ISP_REG_STRIDE 0xa88
> +#define IMG_STR(n) ((n) << 0)
> +
> +#define ISP_REG_ITIIWSR 0xb20
> +#define ITI_HSIZE(n) ((n) << 16)
> +#define ITI_WSIZE(n) ((n) << 0)
> +
> +#define ISP_REG_ITIDWLSR 0xb24
> +#define ITI_WSTRIDE(n) ((n) << 0)
> +
> +#define ISP_REG_ITIPDFR 0xb38
> +#define ITI_PACKAGE_FMT(n) ((n) << 0)
> +
> +#define ISP_REG_ITIDRLSR 0xb3C
> +#define ITI_STRIDE_L(n) ((n) << 0)
> +
> +#define ISP_REG_DNYUV_YSWR0 0xc00
> +#define ISP_REG_DNYUV_YSWR1 0xc04
> +#define ISP_REG_DNYUV_CSWR0 0xc08
> +#define ISP_REG_DNYUV_CSWR1 0xc0c
> +#define YUVSW5(n) ((n) << 20)
> +#define YUVSW4(n) ((n) << 16)
> +#define YUVSW3(n) ((n) << 12)
> +#define YUVSW2(n) ((n) << 8)
> +#define YUVSW1(n) ((n) << 4)
> +#define YUVSW0(n) ((n) << 0)
> +
> +#define ISP_REG_DNYUV_YDR0 0xc10
> +#define ISP_REG_DNYUV_YDR1 0xc14
> +#define ISP_REG_DNYUV_YDR2 0xc18
> +#define ISP_REG_DNYUV_CDR0 0xc1c
> +#define ISP_REG_DNYUV_CDR1 0xc20
> +#define ISP_REG_DNYUV_CDR2 0xc24
> +#define CURVE_D_H(n) ((n) << 16)
> +#define CURVE_D_L(n) ((n) << 0)
> +
> +#define ISP_REG_ICAMD_0 0xc40
> +#define DNRM_F(n) ((n) << 16)
> +#define ISP_REG_ICAMD_12 0xc70
> +#define ISP_REG_ICAMD_20 0xc90
> +#define ISP_REG_ICAMD_24 0xca0
> +#define ISP_REG_ICAMD_25 0xca4
> +#define CCM_M_DAT(n) ((n) << 0)
> +
> +#define ISP_REG_GAMMA_VAL0 0xe00
> +#define ISP_REG_GAMMA_VAL1 0xe04
> +#define ISP_REG_GAMMA_VAL2 0xe08
> +#define ISP_REG_GAMMA_VAL3 0xe0c
> +#define ISP_REG_GAMMA_VAL4 0xe10
> +#define ISP_REG_GAMMA_VAL5 0xe14
> +#define ISP_REG_GAMMA_VAL6 0xe18
> +#define ISP_REG_GAMMA_VAL7 0xe1c
> +#define ISP_REG_GAMMA_VAL8 0xe20
> +#define ISP_REG_GAMMA_VAL9 0xe24
> +#define ISP_REG_GAMMA_VAL10 0xe28
> +#define ISP_REG_GAMMA_VAL11 0xe2c
> +#define ISP_REG_GAMMA_VAL12 0xe30
> +#define ISP_REG_GAMMA_VAL13 0xe34
> +#define ISP_REG_GAMMA_VAL14 0xe38
> +#define GAMMA_S_VAL(n) ((n) << 16)
> +#define GAMMA_VAL(n) ((n) << 0)
> +
> +#define ISP_REG_R2Y_0 0xe40
> +#define ISP_REG_R2Y_1 0xe44
> +#define ISP_REG_R2Y_2 0xe48
> +#define ISP_REG_R2Y_3 0xe4c
> +#define ISP_REG_R2Y_4 0xe50
> +#define ISP_REG_R2Y_5 0xe54
> +#define ISP_REG_R2Y_6 0xe58
> +#define ISP_REG_R2Y_7 0xe5c
> +#define ISP_REG_R2Y_8 0xe60
> +#define CSC_M(n) ((n) << 0)
> +
> +#define ISP_REG_SHARPEN0 0xe80
> +#define ISP_REG_SHARPEN1 0xe84
> +#define ISP_REG_SHARPEN2 0xe88
> +#define ISP_REG_SHARPEN3 0xe8c
> +#define ISP_REG_SHARPEN4 0xe90
> +#define ISP_REG_SHARPEN5 0xe94
> +#define ISP_REG_SHARPEN6 0xe98
> +#define ISP_REG_SHARPEN7 0xe9c
> +#define ISP_REG_SHARPEN8 0xea0
> +#define ISP_REG_SHARPEN9 0xea4
> +#define ISP_REG_SHARPEN10 0xea8
> +#define ISP_REG_SHARPEN11 0xeac
> +#define ISP_REG_SHARPEN12 0xeb0
> +#define ISP_REG_SHARPEN13 0xeb4
> +#define ISP_REG_SHARPEN14 0xeb8
> +#define S_DELTA(n) ((n) << 16)
> +#define S_WEIGHT(n) ((n) << 8)
> +
> +#define ISP_REG_SHARPEN_FS0 0xebc
> +#define ISP_REG_SHARPEN_FS1 0xec0
> +#define ISP_REG_SHARPEN_FS2 0xec4
> +#define ISP_REG_SHARPEN_FS3 0xec8
> +#define ISP_REG_SHARPEN_FS4 0xecc
> +#define ISP_REG_SHARPEN_FS5 0xed0
> +#define S_FACTOR(n) ((n) << 24)
> +#define S_SLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_SHARPEN_WN 0xed4
> +#define PDIRF(n) ((n) << 28)
> +#define NDIRF(n) ((n) << 24)
> +#define WSUM(n) ((n) << 0)
> +
> +#define ISP_REG_IUVS1 0xed8
> +#define UVDIFF2(n) ((n) << 16)
> +#define UVDIFF1(n) ((n) << 0)
> +
> +#define ISP_REG_IUVS2 0xedc
> +#define UVF(n) ((n) << 24)
> +#define UVSLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_IUVCKS1 0xee0
> +#define UVCKDIFF2(n) ((n) << 16)
> +#define UVCKDIFF1(n) ((n) << 0)
> +
> +#define ISP_REG_IUVCKS2 0xee4
> +#define UVCKSLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_ISHRPET 0xee8
> +#define TH(n) ((n) << 8)
> +#define EN(n) ((n) << 0)
> +
> +#define ISP_REG_YCURVE_0 0xf00
> +#define ISP_REG_YCURVE_63 0xffc
> +#define L_PARAM(n) ((n) << 0)
> +
> +#define IMAGE_MAX_WIDTH 1920
> +#define IMAGE_MAX_HEIGH 1080
> +
> +/* The output line of ISP */
> +enum isp_line_id {
> + STF_ISP_LINE_INVALID = -1,
> + STF_ISP_LINE_SRC = 1,
> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
> +};
> +
> +/* pad id for media framework */
> +enum isp_pad_id {
> + STF_ISP_PAD_SINK = 0,
> + STF_ISP_PAD_SRC,
> + STF_ISP_PAD_MAX
> +};
> +
> +enum {
> + EN_INT_NONE = 0,
> + EN_INT_ISP_DONE = (0x1 << 24),
> + EN_INT_CSI_DONE = (0x1 << 25),
> + EN_INT_SC_DONE = (0x1 << 26),
> + EN_INT_LINE_INT = (0x1 << 27),
> + EN_INT_ALL = (0xF << 24),
> +};
> +
> +enum {
> + INTERFACE_DVP = 0,
> + INTERFACE_CSI,
> +};
> +
> +struct isp_format {
> + u32 code;
> + u8 bpp;
> +};
> +
> +struct isp_format_table {
> + const struct isp_format *fmts;
> + int nfmts;
> +};
> +
> +struct regval_t {
> + u32 addr;
> + u32 val;
> + u32 delay_ms;
> +};
> +
> +struct reg_table {
> + const struct regval_t *regval;
> + int regval_num;
> +};
> +
> +struct stf_isp_crop {
> + struct v4l2_rect rect;
> + u32 bpp;
> +};
> +
> +struct stf_isp_dev {
> + struct stfcamss *stfcamss;
> + struct v4l2_subdev subdev;
> + struct media_pad pads[STF_ISP_PAD_MAX];
> + struct v4l2_mbus_framefmt fmt[STF_ISP_PAD_MAX];
> + struct stf_isp_crop rect[STF_ISP_PAD_MAX];
> + const struct isp_format_table *formats;
> + unsigned int nformats;
> + struct mutex stream_lock; /* serialize stream control */
> + int stream_count;
> +};
> +
> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev);
> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev);
> +int stf_isp_reset(struct stf_isp_dev *isp_dev);
> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev);
> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
> + struct stf_isp_crop *crop_array, u32 mcode);
> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev);
> +int stf_isp_subdev_init(struct stfcamss *stfcamss);
> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev);
> +int stf_isp_unregister(struct stf_isp_dev *isp_dev);
> +
> +#endif /* STF_ISP_H */
> diff --git a/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> new file mode 100644
> index 000000000000..2088b6bd0d61
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> @@ -0,0 +1,468 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_isp_hw_ops.c
> + *
> + * Register interface file for StarFive ISP driver
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + *
> + */
> +
> +#include "stf_camss.h"
> +
> +static void stf_isp_config_obc(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG, OBC_W_H(11) | OBC_W_W(11));
> +
> + reg_val = GAIN_D_POINT(0x40) | GAIN_C_POINT(0x40) |
> + GAIN_B_POINT(0x40) | GAIN_A_POINT(0x40);
> + for (reg_add = ISP_REG_OBCG_CFG_0; reg_add <= ISP_REG_OBCG_CFG_3;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + reg_val = OFFSET_D_POINT(0) | OFFSET_C_POINT(0) |
> + OFFSET_B_POINT(0) | OFFSET_A_POINT(0);
> + for (reg_add = ISP_REG_OBCO_CFG_0; reg_add <= ISP_REG_OBCO_CFG_3;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_oecf(struct stfcamss *stfcamss)
> +{
> + u32 reg_add, par_val;
> + u16 par_h, par_l;
> +
> + par_h = 0x10; par_l = 0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG0; reg_add <= ISP_REG_OECF_Y3_CFG0;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x40; par_l = 0x20;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG1; reg_add <= ISP_REG_OECF_Y3_CFG1;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x80; par_l = 0x60;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG2; reg_add <= ISP_REG_OECF_Y3_CFG2;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0xc0; par_l = 0xa0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG3; reg_add <= ISP_REG_OECF_Y3_CFG3;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x100; par_l = 0xe0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG4; reg_add <= ISP_REG_OECF_Y3_CFG4;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x200; par_l = 0x180;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG5; reg_add <= ISP_REG_OECF_Y3_CFG5;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x300; par_l = 0x280;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG6; reg_add <= ISP_REG_OECF_Y3_CFG6;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x3fe; par_l = 0x380;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG7; reg_add <= ISP_REG_OECF_Y3_CFG7;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x80; par_l = 0x80;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_S0_CFG0; reg_add <= ISP_REG_OECF_S3_CFG7;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_lccf(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
> + Y_DISTANCE(0x21C) | X_DISTANCE(0x3C0));
> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1, LCCF_MAX_DIS(0xb));
> +
> + for (reg_add = ISP_REG_LCCF_CFG_2; reg_add <= ISP_REG_LCCF_CFG_5;) {
> + stf_isp_reg_write(stfcamss, reg_add,
> + LCCF_F2_PAR(0x0) | LCCF_F1_PAR(0x0));
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_awb(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> + u16 symbol_h, symbol_l;
> +
> + symbol_h = 0x0; symbol_l = 0x0;
> + reg_val = AWB_X_SYMBOL_H(symbol_h) | AWB_X_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_X0_CFG_0; reg_add <= ISP_REG_AWB_X3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + symbol_h = 0x0, symbol_l = 0x0;
> + reg_val = AWB_Y_SYMBOL_H(symbol_h) | AWB_Y_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_Y0_CFG_0; reg_add <= ISP_REG_AWB_Y3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + symbol_h = 0x80, symbol_l = 0x80;
> + reg_val = AWB_S_SYMBOL_H(symbol_h) | AWB_S_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_S0_CFG_0; reg_add <= ISP_REG_AWB_S3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_grgb(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
> + GF_MODE(1) | MAXGT(0x140) | MINGT(0x40));
> + stf_isp_reg_write(stfcamss, ISP_REG_IDBC, BADGT(0x200) | BADXT(0x200));
> +}
> +
> +static void stf_isp_config_cfa(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG,
> + SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) | SMY3(2) |
> + SMY2(3) | SMY1(2) | SMY0(3));
> + stf_isp_reg_write(stfcamss, ISP_REG_ICFAM, CROSS_COV(3) | HV_W(2));
> +}
> +
> +static void stf_isp_config_ccm(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6) | CCM_M_DAT(0));
> +
> + for (reg_add = ISP_REG_ICAMD_12; reg_add <= ISP_REG_ICAMD_20;) {
> + stf_isp_reg_write(stfcamss, reg_add, CCM_M_DAT(0x80));
> + reg_add += 0x10;
> + }
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_24, CCM_M_DAT(0x700));
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_25, CCM_M_DAT(0x200));
> +}
> +
> +static void stf_isp_config_gamma(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> + u16 gamma_slope_v, gamma_v;
> +
> + gamma_slope_v = 0x2400; gamma_v = 0x0;
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL0, reg_val);
> +
> + gamma_slope_v = 0x800; gamma_v = 0x20;
> + for (reg_add = ISP_REG_GAMMA_VAL1; reg_add <= ISP_REG_GAMMA_VAL7;) {
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + gamma_v += 0x20;
> + }
> +
> + gamma_v = 0x100;
> + for (reg_add = ISP_REG_GAMMA_VAL8; reg_add <= ISP_REG_GAMMA_VAL13;) {
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + gamma_v += 0x80;
> + }
> +
> + gamma_v = 0x3fe;
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL14, reg_val);
> +}
> +
> +static void stf_isp_config_r2y(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_0, CSC_M(0x4C));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_1, CSC_M(0x97));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_2, CSC_M(0x1d));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_3, CSC_M(0x1d5));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_4, CSC_M(0x1ac));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_5, CSC_M(0x80));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_6, CSC_M(0x80));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_7, CSC_M(0x194));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_8, CSC_M(0x1ec));
> +}
> +
> +static void stf_isp_config_y_curve(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> + u16 y_curve;
> +
> + y_curve = 0x0;
> + for (reg_add = ISP_REG_YCURVE_0; reg_add <= ISP_REG_YCURVE_63;) {
> + stf_isp_reg_write(stfcamss, reg_add, y_curve);
> + reg_add += 4;
> + y_curve += 0x10;
> + }
> +}
> +
> +static void stf_isp_config_sharpen(struct stfcamss *sc)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN0, S_DELTA(0x7) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN1, S_DELTA(0x18) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN2, S_DELTA(0x80) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN3, S_DELTA(0x100) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN4, S_DELTA(0x10) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN5, S_DELTA(0x60) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN6, S_DELTA(0x100) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN7, S_DELTA(0x190) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN8, S_DELTA(0x0) | S_WEIGHT(0xf));
> +
> + for (reg_add = ISP_REG_SHARPEN9; reg_add <= ISP_REG_SHARPEN14;) {
> + stf_isp_reg_write(sc, reg_add, S_WEIGHT(0xf));
> + reg_add += 4;
> + }
> +
> + for (reg_add = ISP_REG_SHARPEN_FS0; reg_add <= ISP_REG_SHARPEN_FS5;) {
> + stf_isp_reg_write(sc, reg_add, S_FACTOR(0x10) | S_SLOPE(0x0));
> + reg_add += 4;
> + }
> +
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN_WN,
> + PDIRF(0x8) | NDIRF(0x8) | WSUM(0xd7c));
> + stf_isp_reg_write(sc, ISP_REG_IUVS1, UVDIFF2(0xC0) | UVDIFF1(0x40));
> + stf_isp_reg_write(sc, ISP_REG_IUVS2, UVF(0xff) | UVSLOPE(0x0));
> + stf_isp_reg_write(sc, ISP_REG_IUVCKS1,
> + UVCKDIFF2(0xa0) | UVCKDIFF1(0x40));
> +}
> +
> +static void stf_isp_config_dnyuv(struct stfcamss *stfcamss)
> +{
> + u32 reg_val;
> +
> + reg_val = YUVSW5(7) | YUVSW4(7) | YUVSW3(7) | YUVSW2(7) |
> + YUVSW1(7) | YUVSW0(7);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0, reg_val);
> +
> + reg_val = YUVSW3(7) | YUVSW2(7) | YUVSW1(7) | YUVSW0(7);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1, reg_val);
> +
> + reg_val = CURVE_D_H(0x60) | CURVE_D_L(0x40);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0, reg_val);
> +
> + reg_val = CURVE_D_H(0xd8) | CURVE_D_L(0x90);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1, reg_val);
> +
> + reg_val = CURVE_D_H(0x1e6) | CURVE_D_L(0x144);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2, reg_val);
> +}
> +
> +static void stf_isp_config_sat(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN, CMAD(0x0) | CMAB(0x100));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD, CMD(0x1f) | CMB(0x1));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET, VOFF(0x0) | UOFF(0x0));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F, SIN(0x0) | COS(0x100));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, CMSF(0x8));
> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ0, YOIR(0x401) | YIMIN(0x1));
> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
> +}
> +
> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + clk_prepare_enable(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
> +
> + return 0;
> +}
> +
> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
> +
> + return 0;
> +}
> +
> +int stf_isp_reset(struct stf_isp_dev *isp_dev)
> +{
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_RST_MASK, ISPC_RST);
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_RST_MASK, 0);
> +
> + return 0;
> +}
> +
> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
> +{
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
> + DEC_V_KEEP(0x0) |
> + DEC_V_PERIOD(0x0) |
> + DEC_H_KEEP(0x0) |
> + DEC_H_PERIOD(0x0));
> +
> + stf_isp_config_obc(isp_dev->stfcamss);
> + stf_isp_config_oecf(isp_dev->stfcamss);
> + stf_isp_config_lccf(isp_dev->stfcamss);
> + stf_isp_config_awb(isp_dev->stfcamss);
> + stf_isp_config_grgb(isp_dev->stfcamss);
> + stf_isp_config_cfa(isp_dev->stfcamss);
> + stf_isp_config_ccm(isp_dev->stfcamss);
> + stf_isp_config_gamma(isp_dev->stfcamss);
> + stf_isp_config_r2y(isp_dev->stfcamss);
> + stf_isp_config_y_curve(isp_dev->stfcamss);
> + stf_isp_config_sharpen(isp_dev->stfcamss);
> + stf_isp_config_dnyuv(isp_dev->stfcamss);
> + stf_isp_config_sat(isp_dev->stfcamss);
> +
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
> +}
> +
> +static void stf_isp_config_crop(struct stfcamss *stfcamss,
> + struct stf_isp_crop *crop)
> +{
> + struct v4l2_rect *rect = &crop[STF_ISP_PAD_SRC].rect;
> + u32 bpp = crop[STF_ISP_PAD_SRC].bpp;
> + u32 val;
> +
> + val = VSTART_CAP(rect->top) | HSTART_CAP(rect->left);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_START_CFG, val);
> +
> + val = VEND_CAP(rect->height + rect->top - 1) |
> + HEND_CAP(rect->width + rect->left - 1);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_END_CFG, val);
> +
> + val = H_ACT_CAP(rect->height) | W_ACT_CAP(rect->width);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIPELINE_XY_SIZE, val);
> +
> + val = ALIGN(rect->width * bpp / 8, STFCAMSS_FRAME_WIDTH_ALIGN_8);
> + stf_isp_reg_write(stfcamss, ISP_REG_STRIDE, val);
> +}
> +
> +static void stf_isp_config_raw_fmt(struct stfcamss *stfcamss, u32 mcode)
> +{
> + u32 val, val1;
> +
> + switch (mcode) {
> + case MEDIA_BUS_FMT_SRGGB10_1X10:
> + case MEDIA_BUS_FMT_SRGGB8_1X8:
> + /* 3 2 3 2 1 0 1 0 B Gb B Gb Gr R Gr R */
> + val = SMY13(3) | SMY12(2) | SMY11(3) | SMY10(2) |
> + SMY3(1) | SMY2(0) | SMY1(1) | SMY0(0);
> + val1 = CTRL_SAT(0x0);
> + break;
> + case MEDIA_BUS_FMT_SGRBG10_1X10:
> + case MEDIA_BUS_FMT_SGRBG8_1X8:
> + /* 2 3 2 3 0 1 0 1, Gb B Gb B R Gr R Gr */
> + val = SMY13(2) | SMY12(3) | SMY11(2) | SMY10(3) |
> + SMY3(0) | SMY2(1) | SMY1(0) | SMY0(1);
> + val1 = CTRL_SAT(0x2);
> + break;
> + case MEDIA_BUS_FMT_SGBRG10_1X10:
> + case MEDIA_BUS_FMT_SGBRG8_1X8:
> + /* 1 0 1 0 3 2 3 2, Gr R Gr R B Gb B Gb */
> + val = SMY13(1) | SMY12(0) | SMY11(1) | SMY10(0) |
> + SMY3(3) | SMY2(2) | SMY1(3) | SMY0(2);
> + val1 = CTRL_SAT(0x3);
> + break;
> + case MEDIA_BUS_FMT_SBGGR10_1X10:
> + case MEDIA_BUS_FMT_SBGGR8_1X8:
> + /* 0 1 0 1 2 3 2 3 R Gr R Gr Gb B Gb B */
> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
> + val1 = CTRL_SAT(0x1);
> + break;
> + default:
> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
> + val1 = CTRL_SAT(0x1);
> + break;
> + }
> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG, val);
> + stf_isp_reg_set_bit(stfcamss, ISP_REG_ISP_CTRL_1, CTRL_SAT_MASK, val1);
> +}
> +
> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
> + struct stf_isp_crop *crop, u32 mcode)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + stf_isp_config_crop(stfcamss, crop);
> + stf_isp_config_raw_fmt(stfcamss, mcode);
> +
> + stf_isp_reg_set_bit(stfcamss, ISP_REG_DUMP_CFG_1,
> + DUMP_BURST_LEN_MASK | DUMP_SD_MASK,
> + DUMP_BURST_LEN(3));
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIIWSR,
> + ITI_HSIZE(IMAGE_MAX_HEIGH) |
> + ITI_WSIZE(IMAGE_MAX_WIDTH));
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDWLSR, ITI_WSTRIDE(0x960));
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDRLSR, ITI_STRIDE_L(0x960));
> + stf_isp_reg_write(stfcamss, ISP_REG_SENSOR, 0x1);
> +}
> +
> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS | ISPC_RST, 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS, 10);
> + stf_isp_reg_write(stfcamss, ISP_REG_IESHD, SHAD_UP_M);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS | ISPC_EN, 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
> + CSI_INTS(1) | CSI_SHA_M(4), 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
> + CSI_INTS(2) | CSI_SHA_M(4), 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSI_INPUT_EN_AND_STATUS,
> + CSI_EN_S, 10);
> +}
Hi Hans,
Thank you for your comment.
On 2023/7/26 17:11, Hans Verkuil wrote:
> Hi Jack,
>
> On 19/06/2023 13:28, Jack Zhu wrote:
>> Add ISP driver for StarFive Camera Subsystem.
>>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../media/platform/starfive/camss/Makefile | 2 +
>> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
>> .../media/platform/starfive/camss/stf_camss.h | 3 +
>> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
>> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
>> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
>> 6 files changed, 1544 insertions(+), 3 deletions(-)
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>
> Just a quick note:
>
> When I compile this I get the following warnings:
>
> drivers/media/platform/starfive/camss/stf_camss.c: In function 'stfcamss_subdev_notifier_bound':
> drivers/media/platform/starfive/camss/stf_camss.c:217:23: warning: 'pad[0]' may be used uninitialized [-Wmaybe-uninitialized]
> 217 | ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> drivers/media/platform/starfive/camss/stf_camss.c:202:27: note: 'pad[0]' was declared here
> 202 | struct media_pad *pad[STF_PADS_NUM];
> | ^~~
> drivers/media/platform/starfive/camss/stf_isp.c: In function 'isp_try_format.isra':
> drivers/media/platform/starfive/camss/stf_isp.c:105:40: warning: 'formats' may be used uninitialized [-Wmaybe-uninitialized]
> 105 | const struct isp_format_table *formats;
> | ^~~~~~~
> drivers/media/platform/starfive/camss/stf_isp.c:105:40: note: 'formats' was declared here
> 105 | const struct isp_format_table *formats;
> | ^~~~~~~
> drivers/media/platform/starfive/camss/stf_isp.c:106:30: warning: 'rect' may be used uninitialized [-Wmaybe-uninitialized]
> 106 | struct stf_isp_crop *rect;
> | ^~~~
> drivers/media/platform/starfive/camss/stf_isp.c:106:30: note: 'rect' was declared here
> 106 | struct stf_isp_crop *rect;
> | ^~~~
>
> That's when I build with: make W=1 KCFLAGS=-Wmaybe-uninitialized
>
OK, I will fix these warnings.
> Regards,
>
> Hans
>
>>
--
Regards,
Jack Zhu
On 19/06/2023 12:28, Jack Zhu wrote:
> Add basic platform driver for StarFive Camera Subsystem.
Another nit here.
"Add basic driver" isn't a great title. Recommend "Add core driver".
Basic implies a thing that will be added to or improved upon but, this
code here is your "core" interface where you do your probing and
resource parsing/allocation etc.
Use "core" not "basic" to describe this.
---
bod
On 19/06/2023 12:28, Jack Zhu wrote:
> +static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
> +{
> + struct device *dev = stfcamss->dev;
> + struct device_node *node = NULL;
> + int ret, num_subdevs = 0;
> +
> + for_each_endpoint_of_node(dev->of_node, node) {
> + struct stfcamss_async_subdev *csd;
> +
> + if (!of_device_is_available(node))
> + continue;
> +
> + csd = v4l2_async_nf_add_fwnode_remote(&stfcamss->notifier,
> + of_fwnode_handle(node),
> + struct stfcamss_async_subdev);
> + if (IS_ERR(csd)) {
> + ret = PTR_ERR(csd);
> + goto err_cleanup;
> + }
> +
> + ret = stfcamss_of_parse_endpoint_node(dev, node, csd);
> + if (ret < 0)
> + goto err_cleanup;
> +
> + num_subdevs++;
> + }
> +
> + return num_subdevs;
> +
> +err_cleanup:
> + of_node_put(node);
Where is the _get() for this and if you are releasing it on the error
path when is the _get() released on the non-error path ?
> + return ret;
> +}
> +
> +static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct media_pad *pad[STF_PADS_NUM];
> + unsigned int i, pad_num = 0;
> +
> + for (i = 0; i < pad_num; ++i) {
Does this loop ever run ?
I don't see how it can..
---
bod
On 19/06/2023 12:28, Jack Zhu wrote:
> Add starfive_camss.rst file that documents the Starfive Camera
> Subsystem driver which is used for handing image sensor data.
>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
> .../media/starfive_camss_graph.dot | 16 ++++++
> .../admin-guide/media/v4l-drivers.rst | 1 +
> MAINTAINERS | 1 +
> 4 files changed, 75 insertions(+)
> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
>
> diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
> new file mode 100644
> index 000000000000..a6378849384f
> --- /dev/null
> +++ b/Documentation/admin-guide/media/starfive_camss.rst
> @@ -0,0 +1,57 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. include:: <isonum.txt>
> +
> +================================
> +Starfive Camera Subsystem driver
> +================================
> +
> +Introduction
> +------------
> +
> +This file documents the driver for the Starfive Camera Subsystem found on
> +Starfive JH7110 SoC. The driver is located under drivers/media/platform/
> +starfive.
> +
> +The driver implements V4L2, Media controller and v4l2_subdev interfaces.
> +Camera sensor using V4L2 subdev interface in the kernel is supported.
> +
> +The driver has been successfully used on the Gstreamer 1.18.5 with
> +v4l2src plugin.
> +
> +
> +Starfive Camera Subsystem hardware
> +----------------------------------
> +
> +The Starfive Camera Subsystem hardware consists of:
> +
> +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd
be consistent with one - recommend MIPI throughout your documentation.
> +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
> + based camera sensor data stream.
> +- ISP: handles the image data streams from the MIPI CSIRx Controller.
Maybe you've done this elsewhere but, it would be worthwhile describing
what the ISP does. Debayering ? 3As ? Just raw dumps ?
> +- VIN(Video In): a top-level module, is responsible for controlling power
> + and clocks to other modules, dumps the input data to memory or transfers the
> + input data to ISP.
> +
> +
> +Topology
> +--------
> +
> +The media controller pipeline graph is as follows:
> +
> +.. _starfive_camss_graph:
> +
> +.. kernel-figure:: starfive_camss_graph.dot
> + :alt: starfive_camss_graph.dot
> + :align: center
> +
> +The driver has 2 video devices:
> +
> +- stf_vin0_wr_video0: capture device for images directly from the VIN module.
OK so you can get the raw images.
> +- stf_vin0_isp0_video1: capture device for images without scaling.
> +
> +The driver has 3 subdevices:
> +
> +- stf_isp0: is responsible for all the isp operations.
> +- stf_vin0_wr: used to dump RAW images to memory.
> +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
But what is being output here ? RGB, jpeg, YUV ?
It would be worth adding a few bits of text to describe that so you
don't have to dive into code to understand it.
---
bod
Hi Bryan,
Thank you for your comments.
On 2023/7/26 18:58, Bryan O'Donoghue wrote:
> On 19/06/2023 12:28, Jack Zhu wrote:
>> Add basic platform driver for StarFive Camera Subsystem.
>
> Another nit here.
>
> "Add basic driver" isn't a great title. Recommend "Add core driver".
>
> Basic implies a thing that will be added to or improved upon but, this code here is your "core" interface where you do your probing and resource parsing/allocation etc.
>
> Use "core" not "basic" to describe this.
>
Ok, I will modify this title.
> ---
> bod
>
--
Regards,
Jack Zhu
Hi Bryan,
Thank you for your comments.
On 2023/7/26 19:26, Bryan O'Donoghue wrote:
> On 19/06/2023 12:28, Jack Zhu wrote:
>> Add starfive_camss.rst file that documents the Starfive Camera
>> Subsystem driver which is used for handing image sensor data.
>>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
>> .../media/starfive_camss_graph.dot | 16 ++++++
>> .../admin-guide/media/v4l-drivers.rst | 1 +
>> MAINTAINERS | 1 +
>> 4 files changed, 75 insertions(+)
>> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
>> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
>>
>> diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
>> new file mode 100644
>> index 000000000000..a6378849384f
>> --- /dev/null
>> +++ b/Documentation/admin-guide/media/starfive_camss.rst
>> @@ -0,0 +1,57 @@
>> +.. SPDX-License-Identifier: GPL-2.0
>> +
>> +.. include:: <isonum.txt>
>> +
>> +================================
>> +Starfive Camera Subsystem driver
>> +================================
>> +
>> +Introduction
>> +------------
>> +
>> +This file documents the driver for the Starfive Camera Subsystem found on
>> +Starfive JH7110 SoC. The driver is located under drivers/media/platform/
>> +starfive.
>> +
>> +The driver implements V4L2, Media controller and v4l2_subdev interfaces.
>> +Camera sensor using V4L2 subdev interface in the kernel is supported.
>> +
>> +The driver has been successfully used on the Gstreamer 1.18.5 with
>> +v4l2src plugin.
>> +
>> +
>> +Starfive Camera Subsystem hardware
>> +----------------------------------
>> +
>> +The Starfive Camera Subsystem hardware consists of:
>> +
>> +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
>
> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd be consistent with one - recommend MIPI throughout your documentation.
>
Okay, I will keep a consistent description.
>> +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
>> + based camera sensor data stream.
>> +- ISP: handles the image data streams from the MIPI CSIRx Controller.
>
> Maybe you've done this elsewhere but, it would be worthwhile describing what the ISP does. Debayering ? 3As ? Just raw dumps ?
>
Ok, I will add more descriptions in the next version.
>> +- VIN(Video In): a top-level module, is responsible for controlling power
>> + and clocks to other modules, dumps the input data to memory or transfers the
>> + input data to ISP.
>> +
>> +
>> +Topology
>> +--------
>> +
>> +The media controller pipeline graph is as follows:
>> +
>> +.. _starfive_camss_graph:
>> +
>> +.. kernel-figure:: starfive_camss_graph.dot
>> + :alt: starfive_camss_graph.dot
>> + :align: center
>> +
>> +The driver has 2 video devices:
>> +
>> +- stf_vin0_wr_video0: capture device for images directly from the VIN module.
>
> OK so you can get the raw images.
>
>> +- stf_vin0_isp0_video1: capture device for images without scaling.
>> +
>> +The driver has 3 subdevices:
>> +
>> +- stf_isp0: is responsible for all the isp operations.
>> +- stf_vin0_wr: used to dump RAW images to memory.
>> +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
>
> But what is being output here ? RGB, jpeg, YUV ?
> It would be worth adding a few bits of text to describe that so you don't have to dive into code to understand it.
>
Ok, I will add more descriptions.
> ---
> bod
--
Regards,
Jack Zhu
Hi Bryan,
Thank you for your comments.
On 2023/7/26 18:55, Bryan O'Donoghue wrote:
> On 19/06/2023 12:28, Jack Zhu wrote:
>
>> +static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
>> +{
>> + struct device *dev = stfcamss->dev;
>> + struct device_node *node = NULL;
>> + int ret, num_subdevs = 0;
>> +
>> + for_each_endpoint_of_node(dev->of_node, node) {
>> + struct stfcamss_async_subdev *csd;
>> +
>> + if (!of_device_is_available(node))
>> + continue;
>> +
>> + csd = v4l2_async_nf_add_fwnode_remote(&stfcamss->notifier,
>> + of_fwnode_handle(node),
>> + struct stfcamss_async_subdev);
>> + if (IS_ERR(csd)) {
>> + ret = PTR_ERR(csd);
>> + goto err_cleanup;
>> + }
>> +
>> + ret = stfcamss_of_parse_endpoint_node(dev, node, csd);
>> + if (ret < 0)
>> + goto err_cleanup;
>> +
>> + num_subdevs++;
>> + }
>> +
>> + return num_subdevs;
>> +
>> +err_cleanup:
>> + of_node_put(node);
>
> Where is the _get() for this and if you are releasing it on the error path when is the _get() released on the non-error path ?
>
OK, I will fix it.
>> + return ret;
>> +}
>> +
>> +static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
>> + struct v4l2_subdev *subdev,
>> + struct v4l2_async_subdev *asd)
>> +{
>> + struct media_pad *pad[STF_PADS_NUM];
>> + unsigned int i, pad_num = 0;
>> +
>> + for (i = 0; i < pad_num; ++i) {
>
> Does this loop ever run ?
> I don't see how it can..
>
OK, I will fix it, although there are modifications to this code in patch 5 and patch 6.
> ---
> bod
--
Regards,
Jack Zhu
On 19/06/2023 13:28, Jack Zhu wrote:
> Add video driver for StarFive Camera Subsystem.
>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../media/platform/starfive/camss/Makefile | 4 +-
> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
> .../media/platform/starfive/camss/stf_video.h | 92 +++
> 3 files changed, 819 insertions(+), 1 deletion(-)
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
>
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> index d56ddd078a71..eb457917a914 100644
> --- a/drivers/media/platform/starfive/camss/Makefile
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -3,6 +3,8 @@
> # Makefile for StarFive Camera Subsystem driver
> #
>
> -starfive-camss-objs += stf_camss.o
> +starfive-camss-objs += \
> + stf_camss.o \
> + stf_video.o
>
> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
> new file mode 100644
> index 000000000000..2e6472fe51c6
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_video.c
> @@ -0,0 +1,724 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_video.c
> + *
> + * StarFive Camera Subsystem - V4L2 device node
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "stf_camss.h"
> +#include "stf_video.h"
> +
> +static const struct stfcamss_format_info formats_pix_wr[] = {
> + {
> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SRGGB10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SGRBG10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SGBRG10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SBGGR10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> +};
> +
> +static const struct stfcamss_format_info formats_pix_isp[] = {
> + {
> + .code = MEDIA_BUS_FMT_Y12_1X12,
> + .pixelformat = V4L2_PIX_FMT_NV12,
> + .planes = 2,
> + .vsub = { 1, 2 },
> + .bpp = 8,
> + },
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * Helper functions
> + */
> +
> +static int video_find_format(u32 code, u32 pixelformat,
> + struct stfcamss_video *video)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < video->nformats; ++i) {
> + if (video->formats[i].code == code &&
> + video->formats[i].pixelformat == pixelformat)
> + return i;
> + }
> +
> + for (i = 0; i < video->nformats; ++i)
> + if (video->formats[i].code == code)
> + return i;
> +
> + for (i = 0; i < video->nformats; ++i)
> + if (video->formats[i].pixelformat == pixelformat)
> + return i;
> +
> + return -EINVAL;
> +}
> +
> +static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
> +{
> + struct v4l2_pix_format *pix;
> + const struct stfcamss_format_info *fi;
> + u32 width, height;
> + u32 bpl;
> + unsigned int i;
> +
> + pix = &f->fmt.pix;
> +
> + for (i = 0; i < video->nformats; i++)
> + if (pix->pixelformat == video->formats[i].pixelformat)
> + break;
> +
> + if (i == video->nformats)
> + i = 0; /* default format */
> +
> + fi = &video->formats[i];
> + width = pix->width;
> + height = pix->height;
> +
> + memset(pix, 0, sizeof(*pix));
> +
> + pix->pixelformat = fi->pixelformat;
> + pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
> + STFCAMSS_FRAME_MAX_WIDTH);
> + pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
> + STFCAMSS_FRAME_MAX_HEIGHT);
> + bpl = pix->width * fi->bpp / 8;
> + bpl = ALIGN(bpl, video->bpl_alignment);
> + pix->bytesperline = bpl;
> +
> + for (i = 0; i < fi->planes; ++i)
> + pix->sizeimage += bpl * pix->height / fi->vsub[i];
> +
> + pix->field = V4L2_FIELD_NONE;
> + pix->colorspace = V4L2_COLORSPACE_SRGB;
> + pix->flags = 0;
> + pix->ycbcr_enc =
> + V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
> + pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
> + pix->colorspace,
> + pix->ycbcr_enc);
> + pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
> +
> + return 0;
> +}
> +
> +static int stf_video_init_format(struct stfcamss_video *video)
> +{
> + int ret;
> + struct v4l2_format format = {
> + .type = video->type,
> + .fmt.pix = {
> + .width = 1920,
> + .height = 1080,
> + .pixelformat = V4L2_PIX_FMT_RGB565,
> + },
> + };
> +
> + ret = __video_try_fmt(video, &format);
> +
> + if (ret < 0)
> + return ret;
> +
> + video->active_fmt = format;
> +
> + return 0;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Video queue operations
> + */
> +
> +static int video_queue_setup(struct vb2_queue *q,
> + unsigned int *num_buffers,
> + unsigned int *num_planes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
> +
> + if (*num_planes) {
> + if (*num_planes != 1)
> + return -EINVAL;
> +
> + if (sizes[0] < format->sizeimage)
> + return -EINVAL;
> + }
> +
> + *num_planes = 1;
> + sizes[0] = format->sizeimage;
> + if (!sizes[0])
> + dev_err(video->stfcamss->dev,
> + "%s: error size is zero!!!\n", __func__);
> +
> + dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
> + *num_planes, sizes[0]);
> +
> + return 0;
> +}
> +
> +static int video_buf_init(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + struct stfcamss_buffer *buffer =
> + container_of(vbuf, struct stfcamss_buffer, vb);
> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> + dma_addr_t *paddr;
> +
> + paddr = vb2_plane_cookie(vb, 0);
> + buffer->addr[0] = *paddr;
> +
> + if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV61)
> + buffer->addr[1] =
> + buffer->addr[0] + fmt->bytesperline * fmt->height;
> +
> + return 0;
> +}
> +
> +static int video_buf_prepare(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> +
> + if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
> + dev_err(video->stfcamss->dev,
> + "sizeimage = %d, plane size = %d\n",
> + fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
> + return -EINVAL;
> + }
> + vb2_set_plane_payload(vb, 0, fmt->sizeimage);
> +
> + vbuf->field = V4L2_FIELD_NONE;
> +
> + return 0;
> +}
> +
> +static void video_buf_queue(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + struct stfcamss_buffer *buffer =
> + container_of(vbuf, struct stfcamss_buffer, vb);
> +
> + video->ops->queue_buffer(video, buffer);
> +}
> +
> +/*
> + * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
> + * @mbus: v4l2_mbus_framefmt format (input)
> + * @pix: v4l2_pix_format_mplane format (output)
> + * @f: a pointer to formats array element to be used for the conversion
> + * @alignment: bytesperline alignment value
> + *
> + * Fill the output pix structure with information from the input mbus format.
> + *
> + * Return 0 on success or a negative error code otherwise
> + */
> +static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
> + struct v4l2_pix_format *pix,
> + const struct stfcamss_format_info *f,
> + unsigned int alignment)
> +{
> + u32 bytesperline;
> + unsigned int i;
> +
> + memset(pix, 0, sizeof(*pix));
> + v4l2_fill_pix_format(pix, mbus);
> + pix->pixelformat = f->pixelformat;
> + bytesperline = pix->width * f->bpp / 8;
> + bytesperline = ALIGN(bytesperline, alignment);
> + pix->bytesperline = bytesperline;
> +
> + for (i = 0; i < f->planes; ++i)
> + pix->sizeimage += bytesperline * pix->height / f->vsub[i];
> +
> + return 0;
> +}
> +
> +static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
> + u32 *pad)
> +{
> + struct media_pad *remote;
> +
> + remote = media_pad_remote_pad_first(&video->pad);
> +
> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
> + return NULL;
> +
> + if (pad)
> + *pad = remote->index;
> +
> + return media_entity_to_v4l2_subdev(remote->entity);
> +}
> +
> +static int video_get_subdev_format(struct stfcamss_video *video,
> + struct v4l2_format *format)
> +{
> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
> + struct v4l2_subdev_format fmt;
> + struct v4l2_subdev *subdev;
> + u32 pixelformat;
> + u32 pad;
> + int ret;
> +
> + subdev = video_remote_subdev(video, &pad);
> + if (!subdev)
> + return -EPIPE;
> +
> + fmt.pad = pad;
> + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
> + if (ret)
> + return ret;
> +
> + pixelformat = pix->pixelformat;
> + ret = video_find_format(fmt.format.code, pixelformat, video);
> + if (ret < 0)
> + return ret;
> +
> + format->type = video->type;
> +
> + return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
> + &video->formats[ret], video->bpl_alignment);
> +}
> +
> +static int video_check_format(struct stfcamss_video *video)
> +{
> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
> + struct v4l2_format format;
> + struct v4l2_pix_format *sd_pix = &format.fmt.pix;
> + int ret;
> +
> + sd_pix->pixelformat = pix->pixelformat;
> + ret = video_get_subdev_format(video, &format);
> + if (ret < 0)
> + return ret;
> +
> + if (pix->pixelformat != sd_pix->pixelformat ||
> + pix->height != sd_pix->height ||
> + pix->width != sd_pix->width ||
> + pix->field != format.fmt.pix.field) {
> + dev_err(video->stfcamss->dev,
> + "not match:\n"
> + "pixelformat: 0x%x <-> 0x%x\n"
> + "height: %d <-> %d\n"
> + "field: %d <-> %d\n",
> + pix->pixelformat, sd_pix->pixelformat,
> + pix->height, sd_pix->height,
> + pix->field, format.fmt.pix.field);
> + return -EPIPE;
> + }
> +
> + return 0;
> +}
> +
> +static int video_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + struct video_device *vdev = &video->vdev;
> + struct media_entity *entity;
> + struct media_pad *pad;
> + struct v4l2_subdev *subdev;
> + int ret;
> +
> + ret = video_device_pipeline_start(vdev, &video->stfcamss->pipe);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to media_pipeline_start: %d\n", ret);
This doesn't call flush_buffers to return the buffers back to the QUEUED state.
You do that correctly elsewhere in this function, but you missed this case.
> + return ret;
> + }
> +
> + ret = video_check_format(video);
> + if (ret < 0)
> + goto error;
> +
> + ret = pm_runtime_resume_and_get(video->stfcamss->dev);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev, "power up failed %d\n", ret);
> + goto error;
> + }
> +
> + entity = &vdev->entity;
> + while (1) {
> + pad = &entity->pads[0];
> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> + break;
> +
> + pad = media_pad_remote_pad_first(pad);
> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> + break;
> +
> + entity = pad->entity;
> + subdev = media_entity_to_v4l2_subdev(entity);
> +
> + ret = v4l2_subdev_call(subdev, video, s_stream, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD)
> + goto err_pm_put;
> + }
> + return 0;
> +
> +err_pm_put:
> + pm_runtime_put(video->stfcamss->dev);
> +error:
> + video_device_pipeline_stop(vdev);
> + video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
> + return ret;
> +}
> +
> +static void video_stop_streaming(struct vb2_queue *q)
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + struct video_device *vdev = &video->vdev;
> + struct media_entity *entity;
> + struct media_pad *pad;
> + struct v4l2_subdev *subdev;
> + int ret;
> +
> + entity = &vdev->entity;
> + while (1) {
> + pad = &entity->pads[0];
> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> + break;
> +
> + pad = media_pad_remote_pad_first(pad);
> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> + break;
> +
> + entity = pad->entity;
> + subdev = media_entity_to_v4l2_subdev(entity);
> +
> + v4l2_subdev_call(subdev, video, s_stream, 0);
> + }
> +
> + ret = pm_runtime_put(video->stfcamss->dev);
> + if (ret < 0)
> + dev_err(video->stfcamss->dev, "power down failed:%d\n", ret);
> +
> + video_device_pipeline_stop(vdev);
> + video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
> +}
> +
> +static const struct vb2_ops stf_video_vb2_q_ops = {
> + .queue_setup = video_queue_setup,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .buf_init = video_buf_init,
> + .buf_prepare = video_buf_prepare,
> + .buf_queue = video_buf_queue,
> + .start_streaming = video_start_streaming,
> + .stop_streaming = video_stop_streaming,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 ioctls
> + */
> +
> +static int video_querycap(struct file *file, void *fh,
> + struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, "stf camss", sizeof(cap->driver));
> + strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static int video_get_pfmt_by_index(struct stfcamss_video *video, int ndx)
> +{
> + int i, j, k;
> +
> + /* find index "i" of "k"th unique pixelformat in formats array */
> + k = -1;
> + for (i = 0; i < video->nformats; i++) {
> + for (j = 0; j < i; j++) {
> + if (video->formats[i].pixelformat ==
> + video->formats[j].pixelformat)
> + break;
> + }
> +
> + if (j == i)
> + k++;
> +
> + if (k == ndx)
> + return i;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int video_get_pfmt_by_mcode(struct stfcamss_video *video, u32 mcode)
> +{
> + int i;
> +
> + for (i = 0; i < video->nformats; i++) {
> + if (video->formats[i].code == mcode)
> + return i;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int video_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int i;
> +
> + if (f->type != video->type)
> + return -EINVAL;
> + if (f->index >= video->nformats)
> + return -EINVAL;
> +
> + if (f->mbus_code) {
> + /* Each entry in formats[] table has unique mbus_code */
> + if (f->index > 0)
> + return -EINVAL;
> +
> + i = video_get_pfmt_by_mcode(video, f->mbus_code);
> + } else {
> + i = video_get_pfmt_by_index(video, f->index);
> + }
> +
> + if (i < 0)
> + return -EINVAL;
> +
> + f->pixelformat = video->formats[i].pixelformat;
> +
> + return 0;
> +}
> +
> +static int video_enum_framesizes(struct file *file, void *fh,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int i;
> +
> + if (fsize->index)
> + return -EINVAL;
> +
> + for (i = 0; i < video->nformats; i++) {
> + if (video->formats[i].pixelformat == fsize->pixel_format)
> + break;
> + }
> +
> + if (i == video->nformats)
> + return -EINVAL;
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> + fsize->stepwise.min_width = STFCAMSS_FRAME_MIN_WIDTH;
> + fsize->stepwise.max_width = STFCAMSS_FRAME_MAX_WIDTH;
> + fsize->stepwise.min_height = STFCAMSS_FRAME_MIN_HEIGHT;
> + fsize->stepwise.max_height = STFCAMSS_FRAME_MAX_HEIGHT;
> + fsize->stepwise.step_width = 1;
> + fsize->stepwise.step_height = 1;
> +
> + return 0;
> +}
> +
> +static int video_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> +
> + *f = video->active_fmt;
> +
> + return 0;
> +}
> +
> +static int video_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int ret;
> +
> + if (vb2_is_busy(&video->vb2_q))
> + return -EBUSY;
> +
> + ret = __video_try_fmt(video, f);
> + if (ret < 0)
> + return ret;
> +
> + video->active_fmt = *f;
> +
> + return 0;
> +}
> +
> +static int video_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> +
> + return __video_try_fmt(video, f);
> +}
> +
> +static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
> + .vidioc_querycap = video_querycap,
> + .vidioc_enum_fmt_vid_cap = video_enum_fmt,
> + .vidioc_enum_fmt_vid_out = video_enum_fmt,
> + .vidioc_enum_framesizes = video_enum_framesizes,
> + .vidioc_g_fmt_vid_cap = video_g_fmt,
> + .vidioc_s_fmt_vid_cap = video_s_fmt,
> + .vidioc_try_fmt_vid_cap = video_try_fmt,
> + .vidioc_g_fmt_vid_out = video_g_fmt,
> + .vidioc_s_fmt_vid_out = video_s_fmt,
> + .vidioc_try_fmt_vid_out = video_try_fmt,
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 file operations
> + */
> +
> +static const struct v4l2_file_operations stf_vid_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> + .read = vb2_fop_read,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * STFCAMSS video core
> + */
> +
> +static void stf_video_release(struct video_device *vdev)
> +{
> + struct stfcamss_video *video = video_get_drvdata(vdev);
> +
> + media_entity_cleanup(&vdev->entity);
> +
> + mutex_destroy(&video->q_lock);
> + mutex_destroy(&video->lock);
> +}
> +
> +int stf_video_register(struct stfcamss_video *video,
> + struct v4l2_device *v4l2_dev, const char *name)
> +{
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + struct media_pad *pad = &video->pad;
> + int ret;
> +
> + vdev = &video->vdev;
> +
> + mutex_init(&video->q_lock);
> +
> + q = &video->vb2_q;
> + q->drv_priv = video;
> + q->mem_ops = &vb2_dma_contig_memops;
> + q->ops = &stf_video_vb2_q_ops;
> + q->type = video->type;
> + q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
VB2_READ support does not generally make sense for uncompressed video since
read() always requires a memcpy, and that makes it very inefficient.
It doesn't hurt though, so it is up to you whether or not you want this.
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->buf_struct_size = sizeof(struct stfcamss_buffer);
> + q->dev = video->stfcamss->dev;
> + q->lock = &video->q_lock;
> + q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
> + ret = vb2_queue_init(q);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init vb2 queue: %d\n", ret);
> + goto err_vb2_init;
> + }
> +
> + pad->flags = MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(&vdev->entity, 1, pad);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init video entity: %d\n", ret);
> + goto err_vb2_init;
> + }
> +
> + mutex_init(&video->lock);
> +
> + if (video->id == STF_V_LINE_WR) {
> + video->formats = formats_pix_wr;
> + video->nformats = ARRAY_SIZE(formats_pix_wr);
> + video->bpl_alignment = 8;
> + } else {
> + video->formats = formats_pix_isp;
> + video->nformats = ARRAY_SIZE(formats_pix_isp);
> + video->bpl_alignment = 1;
> + }
> +
> + ret = stf_video_init_format(video);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init format: %d\n", ret);
> + goto err_vid_init_format;
> + }
> +
> + vdev->fops = &stf_vid_fops;
> + vdev->ioctl_ops = &stf_vid_ioctl_ops;
> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> + vdev->vfl_dir = VFL_DIR_RX;
> + vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
> + vdev->release = stf_video_release;
> + vdev->v4l2_dev = v4l2_dev;
> + vdev->queue = &video->vb2_q;
> + vdev->lock = &video->lock;
> + strscpy(vdev->name, name, sizeof(vdev->name));
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to register video device: %d\n", ret);
> + goto err_vid_reg;
> + }
> +
> + video_set_drvdata(vdev, video);
> + return 0;
> +
> +err_vid_reg:
> +err_vid_init_format:
> + media_entity_cleanup(&vdev->entity);
> + mutex_destroy(&video->lock);
> +err_vb2_init:
> + mutex_destroy(&video->q_lock);
> + return ret;
> +}
> +
> +void stf_video_unregister(struct stfcamss_video *video)
> +{
> + vb2_video_unregister_device(&video->vdev);
> +}
> diff --git a/drivers/media/platform/starfive/camss/stf_video.h b/drivers/media/platform/starfive/camss/stf_video.h
> new file mode 100644
> index 000000000000..60323c23a40c
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_video.h
> @@ -0,0 +1,92 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_video.h
> + *
> + * StarFive Camera Subsystem - V4L2 device node
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_VIDEO_H
> +#define STF_VIDEO_H
> +
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-fh.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define STFCAMSS_FRAME_MIN_WIDTH 64
> +#define STFCAMSS_FRAME_MAX_WIDTH 1920
> +#define STFCAMSS_FRAME_MIN_HEIGHT 64
> +#define STFCAMSS_FRAME_MAX_HEIGHT 1080
> +#define STFCAMSS_FRAME_WIDTH_ALIGN_8 8
> +#define STFCAMSS_FRAME_WIDTH_ALIGN_128 128
> +#define STFCAMSS_MIN_BUFFERS 2
> +
> +#define STFCAMSS_MAX_ENTITY_NAME_LEN 27
> +
> +enum stf_v_line_id {
> + STF_V_LINE_WR = 0,
> + STF_V_LINE_ISP,
> + STF_V_LINE_MAX,
> +};
> +
> +struct stfcamss_buffer {
> + struct vb2_v4l2_buffer vb;
> + dma_addr_t addr[3];
> + struct list_head queue;
> +};
> +
> +struct fract {
> + u8 numerator;
> + u8 denominator;
> +};
> +
> +/*
> + * struct stfcamss_format_info - ISP media bus format information
> + * @code: V4L2 media bus format code
> + * @pixelformat: V4L2 pixel format FCC identifier
> + * @planes: Number of planes
> + * @vsub: Vertical subsampling (for each plane)
> + * @bpp: Bits per pixel when stored in memory (for each plane)
> + */
> +struct stfcamss_format_info {
> + u32 code;
> + u32 pixelformat;
> + u8 planes;
> + u8 vsub[3];
> + u8 bpp;
> +};
> +
> +struct stfcamss_video {
> + struct stfcamss *stfcamss;
> + u8 id;
> + struct vb2_queue vb2_q;
> + struct video_device vdev;
> + struct media_pad pad;
> + struct v4l2_format active_fmt;
> + enum v4l2_buf_type type;
> + const struct stfcamss_video_ops *ops;
> + struct mutex lock; /* serialize device access */
> + struct mutex q_lock; /* protects the queue */
> + unsigned int bpl_alignment;
> + const struct stfcamss_format_info *formats;
> + unsigned int nformats;
> +};
> +
> +struct stfcamss_video_ops {
> + int (*queue_buffer)(struct stfcamss_video *vid,
> + struct stfcamss_buffer *buf);
> + int (*flush_buffers)(struct stfcamss_video *vid,
> + enum vb2_buffer_state state);
> +};
> +
> +int stf_video_register(struct stfcamss_video *video,
> + struct v4l2_device *v4l2_dev, const char *name);
> +
> +void stf_video_unregister(struct stfcamss_video *video);
> +
> +#endif /* STF_VIDEO_H */
Regards,
Hans
On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote:
> On 19/06/2023 12:28, Jack Zhu wrote:
> > Add starfive_camss.rst file that documents the Starfive Camera
> > Subsystem driver which is used for handing image sensor data.
> >
> > Signed-off-by: Jack Zhu <[email protected]>
> > ---
> > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
> > .../media/starfive_camss_graph.dot | 16 ++++++
> > .../admin-guide/media/v4l-drivers.rst | 1 +
> > MAINTAINERS | 1 +
> > 4 files changed, 75 insertions(+)
> > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
> > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
> >
> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
> > new file mode 100644
> > index 000000000000..a6378849384f
> > --- /dev/null
> > +++ b/Documentation/admin-guide/media/starfive_camss.rst
> > @@ -0,0 +1,57 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +.. include:: <isonum.txt>
> > +
> > +================================
> > +Starfive Camera Subsystem driver
> > +================================
> > +
> > +Introduction
> > +------------
> > +
> > +This file documents the driver for the Starfive Camera Subsystem found on
> > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/
> > +starfive.
> > +
> > +The driver implements V4L2, Media controller and v4l2_subdev interfaces.
> > +Camera sensor using V4L2 subdev interface in the kernel is supported.
> > +
> > +The driver has been successfully used on the Gstreamer 1.18.5 with
> > +v4l2src plugin.
> > +
> > +
> > +Starfive Camera Subsystem hardware
> > +----------------------------------
> > +
> > +The Starfive Camera Subsystem hardware consists of:
> > +
> > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
>
> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd
> be consistent with one - recommend MIPI throughout your documentation.
I'd drop the "mipi" and just write "data", not "MIPI data".
> > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
It's "CSI-2", not "CSI2".
> > + based camera sensor data stream.
> > +- ISP: handles the image data streams from the MIPI CSIRx Controller.
>
> Maybe you've done this elsewhere but, it would be worthwhile describing
> what the ISP does. Debayering ? 3As ? Just raw dumps ?
>
> > +- VIN(Video In): a top-level module, is responsible for controlling power
> > + and clocks to other modules, dumps the input data to memory or transfers the
> > + input data to ISP.
A block diagram could be nice. Let me give it a try, based on my
understanding:
Starfive Camera Subsystem hardware
----------------------------------
The Starfive Camera Subsystem hardware consists of::
+---------------------------------------------+
| VIN |
+--------+ | +-------+ +-------+ +-----+ |
| Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | |
| Sensor | | | RX | | RX | | +-----+ |
+--------+ | +-------+ +-------+ | |
| | +-----+ +-----+ |
| \->| ISP |-->| DMA | |
| +-----+ +-----+ |
+---------------------------------------------+
- VIN (Video IN): The top-level module, responsible for controlling
power and clocks, and routing data between modules.
- D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2
camera sensor.
- CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding
the CSI-2 protocol.
- ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and
producing YUV frames.
I haven't tried compiling this, so please fix formatting if anything is
wrong (and of course feel free to adapt the diagram and text).
> > +
> > +
> > +Topology
> > +--------
> > +
> > +The media controller pipeline graph is as follows:
> > +
> > +.. _starfive_camss_graph:
> > +
> > +.. kernel-figure:: starfive_camss_graph.dot
> > + :alt: starfive_camss_graph.dot
> > + :align: center
> > +
> > +The driver has 2 video devices:
> > +
> > +- stf_vin0_wr_video0: capture device for images directly from the VIN module.
>
> OK so you can get the raw images.
>
> > +- stf_vin0_isp0_video1: capture device for images without scaling.
> > +
> > +The driver has 3 subdevices:
> > +
> > +- stf_isp0: is responsible for all the isp operations.
> > +- stf_vin0_wr: used to dump RAW images to memory.
> > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
>
> But what is being output here ? RGB, jpeg, YUV ?
> It would be worth adding a few bits of text to describe that so you
> don't have to dive into code to understand it.
--
Regards,
Laurent Pinchart
Hi Jack,
Thank you for the patch.
On Mon, Jun 19, 2023 at 07:28:35PM +0800, Jack Zhu wrote:
> Add basic platform driver for StarFive Camera Subsystem.
>
> Reviewed-by: Bryan O'Donoghue <[email protected]>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> MAINTAINERS | 1 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/starfive/Kconfig | 5 +
> drivers/media/platform/starfive/Makefile | 2 +
> drivers/media/platform/starfive/camss/Kconfig | 16 +
> .../media/platform/starfive/camss/Makefile | 8 +
> .../media/platform/starfive/camss/stf_camss.c | 338 ++++++++++++++++++
> .../media/platform/starfive/camss/stf_camss.h | 146 ++++++++
> 9 files changed, 518 insertions(+)
> create mode 100644 drivers/media/platform/starfive/Kconfig
> create mode 100644 drivers/media/platform/starfive/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
> create mode 100644 drivers/media/platform/starfive/camss/Makefile
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 71291dc58671..62f9e3593eab 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -20077,6 +20077,7 @@ L: [email protected]
> S: Maintained
> F: Documentation/admin-guide/media/starfive_camss.rst
> F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> +F: drivers/media/platform/starfive/camss
>
> STARFIVE DEVICETREES
> M: Emil Renner Berthing <[email protected]>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ee579916f874..627eaa0ab3ee 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -80,6 +80,7 @@ source "drivers/media/platform/renesas/Kconfig"
> source "drivers/media/platform/rockchip/Kconfig"
> source "drivers/media/platform/samsung/Kconfig"
> source "drivers/media/platform/st/Kconfig"
> +source "drivers/media/platform/starfive/Kconfig"
> source "drivers/media/platform/sunxi/Kconfig"
> source "drivers/media/platform/ti/Kconfig"
> source "drivers/media/platform/verisilicon/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 5453bb868e67..5a038498a370 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -23,6 +23,7 @@ obj-y += renesas/
> obj-y += rockchip/
> obj-y += samsung/
> obj-y += st/
> +obj-y += starfive/
> obj-y += sunxi/
> obj-y += ti/
> obj-y += verisilicon/
> diff --git a/drivers/media/platform/starfive/Kconfig b/drivers/media/platform/starfive/Kconfig
> new file mode 100644
> index 000000000000..7955c2a0a4a3
> --- /dev/null
> +++ b/drivers/media/platform/starfive/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +comment "StarFive media platform drivers"
> +
> +source "drivers/media/platform/starfive/camss/Kconfig"
> diff --git a/drivers/media/platform/starfive/Makefile b/drivers/media/platform/starfive/Makefile
> new file mode 100644
> index 000000000000..4639fa1bca32
> --- /dev/null
> +++ b/drivers/media/platform/starfive/Makefile
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-y += camss/
> diff --git a/drivers/media/platform/starfive/camss/Kconfig b/drivers/media/platform/starfive/camss/Kconfig
> new file mode 100644
> index 000000000000..dafe1d24324b
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/Kconfig
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config VIDEO_STARFIVE_CAMSS
> + tristate "Starfive Camera Subsystem driver"
> + depends on V4L_PLATFORM_DRIVERS
> + depends on VIDEO_DEV && OF
> + depends on HAS_DMA
You need to depend on PM, otherwise the runtime PM operations will be
no-ops and the driver won't work as clocks won't be enabled.
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select VIDEOBUF2_DMA_CONTIG
> + select V4L2_FWNODE
> + help
> + Enable this to support for the Starfive Camera subsystem
> + found on Starfive JH7110 SoC.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called stf-camss.
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> new file mode 100644
> index 000000000000..d56ddd078a71
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for StarFive Camera Subsystem driver
> +#
> +
> +starfive-camss-objs += stf_camss.o
> +
> +obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> new file mode 100644
> index 000000000000..dc2b5dba7bd4
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> @@ -0,0 +1,338 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_camss.c
> + *
> + * Starfive Camera Subsystem driver
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-mc.h>
> +
> +#include "stf_camss.h"
> +
> +static const char * const stfcamss_clocks[] = {
> + "clk_apb_func",
> + "clk_wrapper_clk_c",
> + "clk_dvp_inv",
> + "clk_axiwr",
> + "clk_mipi_rx0_pxl",
> + "clk_ispcore_2x",
> + "clk_isp_axi",
> +};
> +
> +static const char * const stfcamss_resets[] = {
> + "rst_wrapper_p",
> + "rst_wrapper_c",
> + "rst_axird",
> + "rst_axiwr",
> + "rst_isp_top_n",
> + "rst_isp_top_axi",
> +};
> +
> +static int stfcamss_get_mem_res(struct platform_device *pdev,
> + struct stfcamss *stfcamss)
> +{
> + stfcamss->syscon_base =
> + devm_platform_ioremap_resource_byname(pdev, "syscon");
> + if (IS_ERR(stfcamss->syscon_base))
> + return PTR_ERR(stfcamss->syscon_base);
> +
> + stfcamss->isp_base =
> + devm_platform_ioremap_resource_byname(pdev, "isp");
This hols on a single line:
stfcamss->isp_base = devm_platform_ioremap_resource_byname(pdev, "isp");
I would do the same above too.
> + if (IS_ERR(stfcamss->isp_base))
> + return PTR_ERR(stfcamss->isp_base);
> +
> + return 0;
> +}
> +
> +/*
> + * stfcamss_of_parse_endpoint_node - Parse port endpoint node
> + * @dev: Device
> + * @node: Device node to be parsed
> + * @csd: Parsed data from port endpoint node
> + *
> + * Return 0 on success or a negative error code on failure
> + */
> +static int stfcamss_of_parse_endpoint_node(struct device *dev,
For consistency, pass the stfcamss pointer to this function, not the
struct device.
> + struct device_node *node,
> + struct stfcamss_async_subdev *csd)
> +{
> + struct v4l2_fwnode_endpoint vep = { { 0 } };
> +
> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(node), &vep);
> + dev_dbg(dev, "vep.base.port = %d, id = %d\n",
> + vep.base.port, vep.base.id);
There are debug messages in v4l2_fwnode_endpoint_parse() already, you
can drop this one.
> +
> + csd->port = vep.base.port;
> +
> + return 0;
> +}
> +
> +/*
> + * stfcamss_of_parse_ports - Parse ports node
> + * @stfcamss: STFCAMSS device
> + *
> + * Return number of "port" nodes found in "ports" node
> + */
> +static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
> +{
> + struct device *dev = stfcamss->dev;
> + struct device_node *node = NULL;
> + int ret, num_subdevs = 0;
> +
> + for_each_endpoint_of_node(dev->of_node, node) {
for_each_endpoint_of_node(stfcamss->dev->of_node, node) {
and drop the local dev variable.
> + struct stfcamss_async_subdev *csd;
> +
> + if (!of_device_is_available(node))
> + continue;
> +
> + csd = v4l2_async_nf_add_fwnode_remote(&stfcamss->notifier,
> + of_fwnode_handle(node),
> + struct stfcamss_async_subdev);
> + if (IS_ERR(csd)) {
> + ret = PTR_ERR(csd);
> + goto err_cleanup;
> + }
> +
> + ret = stfcamss_of_parse_endpoint_node(dev, node, csd);
> + if (ret < 0)
> + goto err_cleanup;
> +
> + num_subdevs++;
> + }
> +
> + return num_subdevs;
> +
> +err_cleanup:
> + of_node_put(node);
> + return ret;
> +}
> +
> +static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct media_pad *pad[STF_PADS_NUM];
> + unsigned int i, pad_num = 0;
> +
> + for (i = 0; i < pad_num; ++i) {
The loop never runs as pad_num is 0. This is modified in subsequent
patches in the series, but it makes it hard to review the patches
individually. Is there any way you could restructure the series to avoid
this ?
> + int ret;
> +
> + ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
> + if (ret < 0)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int stfcamss_subdev_notifier_complete(struct v4l2_async_notifier *ntf)
> +{
> + struct stfcamss *stfcamss =
> + container_of(ntf, struct stfcamss, notifier);
> +
> + return v4l2_device_register_subdev_nodes(&stfcamss->v4l2_dev);
> +}
> +
> +static const struct v4l2_async_notifier_operations
> +stfcamss_subdev_notifier_ops = {
> + .bound = stfcamss_subdev_notifier_bound,
> + .complete = stfcamss_subdev_notifier_complete,
> +};
> +
> +static const struct media_device_ops stfcamss_media_ops = {
> + .link_notify = v4l2_pipeline_link_notify,
You can drop this, link management notification only serves to call the
subdev .s_power() operations, which are a deprecated API.
> +};
> +
> +static void stfcamss_mc_init(struct platform_device *pdev,
> + struct stfcamss *stfcamss)
> +{
> + stfcamss->media_dev.dev = stfcamss->dev;
> + strscpy(stfcamss->media_dev.model, "Starfive Camera Subsystem",
> + sizeof(stfcamss->media_dev.model));
> + stfcamss->media_dev.ops = &stfcamss_media_ops;
> + media_device_init(&stfcamss->media_dev);
> +
> + stfcamss->v4l2_dev.mdev = &stfcamss->media_dev;
> +}
> +
> +/*
> + * stfcamss_probe - Probe STFCAMSS platform device
> + * @pdev: Pointer to STFCAMSS platform device
> + *
> + * Return 0 on success or a negative error code on failure
> + */
> +static int stfcamss_probe(struct platform_device *pdev)
> +{
> + struct stfcamss *stfcamss;
> + struct device *dev = &pdev->dev;
> + int ret, num_subdevs;
> + unsigned int i;
> +
> + stfcamss = devm_kzalloc(dev, sizeof(*stfcamss), GFP_KERNEL);
> + if (!stfcamss)
> + return -ENOMEM;
> +
> + for (i = 0; i < ARRAY_SIZE(stfcamss->irq); ++i) {
> + stfcamss->irq[i] = platform_get_irq(pdev, i);
> + if (stfcamss->irq[i] < 0)
> + return dev_err_probe(&pdev->dev, stfcamss->irq[i],
> + "Failed to get irq%d", i);
> + }
> +
> + stfcamss->nclks = ARRAY_SIZE(stfcamss->sys_clk);
> + for (i = 0; i < stfcamss->nclks; ++i)
> + stfcamss->sys_clk[i].id = stfcamss_clocks[i];
> + ret = devm_clk_bulk_get(dev, stfcamss->nclks, stfcamss->sys_clk);
> + if (ret) {
> + dev_err(dev, "Failed to get clk controls\n");
> + return ret;
> + }
> +
> + stfcamss->nrsts = ARRAY_SIZE(stfcamss->sys_rst);
> + for (i = 0; i < stfcamss->nrsts; ++i)
> + stfcamss->sys_rst[i].id = stfcamss_resets[i];
> + ret = devm_reset_control_bulk_get_shared(dev, stfcamss->nrsts,
> + stfcamss->sys_rst);
> + if (ret) {
> + dev_err(dev, "Failed to get reset controls\n");
> + return ret;
> + }
> +
> + ret = stfcamss_get_mem_res(pdev, stfcamss);
> + if (ret) {
> + dev_err(dev, "Could not map registers\n");
> + return ret;
> + }
> +
> + stfcamss->dev = dev;
Move this right after allocating stfcamss, and drop the pdev argument to
stfcamss_get_mem_res(). The platform device can be retrieved in the
function using to_platform_device().
> + platform_set_drvdata(pdev, stfcamss);
> +
> + v4l2_async_nf_init(&stfcamss->notifier);
> +
> + num_subdevs = stfcamss_of_parse_ports(stfcamss);
> + if (num_subdevs < 0) {
> + ret = -ENODEV;
An error message would be useful, silent errors are hard to debug.
> + goto err_cleanup_notifier;
> + }
> +
> + stfcamss_mc_init(pdev, stfcamss);
> +
> + ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
> + goto err_cleanup_notifier;
> + }
> +
> + ret = media_device_register(&stfcamss->media_dev);
> + if (ret) {
> + dev_err(dev, "Failed to register media device: %d\n", ret);
> + goto err_unregister_device;
> + }
> +
> + pm_runtime_enable(dev);
Would it be useful to enable autosuspend too, to avoid expensive
suspend/resume cycles when userspace wants to briefly stop capture and
restart it immediately ?
> +
> + stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
> + ret = v4l2_async_nf_register(&stfcamss->v4l2_dev, &stfcamss->notifier);
> + if (ret) {
> + dev_err(dev, "Failed to register async subdev nodes: %d\n",
> + ret);
> + goto err_unregister_media_dev;
You need to disable runtime PM in this error path.
> + }
> +
> + return 0;
> +
> +err_unregister_media_dev:
> + media_device_unregister(&stfcamss->media_dev);
> +err_unregister_device:
> + v4l2_device_unregister(&stfcamss->v4l2_dev);
> +err_cleanup_notifier:
> + v4l2_async_nf_cleanup(&stfcamss->notifier);
> + return ret;
> +}
> +
> +/*
> + * stfcamss_remove - Remove STFCAMSS platform device
> + * @pdev: Pointer to STFCAMSS platform device
> + *
> + * Always returns 0.
> + */
> +static int stfcamss_remove(struct platform_device *pdev)
> +{
> + struct stfcamss *stfcamss = platform_get_drvdata(pdev);
> +
> + v4l2_device_unregister(&stfcamss->v4l2_dev);
> + media_device_cleanup(&stfcamss->media_dev);
> + pm_runtime_disable(&pdev->dev);
> +
> + return 0;
> +}
> +
> +static const struct of_device_id stfcamss_of_match[] = {
> + { .compatible = "starfive,jh7110-camss" },
> + { /* sentinel */ },
> +};
> +
> +MODULE_DEVICE_TABLE(of, stfcamss_of_match);
> +
> +static int __maybe_unused stfcamss_runtime_suspend(struct device *dev)
> +{
> + struct stfcamss *stfcamss = dev_get_drvdata(dev);
> +
> + reset_control_assert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
> + reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
> + reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
The reset bulk API (reset_control_bulk_assert() and
reset_control_bulk_deassert() for below) would help.
> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
Same for clocks, possibly.
> +
> + return 0;
> +}
> +
> +static int __maybe_unused stfcamss_runtime_resume(struct device *dev)
> +{
> + struct stfcamss *stfcamss = dev_get_drvdata(dev);
> + int ret;
> +
> + ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
> + if (ret)
> + return ret;
> +
> + ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
> + if (ret)
You need to disable the STF_CLK_ISPCORE_2X clock here.
> + return ret;
> +
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
These calls can fail too.
Error handling would be easier with the reset and clk bulk APIs.
> +
> + return 0;
> +}
> +
> +static const struct dev_pm_ops stfcamss_pm_ops = {
> + SET_RUNTIME_PM_OPS(stfcamss_runtime_suspend,
> + stfcamss_runtime_resume,
> + NULL)
> +};
> +
> +static struct platform_driver stfcamss_driver = {
> + .probe = stfcamss_probe,
> + .remove = stfcamss_remove,
> + .driver = {
> + .name = "starfive-camss",
> + .pm = &stfcamss_pm_ops,
> + .of_match_table = of_match_ptr(stfcamss_of_match),
The driver depends on OF, so let's drop the of_match_ptr().
> + },
> +};
> +
> +module_platform_driver(stfcamss_driver);
> +
> +MODULE_AUTHOR("StarFive Corporation");
> +MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
> new file mode 100644
> index 000000000000..15c4f34b9054
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
> @@ -0,0 +1,146 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_camss.h
> + *
> + * Starfive Camera Subsystem driver
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_CAMSS_H
> +#define STF_CAMSS_H
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/reset.h>
> +#include <media/media-device.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-device.h>
> +
> +#define STF_DVP_NAME "stf_dvp"
> +#define STF_CSI_NAME "cdns_csi2rx"
> +#define STF_ISP_NAME "stf_isp"
> +#define STF_VIN_NAME "stf_vin"
> +
> +#define STF_PAD_SINK 0
> +#define STF_PAD_SRC 1
> +#define STF_PADS_NUM 2
> +
> +enum stf_port_num {
> + STF_PORT_DVP = 0,
> + STF_PORT_CSI2RX
> +};
> +
> +enum stf_clk {
> + STF_CLK_APB_FUNC = 0,
> + STF_CLK_WRAPPER_CLK_C,
> + STF_CLK_DVP_INV,
> + STF_CLK_AXIWR,
> + STF_CLK_MIPI_RX0_PXL,
> + STF_CLK_ISPCORE_2X,
> + STF_CLK_ISP_AXI,
> + STF_CLK_NUM
> +};
> +
> +enum stf_rst {
> + STF_RST_WRAPPER_P = 0,
> + STF_RST_WRAPPER_C,
> + STF_RST_AXIRD,
> + STF_RST_AXIWR,
> + STF_RST_ISP_TOP_N,
> + STF_RST_ISP_TOP_AXI,
> + STF_RST_NUM
> +};
> +
> +enum stf_irq {
> + STF_IRQ_VINWR = 0,
> + STF_IRQ_ISP,
> + STF_IRQ_ISPCSIL,
> + STF_IRQ_NUM
> +};
> +
> +struct stfcamss {
> + struct v4l2_device v4l2_dev;
> + struct media_device media_dev;
> + struct media_pipeline pipe;
> + struct device *dev;
> + struct v4l2_async_notifier notifier;
> + void __iomem *syscon_base;
> + void __iomem *isp_base;
> + int irq[STF_IRQ_NUM];
> + struct clk_bulk_data sys_clk[STF_CLK_NUM];
> + int nclks;
> + struct reset_control_bulk_data sys_rst[STF_RST_NUM];
> + int nrsts;
> +};
> +
> +struct stfcamss_async_subdev {
> + struct v4l2_async_subdev asd; /* must be first */
> + enum stf_port_num port;
> +};
> +
> +static inline u32 stf_isp_reg_read(struct stfcamss *stfcamss, u32 reg)
> +{
> + return ioread32(stfcamss->isp_base + reg);
> +}
> +
> +static inline void stf_isp_reg_write(struct stfcamss *stfcamss,
> + u32 reg, u32 val)
> +{
> + iowrite32(val, stfcamss->isp_base + reg);
> +}
> +
> +static inline void stf_isp_reg_write_delay(struct stfcamss *stfcamss,
> + u32 reg, u32 val, u32 delay)
> +{
> + iowrite32(val, stfcamss->isp_base + reg);
> + usleep_range(1000 * delay, 1000 * delay + 100);
> +}
> +
> +static inline void stf_isp_reg_set_bit(struct stfcamss *stfcamss,
> + u32 reg, u32 mask, u32 val)
> +{
> + u32 value;
> +
> + value = ioread32(stfcamss->isp_base + reg) & ~mask;
> + val &= mask;
> + val |= value;
> + iowrite32(val, stfcamss->isp_base + reg);
> +}
> +
> +static inline void stf_isp_reg_set(struct stfcamss *stfcamss, u32 reg, u32 mask)
> +{
> + iowrite32(ioread32(stfcamss->isp_base + reg) | mask,
> + stfcamss->isp_base + reg);
> +}
> +
> +static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32 reg)
> +{
> + return ioread32(stfcamss->syscon_base + reg);
> +}
> +
> +static inline void stf_syscon_reg_write(struct stfcamss *stfcamss,
> + u32 reg, u32 val)
> +{
> + iowrite32(val, stfcamss->syscon_base + reg);
> +}
> +
> +static inline void stf_syscon_reg_set_bit(struct stfcamss *stfcamss,
> + u32 reg, u32 bit_mask)
> +{
> + u32 value;
> +
> + value = ioread32(stfcamss->syscon_base + reg);
> + iowrite32(value | bit_mask, stfcamss->syscon_base + reg);
> +}
> +
> +static inline void stf_syscon_reg_clear_bit(struct stfcamss *stfcamss,
> + u32 reg, u32 bit_mask)
> +{
> + u32 value;
> +
> + value = ioread32(stfcamss->syscon_base + reg);
> + iowrite32(value & ~bit_mask, stfcamss->syscon_base + reg);
> +}
> +#endif /* STF_CAMSS_H */
--
Regards,
Laurent Pinchart
Hi Jack,
Thank you for the patch.
On Mon, Jun 19, 2023 at 07:28:36PM +0800, Jack Zhu wrote:
> Add video driver for StarFive Camera Subsystem.
>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../media/platform/starfive/camss/Makefile | 4 +-
> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
> .../media/platform/starfive/camss/stf_video.h | 92 +++
> 3 files changed, 819 insertions(+), 1 deletion(-)
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
>
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> index d56ddd078a71..eb457917a914 100644
> --- a/drivers/media/platform/starfive/camss/Makefile
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -3,6 +3,8 @@
> # Makefile for StarFive Camera Subsystem driver
> #
>
> -starfive-camss-objs += stf_camss.o
> +starfive-camss-objs += \
> + stf_camss.o \
> + stf_video.o
>
> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
> new file mode 100644
> index 000000000000..2e6472fe51c6
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_video.c
> @@ -0,0 +1,724 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_video.c
> + *
> + * StarFive Camera Subsystem - V4L2 device node
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "stf_camss.h"
> +#include "stf_video.h"
> +
> +static const struct stfcamss_format_info formats_pix_wr[] = {
> + {
> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SRGGB10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SGRBG10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SGBRG10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> + {
> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .pixelformat = V4L2_PIX_FMT_SBGGR10,
> + .planes = 1,
> + .vsub = { 1 },
> + .bpp = 10,
> + },
> +};
> +
> +static const struct stfcamss_format_info formats_pix_isp[] = {
> + {
> + .code = MEDIA_BUS_FMT_Y12_1X12,
> + .pixelformat = V4L2_PIX_FMT_NV12,
> + .planes = 2,
> + .vsub = { 1, 2 },
> + .bpp = 8,
> + },
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * Helper functions
> + */
> +
> +static int video_find_format(u32 code, u32 pixelformat,
> + struct stfcamss_video *video)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < video->nformats; ++i) {
> + if (video->formats[i].code == code &&
> + video->formats[i].pixelformat == pixelformat)
> + return i;
> + }
> +
> + for (i = 0; i < video->nformats; ++i)
> + if (video->formats[i].code == code)
> + return i;
> +
> + for (i = 0; i < video->nformats; ++i)
> + if (video->formats[i].pixelformat == pixelformat)
> + return i;
> +
This looks weird, I don't think it does what you expect below. I think
you can drop the function, and instead use video_get_pfmt_by_mcode() to
convert the mbus code to a pixel format, and compare it to the active
pixel format in video_check_format().
> + return -EINVAL;
> +}
> +
> +static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
> +{
> + struct v4l2_pix_format *pix;
> + const struct stfcamss_format_info *fi;
> + u32 width, height;
> + u32 bpl;
> + unsigned int i;
> +
> + pix = &f->fmt.pix;
You can initialize pix when declaring it.
> +
> + for (i = 0; i < video->nformats; i++)
> + if (pix->pixelformat == video->formats[i].pixelformat)
> + break;
> +
for (i = 0; i < video->nformats; i++) {
if (pix->pixelformat == video->formats[i].pixelformat)
break;
}
But a helper function that looks up a format by pixelformat, similar to
video_get_pfmt_by_mcode(), would be useful. I think I would make all
those helpers return a const struct stfcamss_format_info pointer instead
of an index.
> + if (i == video->nformats)
> + i = 0; /* default format */
> +
> + fi = &video->formats[i];
> + width = pix->width;
> + height = pix->height;
> +
> + memset(pix, 0, sizeof(*pix));
> +
> + pix->pixelformat = fi->pixelformat;
> + pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
> + STFCAMSS_FRAME_MAX_WIDTH);
> + pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
> + STFCAMSS_FRAME_MAX_HEIGHT);
> + bpl = pix->width * fi->bpp / 8;
> + bpl = ALIGN(bpl, video->bpl_alignment);
> + pix->bytesperline = bpl;
Does the hardware support configuring the stride ?
> +
> + for (i = 0; i < fi->planes; ++i)
> + pix->sizeimage += bpl * pix->height / fi->vsub[i];
> +
> + pix->field = V4L2_FIELD_NONE;
> + pix->colorspace = V4L2_COLORSPACE_SRGB;
> + pix->flags = 0;
> + pix->ycbcr_enc =
> + V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
> + pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
> + pix->colorspace,
> + pix->ycbcr_enc);
> + pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
This doesn't seem right for the processed output.
> +
> + return 0;
> +}
> +
> +static int stf_video_init_format(struct stfcamss_video *video)
> +{
> + int ret;
> + struct v4l2_format format = {
> + .type = video->type,
> + .fmt.pix = {
> + .width = 1920,
> + .height = 1080,
> + .pixelformat = V4L2_PIX_FMT_RGB565,
That format doesn't seem supported, let's pick V4L2_PIX_FMT_NV12.
> + },
> + };
> +
> + ret = __video_try_fmt(video, &format);
> +
> + if (ret < 0)
> + return ret;
> +
> + video->active_fmt = format;
> +
> + return 0;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Video queue operations
> + */
> +
> +static int video_queue_setup(struct vb2_queue *q,
> + unsigned int *num_buffers,
> + unsigned int *num_planes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
> +
> + if (*num_planes) {
> + if (*num_planes != 1)
> + return -EINVAL;
> +
> + if (sizes[0] < format->sizeimage)
> + return -EINVAL;
> + }
> +
> + *num_planes = 1;
> + sizes[0] = format->sizeimage;
> + if (!sizes[0])
> + dev_err(video->stfcamss->dev,
> + "%s: error size is zero!!!\n", __func__);
Shouldn't you return an error ? Also, use dev_dbg(), printing an error
message based on a condition that can easily be triggered by
unpriviledge userspace opens the door to applications flooding the
kernel log.
> +
> + dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
> + *num_planes, sizes[0]);
> +
> + return 0;
> +}
> +
> +static int video_buf_init(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + struct stfcamss_buffer *buffer =
> + container_of(vbuf, struct stfcamss_buffer, vb);
A static inline to_stfcamss_buffer() function that wraps the
container_of() would be nice. You can use it below too.
> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> + dma_addr_t *paddr;
> +
> + paddr = vb2_plane_cookie(vb, 0);
> + buffer->addr[0] = *paddr;
> +
> + if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
> + fmt->pixelformat == V4L2_PIX_FMT_NV61)
Only V4L2_PIX_FMT_NV12 is listed in formats_pix_isp. Does the hardware
support the other formats ? If so, it would be nice to support them
already.
> + buffer->addr[1] =
> + buffer->addr[0] + fmt->bytesperline * fmt->height;
As the hardware supports non-contiguous planes, you should use the
MPLANE API (V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) and support the NV*M
formats in addition to the NV* formats.
> +
> + return 0;
> +}
> +
> +static int video_buf_prepare(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> +
> + if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
> + dev_err(video->stfcamss->dev,
dev_dbg() here too.
> + "sizeimage = %d, plane size = %d\n",
> + fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
Both are unsigned, use %u instead of %d.
> + return -EINVAL;
> + }
> + vb2_set_plane_payload(vb, 0, fmt->sizeimage);
> +
> + vbuf->field = V4L2_FIELD_NONE;
> +
> + return 0;
> +}
> +
> +static void video_buf_queue(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> + struct stfcamss_buffer *buffer =
> + container_of(vbuf, struct stfcamss_buffer, vb);
> +
> + video->ops->queue_buffer(video, buffer);
> +}
> +
> +/*
> + * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
> + * @mbus: v4l2_mbus_framefmt format (input)
> + * @pix: v4l2_pix_format_mplane format (output)
> + * @f: a pointer to formats array element to be used for the conversion
> + * @alignment: bytesperline alignment value
> + *
> + * Fill the output pix structure with information from the input mbus format.
> + *
> + * Return 0 on success or a negative error code otherwise
> + */
> +static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
> + struct v4l2_pix_format *pix,
> + const struct stfcamss_format_info *f,
> + unsigned int alignment)
> +{
> + u32 bytesperline;
> + unsigned int i;
> +
> + memset(pix, 0, sizeof(*pix));
> + v4l2_fill_pix_format(pix, mbus);
> + pix->pixelformat = f->pixelformat;
> + bytesperline = pix->width * f->bpp / 8;
> + bytesperline = ALIGN(bytesperline, alignment);
> + pix->bytesperline = bytesperline;
> +
> + for (i = 0; i < f->planes; ++i)
> + pix->sizeimage += bytesperline * pix->height / f->vsub[i];
This function is used for validation of the format only, the
bytesperline and sizeimage values are never used. You can simplify the
driver by dropping the function and comparing the width, height and
field of the subdev and video device from the v4l2_mbus_framefmt and
v4l2_pix_format respectively in video_check_format().
video_get_subdev_format() will then take a v4l2_mbus_framefmt pointer,
not a v4l2_pix_format.
The format match check still needs conversion of the
To check the format, you need to convert the mbus code from the subdev
to a pixel format using the
> +
> + return 0;
> +}
> +
> +static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
> + u32 *pad)
> +{
> + struct media_pad *remote;
> +
> + remote = media_pad_remote_pad_first(&video->pad);
> +
> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
> + return NULL;
> +
> + if (pad)
> + *pad = remote->index;
> +
> + return media_entity_to_v4l2_subdev(remote->entity);
As the connected subdev is always the same (the CSI-2 RX for the raw
capture video device and the ISP for the processed capture video
device), I would store a pointer to the connected subdev in the
stfcamss_video structure at registration time. You can pass the pointer
to the stf_video_register() function.
> +}
> +
> +static int video_get_subdev_format(struct stfcamss_video *video,
> + struct v4l2_format *format)
> +{
> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
> + struct v4l2_subdev_format fmt;
> + struct v4l2_subdev *subdev;
> + u32 pixelformat;
> + u32 pad;
> + int ret;
> +
> + subdev = video_remote_subdev(video, &pad);
> + if (!subdev)
> + return -EPIPE;
> +
> + fmt.pad = pad;
> + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
Use v4l2_subdev_call_state_active() to support the subdev state API.
> + if (ret)
> + return ret;
> +
> + pixelformat = pix->pixelformat;
> + ret = video_find_format(fmt.format.code, pixelformat, video);
> + if (ret < 0)
> + return ret;
> +
> + format->type = video->type;
> +
> + return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
> + &video->formats[ret], video->bpl_alignment);
> +}
> +
> +static int video_check_format(struct stfcamss_video *video)
> +{
> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
> + struct v4l2_format format;
> + struct v4l2_pix_format *sd_pix = &format.fmt.pix;
> + int ret;
> +
> + sd_pix->pixelformat = pix->pixelformat;
> + ret = video_get_subdev_format(video, &format);
> + if (ret < 0)
> + return ret;
> +
> + if (pix->pixelformat != sd_pix->pixelformat ||
> + pix->height != sd_pix->height ||
> + pix->width != sd_pix->width ||
> + pix->field != format.fmt.pix.field) {
> + dev_err(video->stfcamss->dev,
> + "not match:\n"
Let's be a bit clearer: "Format mismatch between ISP and video device".
> + "pixelformat: 0x%x <-> 0x%x\n"
> + "height: %d <-> %d\n"
> + "field: %d <-> %d\n",
Use dev_dbg(), and make the message a single line (without '\n'), or it
will mess up formatting in the kernel log.
> + pix->pixelformat, sd_pix->pixelformat,
> + pix->height, sd_pix->height,
> + pix->field, format.fmt.pix.field);
> + return -EPIPE;
> + }
> +
> + return 0;
> +}
> +
> +static int video_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + struct video_device *vdev = &video->vdev;
> + struct media_entity *entity;
> + struct media_pad *pad;
> + struct v4l2_subdev *subdev;
> + int ret;
> +
> + ret = video_device_pipeline_start(vdev, &video->stfcamss->pipe);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to media_pipeline_start: %d\n", ret);
flush_buffers() should be called here to release the buffers that have
been queued before starting streaming.
> + return ret;
> + }
> +
> + ret = video_check_format(video);
> + if (ret < 0)
> + goto error;
> +
> + ret = pm_runtime_resume_and_get(video->stfcamss->dev);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev, "power up failed %d\n", ret);
> + goto error;
> + }
> +
> + entity = &vdev->entity;
> + while (1) {
> + pad = &entity->pads[0];
> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> + break;
> +
> + pad = media_pad_remote_pad_first(pad);
> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> + break;
> +
> + entity = pad->entity;
> + subdev = media_entity_to_v4l2_subdev(entity);
> +
> + ret = v4l2_subdev_call(subdev, video, s_stream, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD)
> + goto err_pm_put;
> + }
The .s_stream() operation should be called recursively, not on all
subdevs in the graph. This function should call .s_stream() on the ISP,
which should call it on the CSI-2 receiver subdev, which should then
call it on its connected subdev. The reason for this is that you don't
know what devices are connected to the CSI-2 RX, it could be just a
camera sensor, but it could also be a more complex chain of devices (for
instance, a sensor, an automotive serializer and a deserializer). Each
device would have its own requirements regarding what operations to
perform before and after starting and stopping its source, so each
subdev should call .s_stream() on its own source.
Same in video_stop_streaming().
> + return 0;
> +
> +err_pm_put:
> + pm_runtime_put(video->stfcamss->dev);
> +error:
> + video_device_pipeline_stop(vdev);
> + video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
> + return ret;
> +}
> +
> +static void video_stop_streaming(struct vb2_queue *q)
> +{
> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> + struct video_device *vdev = &video->vdev;
> + struct media_entity *entity;
> + struct media_pad *pad;
> + struct v4l2_subdev *subdev;
> + int ret;
> +
> + entity = &vdev->entity;
> + while (1) {
> + pad = &entity->pads[0];
> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> + break;
> +
> + pad = media_pad_remote_pad_first(pad);
> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> + break;
> +
> + entity = pad->entity;
> + subdev = media_entity_to_v4l2_subdev(entity);
> +
> + v4l2_subdev_call(subdev, video, s_stream, 0);
> + }
> +
> + ret = pm_runtime_put(video->stfcamss->dev);
> + if (ret < 0)
> + dev_err(video->stfcamss->dev, "power down failed:%d\n", ret);
> +
> + video_device_pipeline_stop(vdev);
> + video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
> +}
> +
> +static const struct vb2_ops stf_video_vb2_q_ops = {
> + .queue_setup = video_queue_setup,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .buf_init = video_buf_init,
> + .buf_prepare = video_buf_prepare,
> + .buf_queue = video_buf_queue,
> + .start_streaming = video_start_streaming,
> + .stop_streaming = video_stop_streaming,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 ioctls
> + */
> +
> +static int video_querycap(struct file *file, void *fh,
> + struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, "stf camss", sizeof(cap->driver));
> + strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static int video_get_pfmt_by_index(struct stfcamss_video *video, int ndx)
> +{
> + int i, j, k;
> +
> + /* find index "i" of "k"th unique pixelformat in formats array */
> + k = -1;
> + for (i = 0; i < video->nformats; i++) {
> + for (j = 0; j < i; j++) {
> + if (video->formats[i].pixelformat ==
> + video->formats[j].pixelformat)
> + break;
> + }
> +
> + if (j == i)
> + k++;
> +
> + if (k == ndx)
> + return i;
> + }
This looks unnecessarily complicated, as all entries in the format array
has unique pixel formats.
> +
> + return -EINVAL;
> +}
> +
> +static int video_get_pfmt_by_mcode(struct stfcamss_video *video, u32 mcode)
> +{
> + int i;
> +
> + for (i = 0; i < video->nformats; i++) {
> + if (video->formats[i].code == mcode)
> + return i;
> + }
> +
> + return -EINVAL;
> +}
Move these two helper functions above, just below video_find_format().
> +
> +static int video_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int i;
> +
> + if (f->type != video->type)
> + return -EINVAL;
I think you can skip this. check_fmt() in v4l2-ioctl.c checks that the
driver implements the .vidioc_g_fmt_* operation corresponding to the
type, which should catch invalid types.
> + if (f->index >= video->nformats)
> + return -EINVAL;
> +
> + if (f->mbus_code) {
> + /* Each entry in formats[] table has unique mbus_code */
> + if (f->index > 0)
> + return -EINVAL;
> +
> + i = video_get_pfmt_by_mcode(video, f->mbus_code);
> + } else {
> + i = video_get_pfmt_by_index(video, f->index);
> + }
> +
> + if (i < 0)
> + return -EINVAL;
> +
> + f->pixelformat = video->formats[i].pixelformat;
> +
> + return 0;
> +}
> +
> +static int video_enum_framesizes(struct file *file, void *fh,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int i;
i never takes negative values, you can make it an unsigned int.
> +
> + if (fsize->index)
> + return -EINVAL;
> +
> + for (i = 0; i < video->nformats; i++) {
> + if (video->formats[i].pixelformat == fsize->pixel_format)
> + break;
> + }
> +
> + if (i == video->nformats)
> + return -EINVAL;
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> + fsize->stepwise.min_width = STFCAMSS_FRAME_MIN_WIDTH;
> + fsize->stepwise.max_width = STFCAMSS_FRAME_MAX_WIDTH;
> + fsize->stepwise.min_height = STFCAMSS_FRAME_MIN_HEIGHT;
> + fsize->stepwise.max_height = STFCAMSS_FRAME_MAX_HEIGHT;
> + fsize->stepwise.step_width = 1;
> + fsize->stepwise.step_height = 1;
> +
> + return 0;
> +}
> +
> +static int video_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> +
> + *f = video->active_fmt;
> +
> + return 0;
> +}
> +
> +static int video_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> + int ret;
> +
> + if (vb2_is_busy(&video->vb2_q))
> + return -EBUSY;
> +
> + ret = __video_try_fmt(video, f);
> + if (ret < 0)
> + return ret;
> +
> + video->active_fmt = *f;
> +
> + return 0;
> +}
> +
> +static int video_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct stfcamss_video *video = video_drvdata(file);
> +
> + return __video_try_fmt(video, f);
> +}
> +
> +static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
> + .vidioc_querycap = video_querycap,
> + .vidioc_enum_fmt_vid_cap = video_enum_fmt,
> + .vidioc_enum_fmt_vid_out = video_enum_fmt,
As far as I can tell the driver doesn't support video output (processing
with the ISP data read from memory), so you can drop this.
> + .vidioc_enum_framesizes = video_enum_framesizes,
> + .vidioc_g_fmt_vid_cap = video_g_fmt,
> + .vidioc_s_fmt_vid_cap = video_s_fmt,
> + .vidioc_try_fmt_vid_cap = video_try_fmt,
> + .vidioc_g_fmt_vid_out = video_g_fmt,
> + .vidioc_s_fmt_vid_out = video_s_fmt,
> + .vidioc_try_fmt_vid_out = video_try_fmt,
These three operations can be dropped too.
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 file operations
> + */
> +
> +static const struct v4l2_file_operations stf_vid_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> + .read = vb2_fop_read,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * STFCAMSS video core
> + */
> +
> +static void stf_video_release(struct video_device *vdev)
> +{
> + struct stfcamss_video *video = video_get_drvdata(vdev);
> +
> + media_entity_cleanup(&vdev->entity);
> +
> + mutex_destroy(&video->q_lock);
> + mutex_destroy(&video->lock);
> +}
> +
> +int stf_video_register(struct stfcamss_video *video,
> + struct v4l2_device *v4l2_dev, const char *name)
> +{
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + struct media_pad *pad = &video->pad;
> + int ret;
> +
> + vdev = &video->vdev;
You can assign vdev above when declaring it.
> +
> + mutex_init(&video->q_lock);
> +
> + q = &video->vb2_q;
> + q->drv_priv = video;
> + q->mem_ops = &vb2_dma_contig_memops;
> + q->ops = &stf_video_vb2_q_ops;
> + q->type = video->type;
> + q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
Drop VB2_READ, that's inefficient and deprecated for uncompressed
formats.
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->buf_struct_size = sizeof(struct stfcamss_buffer);
> + q->dev = video->stfcamss->dev;
> + q->lock = &video->q_lock;
> + q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
> + ret = vb2_queue_init(q);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init vb2 queue: %d\n", ret);
> + goto err_vb2_init;
Error labels should be named based on the cleaning action being
performed, not the function that failed. This should be
err_mutex_destroy (or just err_mutex).
> + }
> +
> + pad->flags = MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(&vdev->entity, 1, pad);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init video entity: %d\n", ret);
> + goto err_vb2_init;
> + }
> +
> + mutex_init(&video->lock);
I'd move this above, with the other mutex_init() call.
> +
> + if (video->id == STF_V_LINE_WR) {
> + video->formats = formats_pix_wr;
> + video->nformats = ARRAY_SIZE(formats_pix_wr);
> + video->bpl_alignment = 8;
> + } else {
> + video->formats = formats_pix_isp;
> + video->nformats = ARRAY_SIZE(formats_pix_isp);
> + video->bpl_alignment = 1;
> + }
> +
> + ret = stf_video_init_format(video);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to init format: %d\n", ret);
> + goto err_vid_init_format;
And this label should be err_media_cleanup.
> + }
> +
> + vdev->fops = &stf_vid_fops;
> + vdev->ioctl_ops = &stf_vid_ioctl_ops;
> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> + vdev->vfl_dir = VFL_DIR_RX;
> + vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
Drop V4L2_CAP_READWRITE for the same reason as VB2_READ.
> + vdev->release = stf_video_release;
> + vdev->v4l2_dev = v4l2_dev;
> + vdev->queue = &video->vb2_q;
> + vdev->lock = &video->lock;
> + strscpy(vdev->name, name, sizeof(vdev->name));
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
> + if (ret < 0) {
> + dev_err(video->stfcamss->dev,
> + "Failed to register video device: %d\n", ret);
> + goto err_vid_reg;
> + }
> +
> + video_set_drvdata(vdev, video);
Call this before video_register_device(). As soon as
video_register_device() gets called, userspace can access the device, so
it needs to be fully initialized.
> + return 0;
> +
> +err_vid_reg:
> +err_vid_init_format:
> + media_entity_cleanup(&vdev->entity);
> + mutex_destroy(&video->lock);
> +err_vb2_init:
> + mutex_destroy(&video->q_lock);
> + return ret;
> +}
> +
> +void stf_video_unregister(struct stfcamss_video *video)
> +{
> + vb2_video_unregister_device(&video->vdev);
> +}
> diff --git a/drivers/media/platform/starfive/camss/stf_video.h b/drivers/media/platform/starfive/camss/stf_video.h
> new file mode 100644
> index 000000000000..60323c23a40c
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_video.h
> @@ -0,0 +1,92 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_video.h
> + *
> + * StarFive Camera Subsystem - V4L2 device node
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_VIDEO_H
> +#define STF_VIDEO_H
> +
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/videodev2.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-fh.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define STFCAMSS_FRAME_MIN_WIDTH 64
> +#define STFCAMSS_FRAME_MAX_WIDTH 1920
> +#define STFCAMSS_FRAME_MIN_HEIGHT 64
> +#define STFCAMSS_FRAME_MAX_HEIGHT 1080
> +#define STFCAMSS_FRAME_WIDTH_ALIGN_8 8
> +#define STFCAMSS_FRAME_WIDTH_ALIGN_128 128
> +#define STFCAMSS_MIN_BUFFERS 2
> +
> +#define STFCAMSS_MAX_ENTITY_NAME_LEN 27
> +
> +enum stf_v_line_id {
> + STF_V_LINE_WR = 0,
> + STF_V_LINE_ISP,
> + STF_V_LINE_MAX,
> +};
> +
> +struct stfcamss_buffer {
> + struct vb2_v4l2_buffer vb;
> + dma_addr_t addr[3];
The driver supports two planes at most (for NV formats), not three.
> + struct list_head queue;
> +};
> +
> +struct fract {
> + u8 numerator;
> + u8 denominator;
> +};
> +
> +/*
> + * struct stfcamss_format_info - ISP media bus format information
> + * @code: V4L2 media bus format code
> + * @pixelformat: V4L2 pixel format FCC identifier
> + * @planes: Number of planes
> + * @vsub: Vertical subsampling (for each plane)
> + * @bpp: Bits per pixel when stored in memory (for each plane)
> + */
> +struct stfcamss_format_info {
> + u32 code;
> + u32 pixelformat;
> + u8 planes;
> + u8 vsub[3];
> + u8 bpp;
> +};
> +
> +struct stfcamss_video {
> + struct stfcamss *stfcamss;
> + u8 id;
> + struct vb2_queue vb2_q;
> + struct video_device vdev;
> + struct media_pad pad;
> + struct v4l2_format active_fmt;
> + enum v4l2_buf_type type;
> + const struct stfcamss_video_ops *ops;
> + struct mutex lock; /* serialize device access */
> + struct mutex q_lock; /* protects the queue */
> + unsigned int bpl_alignment;
> + const struct stfcamss_format_info *formats;
> + unsigned int nformats;
> +};
> +
> +struct stfcamss_video_ops {
> + int (*queue_buffer)(struct stfcamss_video *vid,
> + struct stfcamss_buffer *buf);
> + int (*flush_buffers)(struct stfcamss_video *vid,
> + enum vb2_buffer_state state);
> +};
> +
> +int stf_video_register(struct stfcamss_video *video,
> + struct v4l2_device *v4l2_dev, const char *name);
> +
> +void stf_video_unregister(struct stfcamss_video *video);
> +
> +#endif /* STF_VIDEO_H */
--
Regards,
Laurent Pinchart
Hi Jack,
Thank you for the patch.
On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
> Add Video In Controller driver for StarFive Camera Subsystem.
I haven't reviewed this patch in details, as I have a high-level
question: why do you need VIN subdevs ? They don't seem to represent any
particular piece of hardware, their input and output formats are always
identical, and they're not used to configure registers. The contents of
this patch seems to belong to the video device, I think you can drop the
VIN subdevs.
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../media/platform/starfive/camss/Makefile | 4 +-
> .../media/platform/starfive/camss/stf_camss.c | 42 +-
> .../media/platform/starfive/camss/stf_camss.h | 2 +
> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
> .../media/platform/starfive/camss/stf_vin.h | 173 +++
> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
> 6 files changed, 1478 insertions(+), 4 deletions(-)
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> index cdf57e8c9546..ef574e01ca47 100644
> --- a/drivers/media/platform/starfive/camss/Makefile
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -7,6 +7,8 @@ starfive-camss-objs += \
> stf_camss.o \
> stf_isp.o \
> stf_isp_hw_ops.o \
> - stf_video.o
> + stf_video.o \
> + stf_vin.o \
> + stf_vin_hw_ops.o
>
> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> index 6f56b45f57db..834ea63eb833 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.c
> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
> return ret;
> }
>
> + ret = stf_vin_subdev_init(stfcamss);
> + if (ret < 0) {
> + dev_err(stfcamss->dev, "Failed to init vin subdev: %d\n", ret);
> + return ret;
> + }
> return ret;
> }
>
> static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
> {
> int ret;
> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>
> ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
> if (ret < 0) {
> dev_err(stfcamss->dev,
> "Failed to register stf isp%d entity: %d\n", 0, ret);
> - return ret;
> + goto err_reg_isp;
> + }
> +
> + ret = stf_vin_register(vin_dev, &stfcamss->v4l2_dev);
> + if (ret < 0) {
> + dev_err(stfcamss->dev,
> + "Failed to register vin entity: %d\n", ret);
> + goto err_reg_vin;
> }
>
> + ret = media_create_pad_link(&isp_dev->subdev.entity,
> + STF_ISP_PAD_SRC,
> + &vin_dev->line[VIN_LINE_ISP].subdev.entity,
> + STF_VIN_PAD_SINK,
> + 0);
> + if (ret < 0) {
> + dev_err(stfcamss->dev,
> + "Failed to link %s->%s entities: %d\n",
> + isp_dev->subdev.entity.name,
> + vin_dev->line[VIN_LINE_ISP].subdev.entity.name, ret);
> + goto err_link;
> + }
> +
> + return ret;
> +
> +err_link:
> + stf_vin_unregister(&stfcamss->vin_dev);
> +err_reg_vin:
> + stf_isp_unregister(&stfcamss->isp_dev);
> +err_reg_isp:
> return ret;
> }
>
> static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
> {
> stf_isp_unregister(&stfcamss->isp_dev);
> + stf_vin_unregister(&stfcamss->vin_dev);
> }
>
> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> @@ -164,12 +198,14 @@ static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> container_of(asd, struct stfcamss_async_subdev, asd);
> enum stf_port_num port = csd->port;
> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> struct media_pad *pad[STF_PADS_NUM];
> unsigned int i, pad_num;
>
> if (port == STF_PORT_CSI2RX) {
> - pad[0] = &isp_dev->pads[STF_PAD_SINK];
> - pad_num = 1;
> + pad[0] = &vin_dev->line[VIN_LINE_WR].pads[STF_PAD_SINK];
> + pad[1] = &isp_dev->pads[STF_PAD_SINK];
> + pad_num = 2;
> } else if (port == STF_PORT_DVP) {
> dev_err(stfcamss->dev, "Not support DVP sensor\n");
> return -EPERM;
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
> index 9482081288fa..a14f22bc0742 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.h
> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
> @@ -19,6 +19,7 @@
> #include <media/v4l2-device.h>
>
> #include "stf_isp.h"
> +#include "stf_vin.h"
>
> #define STF_DVP_NAME "stf_dvp"
> #define STF_CSI_NAME "cdns_csi2rx"
> @@ -67,6 +68,7 @@ struct stfcamss {
> struct media_device media_dev;
> struct media_pipeline pipe;
> struct device *dev;
> + struct stf_vin_dev vin_dev;
> struct stf_isp_dev isp_dev;
> struct v4l2_async_notifier notifier;
> void __iomem *syscon_base;
> diff --git a/drivers/media/platform/starfive/camss/stf_vin.c b/drivers/media/platform/starfive/camss/stf_vin.c
> new file mode 100644
> index 000000000000..0efa4bbb079c
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_vin.c
> @@ -0,0 +1,1069 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_vin.c
> + *
> + * StarFive Camera Subsystem - VIN Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#include <linux/dma-mapping.h>
> +#include <linux/pm_runtime.h>
> +
> +#include "stf_camss.h"
> +
> +#define vin_line_array(ptr_line) \
> + ((const struct vin_line (*)[]) &(ptr_line)[-((ptr_line)->id)])
> +
> +#define line_to_vin_dev(ptr_line) \
> + container_of(vin_line_array(ptr_line), struct stf_vin_dev, line)
> +
> +#define VIN_FRAME_DROP_MIN_VAL 4
> +
> +/* ISP ctrl need 1 sec to let frames become stable. */
> +#define VIN_FRAME_DROP_SEC_FOR_ISP_CTRL 1
> +
> +static const struct vin_format vin_formats_wr[] = {
> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10},
> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10},
> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10},
> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10},
> +};
> +
> +static const struct vin_format vin_formats_uo[] = {
> + { MEDIA_BUS_FMT_Y12_1X12, 8},
> +};
> +
> +static const struct vin_format_table vin_formats_table[] = {
> + /* VIN_LINE_WR */
> + { vin_formats_wr, ARRAY_SIZE(vin_formats_wr) },
> + /* VIN_LINE_ISP */
> + { vin_formats_uo, ARRAY_SIZE(vin_formats_uo) },
> +};
> +
> +static void vin_buffer_done(struct vin_line *line);
> +static void vin_change_buffer(struct vin_line *line);
> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output);
> +static void vin_output_init_addrs(struct vin_line *line);
> +static struct v4l2_mbus_framefmt *
> +__vin_get_format(struct vin_line *line,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which);
> +
> +static char *vin_get_line_subdevname(int line_id)
> +{
> + char *name = NULL;
> +
> + switch (line_id) {
> + case VIN_LINE_WR:
> + name = "wr";
> + break;
> + case VIN_LINE_ISP:
> + name = "isp0";
> + break;
> + default:
> + name = "unknown";
> + break;
> + }
> + return name;
> +}
> +
> +static enum isp_line_id vin_map_isp_line(enum vin_line_id line)
> +{
> + enum isp_line_id line_id;
> +
> + if (line > VIN_LINE_WR && line < VIN_LINE_MAX)
> + line_id = STF_ISP_LINE_SRC;
> + else
> + line_id = STF_ISP_LINE_INVALID;
> +
> + return line_id;
> +}
> +
> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def)
> +{
> + enum isp_pad_id pad_id;
> +
> + if (line == VIN_LINE_WR)
> + pad_id = STF_ISP_PAD_SINK;
> + else if ((line > VIN_LINE_WR) && (line < VIN_LINE_MAX))
> + pad_id = (enum isp_pad_id)vin_map_isp_line(line);
> + else
> + pad_id = def;
> +
> + return pad_id;
> +}
> +
> +int stf_vin_subdev_init(struct stfcamss *stfcamss)
> +{
> + struct device *dev = stfcamss->dev;
> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> + int i, ret = 0;
> +
> + vin_dev->stfcamss = stfcamss;
> +
> + vin_dev->isr_ops = devm_kzalloc(dev, sizeof(*vin_dev->isr_ops),
> + GFP_KERNEL);
> + if (!vin_dev->isr_ops)
> + return -ENOMEM;
> + vin_dev->isr_ops->isr_buffer_done = vin_buffer_done;
> + vin_dev->isr_ops->isr_change_buffer = vin_change_buffer;
> +
> + atomic_set(&vin_dev->ref_count, 0);
> +
> + ret = devm_request_irq(dev,
> + stfcamss->irq[STF_IRQ_VINWR],
> + stf_vin_wr_irq_handler,
> + 0, "vin_axiwr_irq", vin_dev);
> + if (ret) {
> + dev_err(dev, "Failed to request irq\n");
> + goto out;
> + }
> +
> + ret = devm_request_irq(dev,
> + stfcamss->irq[STF_IRQ_ISP],
> + stf_vin_isp_irq_handler,
> + 0, "vin_isp_irq", vin_dev);
> + if (ret) {
> + dev_err(dev, "Failed to request isp irq\n");
> + goto out;
> + }
> +
> + ret = devm_request_irq(dev,
> + stfcamss->irq[STF_IRQ_ISPCSIL],
> + stf_vin_isp_irq_csiline_handler,
> + 0, "vin_isp_irq_csiline", vin_dev);
> + if (ret) {
> + dev_err(dev, "failed to request isp irq csiline\n");
> + goto out;
> + }
> +
> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++) {
> + struct dummy_buffer *dummy_buffer = &vin_dev->dummy_buffer[i];
> +
> + mutex_init(&dummy_buffer->stream_lock);
> + dummy_buffer->nums =
> + i == 0 ? VIN_DUMMY_BUFFER_NUMS : ISP_DUMMY_BUFFER_NUMS;
> + dummy_buffer->stream_count = 0;
> + dummy_buffer->buffer =
> + devm_kzalloc(dev,
> + dummy_buffer->nums * sizeof(struct vin_dummy_buffer),
> + GFP_KERNEL);
> + atomic_set(&dummy_buffer->frame_skip, 0);
> + }
> +
> + for (i = VIN_LINE_WR; i < STF_ISP_LINE_MAX + 1; i++) {
> + struct vin_line *l = &vin_dev->line[i];
> +
> + l->video_out.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + l->video_out.stfcamss = stfcamss;
> + l->id = i;
> + l->formats = vin_formats_table[i].fmts;
> + l->nformats = vin_formats_table[i].nfmts;
> + spin_lock_init(&l->output_lock);
> +
> + mutex_init(&l->stream_lock);
> + l->stream_count = 0;
> + }
> +
> + return 0;
> +out:
> + return ret;
> +}
> +
> +static enum link vin_get_link(struct media_entity *entity)
> +{
> + struct v4l2_subdev *subdev;
> + struct media_pad *pad;
> + bool isp = false;
> +
> + while (1) {
> + pad = &entity->pads[0];
> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> + return LINK_ERROR;
> +
> + pad = media_pad_remote_pad_first(pad);
> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> + return LINK_ERROR;
> +
> + entity = pad->entity;
> + subdev = media_entity_to_v4l2_subdev(entity);
> +
> + if (!strncmp(subdev->name, STF_CSI_NAME,
> + strlen(STF_CSI_NAME))) {
> + if (isp)
> + return LINK_CSI_TO_ISP;
> + else
> + return LINK_CSI_TO_WR;
> + } else if (!strncmp(subdev->name, STF_DVP_NAME,
> + strlen(STF_DVP_NAME))) {
> + if (isp)
> + return LINK_DVP_TO_ISP;
> + else
> + return LINK_DVP_TO_WR;
> + } else if (!strncmp(subdev->name, STF_ISP_NAME,
> + strlen(STF_ISP_NAME))) {
> + isp = true;
> + } else {
> + return LINK_ERROR;
> + }
> + }
> +}
> +
> +static int vin_enable_output(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + output->state = VIN_OUTPUT_IDLE;
> +
> + output->buf[0] = vin_buf_get_pending(output);
> +
> + if (!output->buf[0] && output->buf[1]) {
> + output->buf[0] = output->buf[1];
> + output->buf[1] = NULL;
> + }
> +
> + if (output->buf[0])
> + output->state = VIN_OUTPUT_SINGLE;
> +
> + output->sequence = 0;
> +
> + vin_output_init_addrs(line);
> + spin_unlock_irqrestore(&line->output_lock, flags);
> + return 0;
> +}
> +
> +static int vin_disable_output(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + output->state = VIN_OUTPUT_OFF;
> +
> + spin_unlock_irqrestore(&line->output_lock, flags);
> + return 0;
> +}
> +
> +static u32 vin_line_to_dummy_module(struct vin_line *line)
> +{
> + u32 dummy_module = 0;
> +
> + switch (line->id) {
> + case VIN_LINE_WR:
> + dummy_module = STF_DUMMY_VIN;
> + break;
> + case VIN_LINE_ISP:
> + dummy_module = STF_DUMMY_ISP;
> + break;
> + default:
> + dummy_module = STF_DUMMY_VIN;
> + break;
> + }
> +
> + return dummy_module;
> +}
> +
> +static int vin_alloc_dummy_buffer(struct stf_vin_dev *vin_dev,
> + struct v4l2_mbus_framefmt *fmt,
> + int dummy_module)
> +{
> + struct device *dev = vin_dev->stfcamss->dev;
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[dummy_module];
> + struct vin_dummy_buffer *buffer = NULL;
> + int ret = 0, i;
> + u32 aligns;
> +
> + for (i = 0; i < dummy_buffer->nums; i++) {
> + buffer = &vin_dev->dummy_buffer[dummy_module].buffer[i];
> + buffer->width = fmt->width;
> + buffer->height = fmt->height;
> + buffer->mcode = fmt->code;
> + if (i == STF_VIN_PAD_SINK) {
> + aligns = ALIGN(fmt->width * 4,
> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
> + buffer->buffer_size = PAGE_ALIGN(aligns * fmt->height);
> + } else if (i == STF_ISP_PAD_SRC) {
> + aligns = ALIGN(fmt->width,
> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
> + buffer->buffer_size =
> + PAGE_ALIGN(aligns * fmt->height * 3 / 2);
> + } else {
> + continue;
> + }
> +
> + buffer->vaddr = dma_alloc_coherent(dev,
> + buffer->buffer_size,
> + &buffer->paddr[0],
> + GFP_KERNEL | __GFP_NOWARN);
> +
> + if (buffer->vaddr) {
> + if (i == STF_ISP_PAD_SRC)
> + buffer->paddr[1] =
> + (dma_addr_t)(buffer->paddr[0] + aligns * fmt->height);
> + else
> + dev_dbg(dev, "signal plane\n");
> + }
> + }
> +
> + return ret;
> +}
> +
> +static void vin_free_dummy_buffer(struct stf_vin_dev *vin_dev, int dummy_module)
> +{
> + struct device *dev = vin_dev->stfcamss->dev;
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[dummy_module];
> + struct vin_dummy_buffer *buffer = NULL;
> + int i;
> +
> + for (i = 0; i < dummy_buffer->nums; i++) {
> + buffer = &dummy_buffer->buffer[i];
> + if (buffer->vaddr)
> + dma_free_coherent(dev, buffer->buffer_size,
> + buffer->vaddr, buffer->paddr[0]);
> + memset(buffer, 0, sizeof(struct vin_dummy_buffer));
> + }
> +}
> +
> +static void vin_set_dummy_buffer(struct vin_line *line, u32 pad)
> +{
> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> + int dummy_module = vin_line_to_dummy_module(line);
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[dummy_module];
> + struct vin_dummy_buffer *buffer = NULL;
> +
> + switch (pad) {
> + case STF_VIN_PAD_SINK:
> + if (line->id == VIN_LINE_WR) {
> + buffer = &dummy_buffer->buffer[STF_VIN_PAD_SINK];
> + stf_vin_wr_set_ping_addr(vin_dev, buffer->paddr[0]);
> + stf_vin_wr_set_pong_addr(vin_dev, buffer->paddr[0]);
> + } else {
> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
> + stf_vin_isp_set_yuv_addr(vin_dev,
> + buffer->paddr[0],
> + buffer->paddr[1]);
> + }
> + break;
> + case STF_ISP_PAD_SRC:
> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
> + stf_vin_isp_set_yuv_addr(vin_dev,
> + buffer->paddr[0],
> + buffer->paddr[1]);
> + break;
> + default:
> + break;
> + }
> +}
> +
> +static int vin_set_stream(struct v4l2_subdev *sd, int enable)
> +{
> + struct vin_line *line = v4l2_get_subdevdata(sd);
> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> + int dummy_module = vin_line_to_dummy_module(line);
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[dummy_module];
> + struct v4l2_mbus_framefmt *fmt;
> + enum link link;
> +
> + fmt = __vin_get_format(line, NULL,
> + STF_VIN_PAD_SINK, V4L2_SUBDEV_FORMAT_ACTIVE);
> + mutex_lock(&dummy_buffer->stream_lock);
> + if (enable) {
> + if (dummy_buffer->stream_count == 0) {
> + vin_alloc_dummy_buffer(vin_dev, fmt, dummy_module);
> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
> + atomic_set(&dummy_buffer->frame_skip,
> + VIN_FRAME_DROP_MIN_VAL);
> + }
> + dummy_buffer->stream_count++;
> + } else {
> + if (dummy_buffer->stream_count == 1) {
> + vin_free_dummy_buffer(vin_dev, dummy_module);
> + /* set buffer addr to zero */
> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
> + } else {
> + vin_set_dummy_buffer(line,
> + stf_vin_map_isp_pad(line->id, STF_ISP_PAD_SINK));
> + }
> +
> + dummy_buffer->stream_count--;
> + }
> + mutex_unlock(&dummy_buffer->stream_lock);
> +
> + mutex_lock(&line->stream_lock);
> + link = vin_get_link(&sd->entity);
> + if (link == LINK_ERROR)
> + goto exit;
> +
> + if (enable) {
> + if (line->stream_count == 0) {
> + stf_vin_stream_set(vin_dev, link);
> + if (line->id == VIN_LINE_WR) {
> + stf_vin_wr_irq_enable(vin_dev, 1);
> + stf_vin_wr_stream_set(vin_dev);
> + }
> + }
> + line->stream_count++;
> + } else {
> + if (line->stream_count == 1 && line->id == VIN_LINE_WR)
> + stf_vin_wr_irq_enable(vin_dev, 0);
> + line->stream_count--;
> + }
> +exit:
> + mutex_unlock(&line->stream_lock);
> +
> + if (enable)
> + vin_enable_output(line);
> + else
> + vin_disable_output(line);
> +
> + return 0;
> +}
> +
> +static struct v4l2_mbus_framefmt *
> +__vin_get_format(struct vin_line *line,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> + return v4l2_subdev_get_try_format(&line->subdev, state, pad);
> + return &line->fmt[pad];
> +}
> +
> +static void vin_try_format(struct vin_line *line,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + struct v4l2_mbus_framefmt *fmt,
> + enum v4l2_subdev_format_whence which)
> +{
> + unsigned int i;
> +
> + switch (pad) {
> + case STF_VIN_PAD_SINK:
> + /* Set format on sink pad */
> + for (i = 0; i < line->nformats; i++)
> + if (fmt->code == line->formats[i].code)
> + break;
> +
> + /* If not found, use UYVY as default */
> + if (i >= line->nformats)
> + fmt->code = line->formats[0].code;
> +
> + fmt->width = clamp_t(u32, fmt->width,
> + STFCAMSS_FRAME_MIN_WIDTH,
> + STFCAMSS_FRAME_MAX_WIDTH);
> + fmt->height = clamp_t(u32, fmt->height,
> + STFCAMSS_FRAME_MIN_HEIGHT,
> + STFCAMSS_FRAME_MAX_HEIGHT);
> +
> + fmt->field = V4L2_FIELD_NONE;
> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> + fmt->flags = 0;
> + break;
> +
> + case STF_VIN_PAD_SRC:
> + /* Set and return a format same as sink pad */
> + *fmt = *__vin_get_format(line, state, STF_VIN_PAD_SINK, which);
> + break;
> + }
> +
> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> +}
> +
> +static int vin_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct vin_line *line = v4l2_get_subdevdata(sd);
> +
> + if (code->index >= line->nformats)
> + return -EINVAL;
> + if (code->pad == STF_VIN_PAD_SINK) {
> + code->code = line->formats[code->index].code;
> + } else {
> + struct v4l2_mbus_framefmt *sink_fmt;
> +
> + sink_fmt = __vin_get_format(line, state, STF_VIN_PAD_SINK,
> + code->which);
> +
> + code->code = sink_fmt->code;
> + if (!code->code)
> + return -EINVAL;
> + }
> + code->flags = 0;
> +
> + return 0;
> +}
> +
> +static int vin_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + struct vin_line *line = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt format;
> +
> + if (fse->index != 0)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = 1;
> + format.height = 1;
> + vin_try_format(line, state, fse->pad, &format, fse->which);
> + fse->min_width = format.width;
> + fse->min_height = format.height;
> +
> + if (format.code != fse->code)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = -1;
> + format.height = -1;
> + vin_try_format(line, state, fse->pad, &format, fse->which);
> + fse->max_width = format.width;
> + fse->max_height = format.height;
> +
> + return 0;
> +}
> +
> +static int vin_get_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct vin_line *line = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + fmt->format = *format;
> +
> + return 0;
> +}
> +
> +static int vin_set_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct vin_line *line = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + mutex_lock(&line->stream_lock);
> + if (line->stream_count) {
> + fmt->format = *format;
> + mutex_unlock(&line->stream_lock);
> + goto out;
> + } else {
> + vin_try_format(line, state, fmt->pad, &fmt->format, fmt->which);
> + *format = fmt->format;
> + }
> + mutex_unlock(&line->stream_lock);
> +
> + if (fmt->pad == STF_VIN_PAD_SINK) {
> + /* Propagate the format from sink to source */
> + format = __vin_get_format(line, state, STF_VIN_PAD_SRC,
> + fmt->which);
> +
> + *format = fmt->format;
> + vin_try_format(line, state, STF_VIN_PAD_SRC, format,
> + fmt->which);
> + }
> +
> +out:
> + return 0;
> +}
> +
> +static int vin_init_formats(struct v4l2_subdev *sd,
> + struct v4l2_subdev_fh *fh)
> +{
> + struct v4l2_subdev_format format = {
> + .pad = STF_VIN_PAD_SINK,
> + .which = fh ?
> + V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
> + .format = {
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> + .width = 1920,
> + .height = 1080
> + }
> + };
> +
> + return vin_set_format(sd, fh ? fh->state : NULL, &format);
> +}
> +
> +static void vin_output_init_addrs(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> + dma_addr_t ping_addr;
> + dma_addr_t y_addr, uv_addr;
> +
> + output->active_buf = 0;
> +
> + if (output->buf[0]) {
> + ping_addr = output->buf[0]->addr[0];
> + y_addr = output->buf[0]->addr[0];
> + uv_addr = output->buf[0]->addr[1];
> + } else {
> + return;
> + }
> +
> + switch (vin_map_isp_line(line->id)) {
> + case STF_ISP_LINE_SRC:
> + stf_vin_isp_set_yuv_addr(vin_dev, y_addr, uv_addr);
> + break;
> + default:
> + if (line->id == VIN_LINE_WR) {
> + stf_vin_wr_set_ping_addr(vin_dev, ping_addr);
> + stf_vin_wr_set_pong_addr(vin_dev, ping_addr);
> + }
> + break;
> + }
> +}
> +
> +static void vin_init_outputs(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> +
> + output->state = VIN_OUTPUT_OFF;
> + output->buf[0] = NULL;
> + output->buf[1] = NULL;
> + output->active_buf = 0;
> + INIT_LIST_HEAD(&output->pending_bufs);
> + INIT_LIST_HEAD(&output->ready_bufs);
> +}
> +
> +static void vin_buf_add_ready(struct vin_output *output,
> + struct stfcamss_buffer *buffer)
> +{
> + INIT_LIST_HEAD(&buffer->queue);
> + list_add_tail(&buffer->queue, &output->ready_bufs);
> +}
> +
> +static struct stfcamss_buffer *vin_buf_get_ready(struct vin_output *output)
> +{
> + struct stfcamss_buffer *buffer = NULL;
> +
> + if (!list_empty(&output->ready_bufs)) {
> + buffer = list_first_entry(&output->ready_bufs,
> + struct stfcamss_buffer,
> + queue);
> + list_del(&buffer->queue);
> + }
> +
> + return buffer;
> +}
> +
> +static void vin_buf_add_pending(struct vin_output *output,
> + struct stfcamss_buffer *buffer)
> +{
> + INIT_LIST_HEAD(&buffer->queue);
> + list_add_tail(&buffer->queue, &output->pending_bufs);
> +}
> +
> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output)
> +{
> + struct stfcamss_buffer *buffer = NULL;
> +
> + if (!list_empty(&output->pending_bufs)) {
> + buffer = list_first_entry(&output->pending_bufs,
> + struct stfcamss_buffer,
> + queue);
> + list_del(&buffer->queue);
> + }
> +
> + return buffer;
> +}
> +
> +static void vin_buf_update_on_last(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> +
> + switch (output->state) {
> + case VIN_OUTPUT_CONTINUOUS:
> + output->state = VIN_OUTPUT_SINGLE;
> + output->active_buf = !output->active_buf;
> + break;
> + case VIN_OUTPUT_SINGLE:
> + output->state = VIN_OUTPUT_STOPPING;
> + break;
> + default:
> + break;
> + }
> +}
> +
> +static void vin_buf_update_on_next(struct vin_line *line)
> +{
> + struct vin_output *output = &line->output;
> +
> + switch (output->state) {
> + case VIN_OUTPUT_CONTINUOUS:
> + output->active_buf = !output->active_buf;
> + break;
> + case VIN_OUTPUT_SINGLE:
> + default:
> + break;
> + }
> +}
> +
> +static void vin_buf_update_on_new(struct vin_line *line,
> + struct vin_output *output,
> + struct stfcamss_buffer *new_buf)
> +{
> + switch (output->state) {
> + case VIN_OUTPUT_SINGLE:
> + vin_buf_add_pending(output, new_buf);
> + break;
> + case VIN_OUTPUT_IDLE:
> + if (!output->buf[0]) {
> + output->buf[0] = new_buf;
> + vin_output_init_addrs(line);
> + output->state = VIN_OUTPUT_SINGLE;
> + } else {
> + vin_buf_add_pending(output, new_buf);
> + }
> + break;
> + case VIN_OUTPUT_STOPPING:
> + if (output->last_buffer) {
> + output->buf[output->active_buf] = output->last_buffer;
> + output->last_buffer = NULL;
> + }
> +
> + output->state = VIN_OUTPUT_SINGLE;
> + vin_buf_add_pending(output, new_buf);
> + break;
> + case VIN_OUTPUT_CONTINUOUS:
> + default:
> + vin_buf_add_pending(output, new_buf);
> + break;
> + }
> +}
> +
> +static void vin_buf_flush(struct vin_output *output,
> + enum vb2_buffer_state state)
> +{
> + struct stfcamss_buffer *buf;
> + struct stfcamss_buffer *t;
> +
> + list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> + list_del(&buf->queue);
> + }
> + list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> + list_del(&buf->queue);
> + }
> +}
> +
> +static void vin_buffer_done(struct vin_line *line)
> +{
> + struct stfcamss_buffer *ready_buf;
> + struct vin_output *output = &line->output;
> + unsigned long flags;
> + u64 ts = ktime_get_ns();
> +
> + if (output->state == VIN_OUTPUT_OFF ||
> + output->state == VIN_OUTPUT_RESERVED)
> + return;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + while ((ready_buf = vin_buf_get_ready(output))) {
> + ready_buf->vb.vb2_buf.timestamp = ts;
> + ready_buf->vb.sequence = output->sequence++;
> +
> + vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> + }
> +
> + spin_unlock_irqrestore(&line->output_lock, flags);
> +}
> +
> +static void vin_change_buffer(struct vin_line *line)
> +{
> + struct stfcamss_buffer *ready_buf;
> + struct vin_output *output = &line->output;
> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> + dma_addr_t *new_addr;
> + unsigned long flags;
> + u32 active_index;
> +
> + if (output->state == VIN_OUTPUT_OFF ||
> + output->state == VIN_OUTPUT_STOPPING ||
> + output->state == VIN_OUTPUT_RESERVED ||
> + output->state == VIN_OUTPUT_IDLE)
> + return;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + active_index = output->active_buf;
> +
> + ready_buf = output->buf[active_index];
> + if (!ready_buf) {
> + dev_warn(vin_dev->stfcamss->dev, "Missing ready buf %d %d!\n",
> + active_index, output->state);
> + active_index = !active_index;
> + ready_buf = output->buf[active_index];
> + if (!ready_buf) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Missing ready buf2 %d %d!\n",
> + active_index, output->state);
> + goto out_unlock;
> + }
> + }
> +
> + /* Get next buffer */
> + output->buf[active_index] = vin_buf_get_pending(output);
> + if (!output->buf[active_index]) {
> + /* No next buffer - set same address */
> + new_addr = ready_buf->addr;
> + vin_buf_update_on_last(line);
> + } else {
> + new_addr = output->buf[active_index]->addr;
> + vin_buf_update_on_next(line);
> + }
> +
> + if (output->state == VIN_OUTPUT_STOPPING) {
> + output->last_buffer = ready_buf;
> + } else {
> + switch (vin_map_isp_line(line->id)) {
> + case STF_ISP_LINE_SRC:
> + stf_vin_isp_set_yuv_addr(vin_dev,
> + new_addr[0],
> + new_addr[1]);
> + break;
> + default:
> + if (line->id == VIN_LINE_WR) {
> + stf_vin_wr_set_ping_addr(vin_dev, new_addr[0]);
> + stf_vin_wr_set_pong_addr(vin_dev, new_addr[0]);
> + }
> + break;
> + }
> +
> + vin_buf_add_ready(output, ready_buf);
> + }
> +
> + spin_unlock_irqrestore(&line->output_lock, flags);
> + return;
> +
> +out_unlock:
> + spin_unlock_irqrestore(&line->output_lock, flags);
> +}
> +
> +static int vin_queue_buffer(struct stfcamss_video *vid,
> + struct stfcamss_buffer *buf)
> +{
> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
> + struct vin_output *output;
> + unsigned long flags;
> +
> + output = &line->output;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + vin_buf_update_on_new(line, output, buf);
> +
> + spin_unlock_irqrestore(&line->output_lock, flags);
> +
> + return 0;
> +}
> +
> +static int vin_flush_buffers(struct stfcamss_video *vid,
> + enum vb2_buffer_state state)
> +{
> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
> + struct vin_output *output = &line->output;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&line->output_lock, flags);
> +
> + vin_buf_flush(output, state);
> + if (output->buf[0])
> + vb2_buffer_done(&output->buf[0]->vb.vb2_buf, state);
> +
> + if (output->buf[1])
> + vb2_buffer_done(&output->buf[1]->vb.vb2_buf, state);
> +
> + if (output->last_buffer) {
> + vb2_buffer_done(&output->last_buffer->vb.vb2_buf, state);
> + output->last_buffer = NULL;
> + }
> + output->buf[0] = NULL;
> + output->buf[1] = NULL;
> +
> + spin_unlock_irqrestore(&line->output_lock, flags);
> + return 0;
> +}
> +
> +static int vin_link_setup(struct media_entity *entity,
> + const struct media_pad *local,
> + const struct media_pad *remote, u32 flags)
> +{
> + if (flags & MEDIA_LNK_FL_ENABLED)
> + if (media_pad_remote_pad_first(local))
> + return -EBUSY;
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_video_ops vin_video_ops = {
> + .s_stream = vin_set_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops vin_pad_ops = {
> + .enum_mbus_code = vin_enum_mbus_code,
> + .enum_frame_size = vin_enum_frame_size,
> + .get_fmt = vin_get_format,
> + .set_fmt = vin_set_format,
> +};
> +
> +static const struct v4l2_subdev_ops vin_v4l2_ops = {
> + .video = &vin_video_ops,
> + .pad = &vin_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops vin_v4l2_internal_ops = {
> + .open = vin_init_formats,
> +};
> +
> +static const struct stfcamss_video_ops stfcamss_vin_video_ops = {
> + .queue_buffer = vin_queue_buffer,
> + .flush_buffers = vin_flush_buffers,
> +};
> +
> +static const struct media_entity_operations vin_media_ops = {
> + .link_setup = vin_link_setup,
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev)
> +{
> + struct v4l2_subdev *sd;
> + struct stfcamss_video *video_out;
> + struct media_pad *pads;
> + int ret;
> + int i;
> +
> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
> + char name[32];
> + char *sub_name = vin_get_line_subdevname(i);
> +
> + sd = &vin_dev->line[i].subdev;
> + pads = vin_dev->line[i].pads;
> + video_out = &vin_dev->line[i].video_out;
> + video_out->id = i;
> +
> + vin_init_outputs(&vin_dev->line[i]);
> +
> + v4l2_subdev_init(sd, &vin_v4l2_ops);
> + sd->internal_ops = &vin_v4l2_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d_%s",
> + STF_VIN_NAME, 0, sub_name);
> + v4l2_set_subdevdata(sd, &vin_dev->line[i]);
> +
> + ret = vin_init_formats(sd, NULL);
> + if (ret < 0) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Failed to init format: %d\n", ret);
> + goto err_init;
> + }
> +
> + pads[STF_VIN_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + pads[STF_VIN_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +
> + sd->entity.function =
> + MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> + sd->entity.ops = &vin_media_ops;
> + ret = media_entity_pads_init(&sd->entity,
> + STF_VIN_PADS_NUM, pads);
> + if (ret < 0) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Failed to init media entity: %d\n",
> + ret);
> + goto err_init;
> + }
> +
> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
> + if (ret < 0) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Failed to register subdev: %d\n", ret);
> + goto err_reg_subdev;
> + }
> +
> + video_out->ops = &stfcamss_vin_video_ops;
> + video_out->bpl_alignment = 16 * 8;
> +
> + snprintf(name, ARRAY_SIZE(name), "%s_%s%d",
> + sd->name, "video", i);
> + ret = stf_video_register(video_out, v4l2_dev, name);
> + if (ret < 0) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Failed to register video node: %d\n", ret);
> + goto err_vid_reg;
> + }
> +
> + ret = media_create_pad_link(
> + &sd->entity, STF_VIN_PAD_SRC,
> + &video_out->vdev.entity, 0,
> + MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> + if (ret < 0) {
> + dev_err(vin_dev->stfcamss->dev,
> + "Failed to link %s->%s entities: %d\n",
> + sd->entity.name, video_out->vdev.entity.name,
> + ret);
> + goto err_create_link;
> + }
> + }
> +
> + return 0;
> +
> +err_create_link:
> + stf_video_unregister(video_out);
> +err_vid_reg:
> + v4l2_device_unregister_subdev(sd);
> +err_reg_subdev:
> + media_entity_cleanup(&sd->entity);
> +err_init:
> + for (i--; i >= 0; i--) {
> + sd = &vin_dev->line[i].subdev;
> + video_out = &vin_dev->line[i].video_out;
> +
> + stf_video_unregister(video_out);
> + v4l2_device_unregister_subdev(sd);
> + media_entity_cleanup(&sd->entity);
> + }
> + return ret;
> +}
> +
> +int stf_vin_unregister(struct stf_vin_dev *vin_dev)
> +{
> + struct v4l2_subdev *sd;
> + struct stfcamss_video *video_out;
> + int i;
> +
> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++)
> + mutex_destroy(&vin_dev->dummy_buffer[i].stream_lock);
> +
> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
> + sd = &vin_dev->line[i].subdev;
> + video_out = &vin_dev->line[i].video_out;
> +
> + stf_video_unregister(video_out);
> + v4l2_device_unregister_subdev(sd);
> + media_entity_cleanup(&sd->entity);
> + mutex_destroy(&vin_dev->line[i].stream_lock);
> + }
> + return 0;
> +}
> diff --git a/drivers/media/platform/starfive/camss/stf_vin.h b/drivers/media/platform/starfive/camss/stf_vin.h
> new file mode 100644
> index 000000000000..28572eb6abe4
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_vin.h
> @@ -0,0 +1,173 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_vin.h
> + *
> + * StarFive Camera Subsystem - VIN Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_VIN_H
> +#define STF_VIN_H
> +
> +#include <linux/interrupt.h>
> +#include <linux/spinlock_types.h>
> +#include <media/v4l2-subdev.h>
> +
> +#include "stf_video.h"
> +
> +#define SYSCONSAIF_SYSCFG(x) (x)
> +
> +/* syscon offset 0 */
> +#define U0_VIN_CNFG_AXI_DVP_EN BIT(2)
> +
> +/* syscon offset 20 */
> +#define U0_VIN_CHANNEL_SEL_MASK GENMASK(3, 0)
> +#define U0_VIN_AXIWR0_EN BIT(4)
> +#define CHANNEL(x) ((x) << 0)
> +
> +/* syscon offset 32 */
> +#define U0_VIN_INTR_CLEAN BIT(0)
> +#define U0_VIN_INTR_M BIT(1)
> +#define U0_VIN_PIX_CNT_END_MASK GENMASK(12, 2)
> +#define U0_VIN_PIX_CT_MASK GENMASK(14, 13)
> +#define U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS GENMASK(16, 15)
> +
> +#define PIX_CNT_END(x) ((x) << 2)
> +#define PIX_CT(x) ((x) << 13)
> +#define PIXEL_HEIGH_BIT_SEL(x) ((x) << 15)
> +
> +/* syscon offset 36 */
> +#define U0_VIN_CNFG_DVP_HS_POS BIT(1)
> +#define U0_VIN_CNFG_DVP_SWAP_EN BIT(2)
> +#define U0_VIN_CNFG_DVP_VS_POS BIT(3)
> +#define U0_VIN_CNFG_GEN_EN_AXIRD BIT(4)
> +#define U0_VIN_CNFG_ISP_DVP_EN0 BIT(5)
> +#define U0_VIN_MIPI_BYTE_EN_ISP0(n) ((n) << 6)
> +#define U0_VIN_MIPI_CHANNEL_SEL0(n) ((n) << 8)
> +#define U0_VIN_P_I_MIPI_HAEDER_EN0(n) ((n) << 12)
> +#define U0_VIN_PIX_NUM(n) ((n) << 13)
> +#define U0_VIN_MIPI_BYTE_EN_ISP0_MASK GENMASK(7, 6)
> +#define U0_VIN_MIPI_CHANNEL_SEL0_MASK GENMASK(11, 8)
> +#define U0_VIN_P_I_MIPI_HAEDER_EN0_MASK BIT(12)
> +#define U0_VIN_PIX_NUM_MASK GENMASK(16, 13)
> +
> +#define STF_VIN_PAD_SINK 0
> +#define STF_VIN_PAD_SRC 1
> +#define STF_VIN_PADS_NUM 2
> +
> +#define ISP_DUMMY_BUFFER_NUMS STF_ISP_PAD_MAX
> +#define VIN_DUMMY_BUFFER_NUMS 1
> +
> +enum {
> + STF_DUMMY_VIN,
> + STF_DUMMY_ISP,
> + STF_DUMMY_MODULE_NUMS,
> +};
> +
> +enum link {
> + LINK_ERROR = -1,
> + LINK_DVP_TO_WR,
> + LINK_DVP_TO_ISP,
> + LINK_CSI_TO_WR,
> + LINK_CSI_TO_ISP,
> +};
> +
> +struct vin_format {
> + u32 code;
> + u8 bpp;
> +};
> +
> +struct vin_format_table {
> + const struct vin_format *fmts;
> + int nfmts;
> +};
> +
> +enum vin_output_state {
> + VIN_OUTPUT_OFF,
> + VIN_OUTPUT_RESERVED,
> + VIN_OUTPUT_SINGLE,
> + VIN_OUTPUT_CONTINUOUS,
> + VIN_OUTPUT_IDLE,
> + VIN_OUTPUT_STOPPING
> +};
> +
> +struct vin_output {
> + int active_buf;
> + struct stfcamss_buffer *buf[2];
> + struct stfcamss_buffer *last_buffer;
> + struct list_head pending_bufs;
> + struct list_head ready_bufs;
> + enum vin_output_state state;
> + unsigned int sequence;
> + unsigned int frame_skip;
> +};
> +
> +/* The vin output lines */
> +enum vin_line_id {
> + VIN_LINE_NONE = -1,
> + VIN_LINE_WR = 0,
> + VIN_LINE_ISP,
> + VIN_LINE_MAX,
> +};
> +
> +struct vin_line {
> + enum vin_line_id id;
> + struct v4l2_subdev subdev;
> + struct media_pad pads[STF_VIN_PADS_NUM];
> + struct v4l2_mbus_framefmt fmt[STF_VIN_PADS_NUM];
> + struct stfcamss_video video_out;
> + struct mutex stream_lock; /* serialize stream control */
> + int stream_count;
> + struct vin_output output; /* pipeline and stream states */
> + spinlock_t output_lock;
> + const struct vin_format *formats;
> + unsigned int nformats;
> +};
> +
> +struct vin_dummy_buffer {
> + dma_addr_t paddr[3];
> + void *vaddr;
> + u32 buffer_size;
> + u32 width;
> + u32 height;
> + u32 mcode;
> +};
> +
> +struct dummy_buffer {
> + struct vin_dummy_buffer *buffer;
> + u32 nums;
> + struct mutex stream_lock; /* protects buffer data */
> + int stream_count;
> + atomic_t frame_skip;
> +};
> +
> +struct vin_isr_ops {
> + void (*isr_buffer_done)(struct vin_line *line);
> + void (*isr_change_buffer)(struct vin_line *line);
> +};
> +
> +struct stf_vin_dev {
> + struct stfcamss *stfcamss;
> + struct vin_line line[VIN_LINE_MAX];
> + struct dummy_buffer dummy_buffer[STF_DUMMY_MODULE_NUMS];
> + struct vin_isr_ops *isr_ops;
> + atomic_t ref_count;
> +};
> +
> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev);
> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link);
> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable);
> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
> + dma_addr_t y_addr, dma_addr_t uv_addr);
> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv);
> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv);
> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv);
> +int stf_vin_subdev_init(struct stfcamss *stfcamss);
> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev);
> +int stf_vin_unregister(struct stf_vin_dev *vin_dev);
> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def);
> +
> +#endif /* STF_VIN_H */
> diff --git a/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> new file mode 100644
> index 000000000000..7bd3265128d0
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> @@ -0,0 +1,192 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_vin_hw_ops.c
> + *
> + * Register interface file for StarFive VIN module driver
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +#include "stf_camss.h"
> +
> +static void vin_intr_clear(struct stfcamss *stfcamss)
> +{
> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_CLEAN);
> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_CLEAN);
> +}
> +
> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv)
> +{
> + struct stf_vin_dev *vin_dev = priv;
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[STF_DUMMY_VIN];
> +
> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
> + vin_dev->isr_ops->isr_change_buffer(&vin_dev->line[VIN_LINE_WR]);
> + vin_dev->isr_ops->isr_buffer_done(&vin_dev->line[VIN_LINE_WR]);
> + }
> +
> + vin_intr_clear(stfcamss);
> +
> + return IRQ_HANDLED;
> +}
> +
> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv)
> +{
> + struct stf_vin_dev *vin_dev = priv;
> + u32 int_status;
> +
> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
> +
> + if (int_status & ISPC_INTS) {
> + if ((int_status & ISPC_ENUO))
> + vin_dev->isr_ops->isr_buffer_done(
> + &vin_dev->line[VIN_LINE_ISP]);
> +
> + /* clear interrupt */
> + stf_isp_reg_write(vin_dev->stfcamss,
> + ISP_REG_ISP_CTRL_0,
> + (int_status & ~EN_INT_ALL) |
> + EN_INT_ISP_DONE |
> + EN_INT_CSI_DONE |
> + EN_INT_SC_DONE);
> + }
> +
> + return IRQ_HANDLED;
> +}
> +
> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv)
> +{
> + struct stf_vin_dev *vin_dev = priv;
> + struct stf_isp_dev *isp_dev;
> + u32 int_status;
> +
> + isp_dev = &vin_dev->stfcamss->isp_dev;
> +
> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
> + if (int_status & ISPC_SCFEINT) {
> + struct dummy_buffer *dummy_buffer =
> + &vin_dev->dummy_buffer[STF_DUMMY_ISP];
> +
> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
> + if ((int_status & ISPC_ENUO))
> + vin_dev->isr_ops->isr_change_buffer(
> + &vin_dev->line[VIN_LINE_ISP]);
> + }
> +
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_CSIINTS,
> + CSI_INTS_MASK, CSI_INTS(0x3));
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_IESHD,
> + SHAD_UP_M | SHAD_UP_EN, 0x3);
> +
> + /* clear interrupt */
> + stf_isp_reg_write(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> + (int_status & ~EN_INT_ALL) | EN_INT_LINE_INT);
> + }
> +
> + return IRQ_HANDLED;
> +}
> +
> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev)
> +{
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> +
> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(20),
> + U0_VIN_AXIWR0_EN);
> +
> + return 0;
> +}
> +
> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link)
> +{
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> + u32 val;
> +
> + switch (link) {
> + case LINK_CSI_TO_WR:
> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(20));
> + val &= ~U0_VIN_CHANNEL_SEL_MASK;
> + val |= CHANNEL(0);
> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(20), val);
> +
> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(28));
> + val &= ~U0_VIN_PIX_CT_MASK;
> + val |= PIX_CT(1);
> +
> + val &= ~U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS;
> + val |= PIXEL_HEIGH_BIT_SEL(0);
> +
> + val &= ~U0_VIN_PIX_CNT_END_MASK;
> + val |= PIX_CNT_END(IMAGE_MAX_WIDTH / 4 - 1);
> +
> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(28), val);
> + break;
> + case LINK_CSI_TO_ISP:
> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(36));
> + val &= ~U0_VIN_MIPI_BYTE_EN_ISP0_MASK;
> + val |= U0_VIN_MIPI_BYTE_EN_ISP0(0);
> +
> + val &= ~U0_VIN_MIPI_CHANNEL_SEL0_MASK;
> + val |= U0_VIN_MIPI_CHANNEL_SEL0(0);
> +
> + val &= ~U0_VIN_PIX_NUM_MASK;
> + val |= U0_VIN_PIX_NUM(0);
> +
> + val &= ~U0_VIN_P_I_MIPI_HAEDER_EN0_MASK;
> + val |= U0_VIN_P_I_MIPI_HAEDER_EN0(1);
> +
> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(36), val);
> + break;
> + case LINK_DVP_TO_WR:
> + case LINK_DVP_TO_ISP:
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable)
> +{
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> +
> + if (enable) {
> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_M);
> + } else {
> + /* clear vin interrupt */
> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_CLEAN);
> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_CLEAN);
> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> + U0_VIN_INTR_M);
> + }
> +}
> +
> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
> +{
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> +
> + /* set the start address */
> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(32), (long)addr);
> +}
> +
> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
> +{
> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> +
> + /* set the start address */
> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(24), (long)addr);
> +}
> +
> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
> + dma_addr_t y_addr, dma_addr_t uv_addr)
> +{
> + stf_isp_reg_write(vin_dev->stfcamss,
> + ISP_REG_Y_PLANE_START_ADDR, y_addr);
> + stf_isp_reg_write(vin_dev->stfcamss,
> + ISP_REG_UV_PLANE_START_ADDR, uv_addr);
> +}
--
Regards,
Laurent Pinchart
Hi Jack,
Thank you for the patch.
On Mon, Jun 19, 2023 at 07:28:37PM +0800, Jack Zhu wrote:
> Add ISP driver for StarFive Camera Subsystem.
>
> Signed-off-by: Jack Zhu <[email protected]>
> ---
> .../media/platform/starfive/camss/Makefile | 2 +
> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
> .../media/platform/starfive/camss/stf_camss.h | 3 +
> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
> 6 files changed, 1544 insertions(+), 3 deletions(-)
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>
> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> index eb457917a914..cdf57e8c9546 100644
> --- a/drivers/media/platform/starfive/camss/Makefile
> +++ b/drivers/media/platform/starfive/camss/Makefile
> @@ -5,6 +5,8 @@
>
> starfive-camss-objs += \
> stf_camss.o \
> + stf_isp.o \
> + stf_isp_hw_ops.o \
> stf_video.o
>
> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> index dc2b5dba7bd4..6f56b45f57db 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.c
> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> @@ -115,12 +115,65 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
> return ret;
> }
>
> +/*
> + * stfcamss_init_subdevices - Initialize subdev structures and resources
> + * @stfcamss: STFCAMSS device
> + *
> + * Return 0 on success or a negative error code on failure
> + */
> +static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
> +{
> + int ret;
> +
> + ret = stf_isp_subdev_init(stfcamss);
> + if (ret < 0) {
> + dev_err(stfcamss->dev, "Failed to init isp subdev: %d\n", ret);
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
> +{
> + int ret;
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> +
> + ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
> + if (ret < 0) {
> + dev_err(stfcamss->dev,
> + "Failed to register stf isp%d entity: %d\n", 0, ret);
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
> +{
> + stf_isp_unregister(&stfcamss->isp_dev);
> +}
> +
> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> struct v4l2_subdev *subdev,
> struct v4l2_async_subdev *asd)
> {
> + struct stfcamss *stfcamss =
> + container_of(async, struct stfcamss, notifier);
> + struct stfcamss_async_subdev *csd =
> + container_of(asd, struct stfcamss_async_subdev, asd);
> + enum stf_port_num port = csd->port;
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> struct media_pad *pad[STF_PADS_NUM];
> - unsigned int i, pad_num = 0;
> + unsigned int i, pad_num;
> +
> + if (port == STF_PORT_CSI2RX) {
> + pad[0] = &isp_dev->pads[STF_PAD_SINK];
> + pad_num = 1;
> + } else if (port == STF_PORT_DVP) {
> + dev_err(stfcamss->dev, "Not support DVP sensor\n");
> + return -EPERM;
> + }
>
> for (i = 0; i < pad_num; ++i) {
> int ret;
> @@ -223,12 +276,18 @@ static int stfcamss_probe(struct platform_device *pdev)
> goto err_cleanup_notifier;
> }
>
> + ret = stfcamss_init_subdevices(stfcamss);
> + if (ret < 0) {
> + dev_err(dev, "Failed to init subdevice: %d\n", ret);
> + goto err_cleanup_notifier;
> + }
> +
> stfcamss_mc_init(pdev, stfcamss);
>
> ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
> if (ret < 0) {
> dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
> - goto err_cleanup_notifier;
> + goto err_cleanup_media_device;
> }
>
> ret = media_device_register(&stfcamss->media_dev);
> @@ -237,6 +296,12 @@ static int stfcamss_probe(struct platform_device *pdev)
> goto err_unregister_device;
> }
>
> + ret = stfcamss_register_subdevices(stfcamss);
> + if (ret < 0) {
> + dev_err(dev, "Failed to register subdevice: %d\n", ret);
> + goto err_unregister_media_dev;
> + }
> +
> pm_runtime_enable(dev);
>
> stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
> @@ -244,15 +309,19 @@ static int stfcamss_probe(struct platform_device *pdev)
> if (ret) {
> dev_err(dev, "Failed to register async subdev nodes: %d\n",
> ret);
> - goto err_unregister_media_dev;
> + goto err_unregister_subdevs;
> }
>
> return 0;
>
> +err_unregister_subdevs:
> + stfcamss_unregister_subdevices(stfcamss);
> err_unregister_media_dev:
> media_device_unregister(&stfcamss->media_dev);
> err_unregister_device:
> v4l2_device_unregister(&stfcamss->v4l2_dev);
> +err_cleanup_media_device:
> + media_device_cleanup(&stfcamss->media_dev);
> err_cleanup_notifier:
> v4l2_async_nf_cleanup(&stfcamss->notifier);
> return ret;
> @@ -268,6 +337,7 @@ static int stfcamss_remove(struct platform_device *pdev)
> {
> struct stfcamss *stfcamss = platform_get_drvdata(pdev);
>
> + stfcamss_unregister_subdevices(stfcamss);
> v4l2_device_unregister(&stfcamss->v4l2_dev);
> media_device_cleanup(&stfcamss->media_dev);
> pm_runtime_disable(&pdev->dev);
> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
> index 15c4f34b9054..9482081288fa 100644
> --- a/drivers/media/platform/starfive/camss/stf_camss.h
> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
> @@ -18,6 +18,8 @@
> #include <media/v4l2-async.h>
> #include <media/v4l2-device.h>
>
> +#include "stf_isp.h"
> +
> #define STF_DVP_NAME "stf_dvp"
> #define STF_CSI_NAME "cdns_csi2rx"
> #define STF_ISP_NAME "stf_isp"
> @@ -65,6 +67,7 @@ struct stfcamss {
> struct media_device media_dev;
> struct media_pipeline pipe;
> struct device *dev;
> + struct stf_isp_dev isp_dev;
> struct v4l2_async_notifier notifier;
> void __iomem *syscon_base;
> void __iomem *isp_base;
> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
> new file mode 100644
> index 000000000000..933a583b398c
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
> @@ -0,0 +1,519 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_isp.c
> + *
> + * StarFive Camera Subsystem - ISP Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +#include <linux/firmware.h>
This doesn't seem needed.
> +#include <media/v4l2-event.h>
> +
> +#include "stf_camss.h"
> +
> +#define SINK_FORMATS_INDEX 0
> +#define UO_FORMATS_INDEX 1
What does "UO" stand for ?
> +
> +static int isp_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel);
> +
> +static const struct isp_format isp_formats_sink[] = {
> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
> +};
> +
> +static const struct isp_format isp_formats_uo[] = {
> + { MEDIA_BUS_FMT_Y12_1X12, 8 },
Y12 is a greyscale format, I don't think that's what you need here.
> +};
> +
> +static const struct isp_format_table isp_formats_st7110[] = {
> + { isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
> + { isp_formats_uo, ARRAY_SIZE(isp_formats_uo) },
> +};
> +
> +int stf_isp_subdev_init(struct stfcamss *stfcamss)
This function doesn't initialize the subdev, I'd call it stf_isp_init().
I would also create a stf_isp_cleanup() function to be consistent, and
move the mutex_destroy() call there.
> +{
> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> +
> + isp_dev->stfcamss = stfcamss;
> + isp_dev->formats = isp_formats_st7110;
> + isp_dev->nformats = ARRAY_SIZE(isp_formats_st7110);
> +
> + mutex_init(&isp_dev->stream_lock);
> + return 0;
> +}
> +
> +static struct v4l2_mbus_framefmt *
> +__isp_get_format(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> + return v4l2_subdev_get_try_format(&isp_dev->subdev, state, pad);
> +
> + return &isp_dev->fmt[pad];
Use the subdev active state API to store the active format on subdev
pads, it will simplify the driver. See commit a2514b9a634a ("media: i2c:
imx290: Use V4L2 subdev active state") for an example of a subdev driver
being converted to that API, it should help understanding how to use it.
You will be able to drop the stf_isp_dev fmt and rect fields. For the
bpp value stored in the rect structure, replace it with a const struct
isp_format pointer in stf_isp_dev that you set in .set_fmt() (as well as
initialization time) to the active format information.
> +}
> +
> +static int isp_set_stream(struct v4l2_subdev *sd, int enable)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + int ret = 0;
> + struct v4l2_mbus_framefmt *fmt;
> + struct v4l2_event src_ch = { 0 };
> +
> + fmt = __isp_get_format(isp_dev, NULL, STF_ISP_PAD_SINK,
> + V4L2_SUBDEV_FORMAT_ACTIVE);
> + mutex_lock(&isp_dev->stream_lock);
> + if (enable) {
> + if (isp_dev->stream_count == 0) {
> + stf_isp_clk_enable(isp_dev);
> + stf_isp_reset(isp_dev);
> + stf_isp_init_cfg(isp_dev);
> + stf_isp_settings(isp_dev, isp_dev->rect, fmt->code);
> + stf_isp_stream_set(isp_dev);
> + }
> + isp_dev->stream_count++;
The subdev .s_stream() operation isn't supposed to be called multiple
times. If you need to count how many video nodes have started streaming,
that should be handled in the vb2 queue .start_streaming() and
.stop_streaming() operations.
> + } else {
> + if (isp_dev->stream_count == 0)
> + goto exit;
> +
> + if (isp_dev->stream_count == 1)
> + stf_isp_clk_disable(isp_dev);
> +
> + isp_dev->stream_count--;
> + }
> + src_ch.type = V4L2_EVENT_SOURCE_CHANGE,
; instead of , at the end of the line. Same for the next line.
> + src_ch.u.src_change.changes = isp_dev->stream_count,
> +
> + v4l2_subdev_notify_event(sd, &src_ch);
This doesn't seem right. V4L2_EVENT_SOURCE_CHANGE is meant to report
changes to the source format. I would drop support for events
completely from this patch, it doesn't seem to be needed.
> +exit:
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + return ret;
ret is never set to a value other than 9, return 0 and drop the
variable.
> +}
> +
> +static void isp_try_format(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + struct v4l2_mbus_framefmt *fmt,
> + enum v4l2_subdev_format_whence which)
> +{
> + const struct isp_format_table *formats;
> + struct stf_isp_crop *rect;
> + unsigned int i;
> +
> + if (pad == STF_ISP_PAD_SINK) {
> + /* Set format on sink pad */
> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
> + rect = &isp_dev->rect[SINK_FORMATS_INDEX];
> + } else if (pad == STF_ISP_PAD_SRC) {
> + formats = &isp_dev->formats[UO_FORMATS_INDEX];
> + rect = &isp_dev->rect[UO_FORMATS_INDEX];
> + }
> +
> + fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> + STFCAMSS_FRAME_MAX_WIDTH);
> + fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> + STFCAMSS_FRAME_MAX_HEIGHT);
> + fmt->height &= ~0x1;
> + fmt->field = V4L2_FIELD_NONE;
> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> + fmt->flags = 0;
> +
> + for (i = 0; i < formats->nfmts; i++) {
> + if (fmt->code == formats->fmts[i].code)
> + break;
> + }
> +
> + if (i >= formats->nfmts) {
> + fmt->code = formats->fmts[0].code;
> + rect->bpp = formats->fmts[0].bpp;
> + } else {
> + rect->bpp = formats->fmts[i].bpp;
> + }
> +}
> +
> +static int isp_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + const struct isp_format_table *formats;
> +
> + if (code->index >= isp_dev->nformats)
That doesn't seem right, isp_dev->nformats isn't the number of formats
supported on the ISP, but the number of format types (sink and uo).
> + return -EINVAL;
Add a blank line here, it makes code easier to read.
> + if (code->pad == STF_ISP_PAD_SINK) {
> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
> + code->code = formats->fmts[code->index].code;
> + } else {
> + struct v4l2_mbus_framefmt *sink_fmt;
> +
> + sink_fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK,
> + code->which);
> +
> + code->code = sink_fmt->code;
This doesn't seem right, you need to enumerate the ISP output media bus
codes here, and the function returns the sink media bus code instead.
> + if (!code->code)
> + return -EINVAL;
> + }
> + code->flags = 0;
> +
> + return 0;
> +}
> +
> +static int isp_enum_frame_size(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt format;
> +
> + if (fse->index != 0)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = 1;
> + format.height = 1;
> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
> + fse->min_width = format.width;
> + fse->min_height = format.height;
> +
> + if (format.code != fse->code)
> + return -EINVAL;
> +
> + format.code = fse->code;
> + format.width = -1;
> + format.height = -1;
> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
> + fse->max_width = format.width;
> + fse->max_height = format.height;
This seems unnecessarily complicated, you can just set the min and max
width and height to STFCAMSS_FRAME_MIN_WIDTH, STFCAMSS_FRAME_MAX_WIDTH,
STFCAMSS_FRAME_MIN_HEIGHT and STFCAMSS_FRAME_MAX_HEIGHT here.
> +
> + return 0;
> +}
> +
> +static int isp_get_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + fmt->format = *format;
> +
> + return 0;
> +}
> +
> +static int isp_set_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_mbus_framefmt *format;
> +
> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
> + if (!format)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> +
> + isp_try_format(isp_dev, state, fmt->pad, &fmt->format, fmt->which);
> + *format = fmt->format;
> +
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Propagate to in crop */
> + if (fmt->pad == STF_ISP_PAD_SINK) {
> + struct v4l2_subdev_selection sel = { 0 };
> + int ret;
> +
> + /* Reset sink pad compose selection */
> + sel.which = fmt->which;
> + sel.pad = STF_ISP_PAD_SINK;
> + sel.target = V4L2_SEL_TGT_CROP;
> + sel.r.width = fmt->format.width;
> + sel.r.height = fmt->format.height;
> + ret = isp_set_selection(sd, state, &sel);
> + if (ret < 0)
> + return ret;
You're setting the which, pad and target values manually, so they're
guaranteed to be correct. isp_set_selection() will thus never return an
error. You can drop the error check.
> + }
> +
> + return 0;
> +}
> +
> +static struct v4l2_rect *
> +__isp_get_crop(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + unsigned int pad,
> + enum v4l2_subdev_format_whence which)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> + return v4l2_subdev_get_try_crop(&isp_dev->subdev, state,
> + STF_ISP_PAD_SINK);
> +
> + return &isp_dev->rect[pad].rect;
> +}
> +
> +static void isp_try_crop(struct stf_isp_dev *isp_dev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_rect *rect,
> + enum v4l2_subdev_format_whence which)
> +{
> + struct v4l2_mbus_framefmt *fmt;
> +
> + fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK, which);
> +
> + if (rect->width > fmt->width)
> + rect->width = fmt->width;
> +
> + if (rect->width + rect->left > fmt->width)
> + rect->left = fmt->width - rect->width;
> +
> + if (rect->height > fmt->height)
> + rect->height = fmt->height;
> +
> + if (rect->height + rect->top > fmt->height)
> + rect->top = fmt->height - rect->height;
> +
> + if (rect->width < STFCAMSS_FRAME_MIN_WIDTH) {
> + rect->left = 0;
> + rect->width = STFCAMSS_FRAME_MAX_WIDTH;
> + }
> +
> + if (rect->height < STFCAMSS_FRAME_MIN_HEIGHT) {
> + rect->top = 0;
> + rect->height = STFCAMSS_FRAME_MAX_HEIGHT;
> + }
> + rect->height &= ~0x1;
The v4l2_rect_set_min_size() and v4l2_rect_map_inside() functions can
simplify this code.
> +}
> +
> +static int isp_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_subdev_format fmt = { 0 };
> + struct v4l2_rect *rect;
> + int ret;
> +
> + switch (sel->target) {
> + case V4L2_SEL_TGT_CROP_BOUNDS:
> + if (sel->pad == STF_ISP_PAD_SINK) {
> + fmt.pad = sel->pad;
> + fmt.which = sel->which;
> + ret = isp_get_format(sd, state, &fmt);
> + if (ret < 0)
> + return ret;
> +
> + sel->r.left = 0;
> + sel->r.top = 0;
> + sel->r.width = fmt.format.width;
> + sel->r.height = fmt.format.height;
> + } else if (sel->pad == STF_ISP_PAD_SRC) {
> + rect = __isp_get_crop(isp_dev, state,
> + sel->pad, sel->which);
> + sel->r = *rect;
> + }
> + break;
> +
> + case V4L2_SEL_TGT_CROP:
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + sel->r = *rect;
> + break;
> +
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int isp_set_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> + struct v4l2_rect *rect;
> + int ret = 0;
> +
if (sel->target != V4L2_SEL_TGT_CROP)
return -EINVAL;
> + if (sel->target == V4L2_SEL_TGT_CROP &&
> + sel->pad == STF_ISP_PAD_SINK) {
> + struct v4l2_subdev_selection crop = { 0 };
> +
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
> + *rect = sel->r;
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Reset source crop selection */
> + crop.which = sel->which;
> + crop.pad = STF_ISP_PAD_SRC;
> + crop.target = V4L2_SEL_TGT_CROP;
> + crop.r = *rect;
> + ret = isp_set_selection(sd, state, &crop);
> + } else if (sel->target == V4L2_SEL_TGT_CROP &&
> + sel->pad == STF_ISP_PAD_SRC) {
> + struct v4l2_subdev_format fmt = { 0 };
> +
> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
> + if (!rect)
> + return -EINVAL;
> +
> + mutex_lock(&isp_dev->stream_lock);
> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
> + *rect = sel->r;
> + mutex_unlock(&isp_dev->stream_lock);
> +
> + /* Reset source pad format width and height */
> + fmt.which = sel->which;
> + fmt.pad = STF_ISP_PAD_SRC;
> + fmt.format.width = rect->width;
> + fmt.format.height = rect->height;
> + ret = isp_set_format(sd, state, &fmt);
> + if (ret < 0)
> + return ret;
You can drop the error check here too.
> + }
> +
> + dev_dbg(isp_dev->stfcamss->dev, "pad: %d sel(%d,%d)/%dx%d\n",
> + sel->pad, sel->r.left, sel->r.top, sel->r.width, sel->r.height);
> +
> + return 0;
> +}
> +
> +static int isp_init_formats(struct v4l2_subdev *sd,
> + struct v4l2_subdev_fh *fh)
> +{
> + struct v4l2_subdev_format format = {
> + .pad = STF_ISP_PAD_SINK,
> + .which =
> + fh ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
> + .format = {
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
Pick a format that the driver supports.
> + .width = 1920,
> + .height = 1080
> + }
> + };
> +
> + return isp_set_format(sd, fh ? fh->state : NULL, &format);
> +}
> +
> +static int isp_link_setup(struct media_entity *entity,
> + const struct media_pad *local,
> + const struct media_pad *remote, u32 flags)
> +{
> + if (flags & MEDIA_LNK_FL_ENABLED)
> + if (media_pad_remote_pad_first(local))
> + return -EBUSY;
> + return 0;
Is this check really needed ?
> +}
> +
> +static int stf_isp_subscribe_event(struct v4l2_subdev *sd,
> + struct v4l2_fh *fh,
> + struct v4l2_event_subscription *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return v4l2_src_change_event_subdev_subscribe(sd, fh, sub);
> + default:
> + return -EINVAL;
> + }
> +}
> +
> +static const struct v4l2_subdev_core_ops isp_core_ops = {
> + .subscribe_event = stf_isp_subscribe_event,
> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
> +};
Drop events support.
> +
> +static const struct v4l2_subdev_video_ops isp_video_ops = {
> + .s_stream = isp_set_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops isp_pad_ops = {
> + .enum_mbus_code = isp_enum_mbus_code,
> + .enum_frame_size = isp_enum_frame_size,
> + .get_fmt = isp_get_format,
> + .set_fmt = isp_set_format,
> + .get_selection = isp_get_selection,
> + .set_selection = isp_set_selection,
> +};
> +
> +static const struct v4l2_subdev_ops isp_v4l2_ops = {
> + .core = &isp_core_ops,
> + .video = &isp_video_ops,
> + .pad = &isp_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops isp_v4l2_internal_ops = {
> + .open = isp_init_formats,
> +};
Replace this with the .init_cfg() pad operation.
> +
> +static const struct media_entity_operations isp_media_ops = {
> + .link_setup = isp_link_setup,
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
> +{
> + struct v4l2_subdev *sd = &isp_dev->subdev;
> + struct media_pad *pads = isp_dev->pads;
> + int ret;
> +
> + v4l2_subdev_init(sd, &isp_v4l2_ops);
> + sd->internal_ops = &isp_v4l2_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
Drop events here too.
> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d", STF_ISP_NAME, 0);
> + v4l2_set_subdevdata(sd, isp_dev);
> +
> + ret = isp_init_formats(sd, NULL);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev, "Failed to init format: %d\n",
> + ret);
> + return ret;
> + }
> +
> + pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +
> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
MEDIA_ENT_F_PROC_VIDEO_ISP seems a better match.
> + sd->entity.ops = &isp_media_ops;
> + ret = media_entity_pads_init(&sd->entity, STF_ISP_PAD_MAX, pads);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev,
> + "Failed to init media entity: %d\n", ret);
> + return ret;
> + }
> +
> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
> + if (ret < 0) {
> + dev_err(isp_dev->stfcamss->dev,
> + "Failed to register subdev: %d\n", ret);
> + goto err_sreg;
> + }
> +
> + return 0;
> +
> +err_sreg:
> + media_entity_cleanup(&sd->entity);
> + return ret;
> +}
> +
> +int stf_isp_unregister(struct stf_isp_dev *isp_dev)
> +{
> + v4l2_device_unregister_subdev(&isp_dev->subdev);
> + media_entity_cleanup(&isp_dev->subdev.entity);
> + mutex_destroy(&isp_dev->stream_lock);
> + return 0;
> +}
> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
> new file mode 100644
> index 000000000000..1e5c98482350
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
> @@ -0,0 +1,479 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * stf_isp.h
> + *
> + * StarFive Camera Subsystem - ISP Module
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_ISP_H
> +#define STF_ISP_H
> +
> +#include <media/v4l2-subdev.h>
> +
> +#include "stf_video.h"
> +
> +#define ISP_RAW_DATA_BITS 12
> +#define SCALER_RATIO_MAX 1
> +#define STF_ISP_REG_OFFSET_MAX 0x0fff
> +#define STF_ISP_REG_DELAY_MAX 100
> +
> +/* isp registers */
> +#define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
> +#define CSI_SCD_ERR BIT(6)
> +#define CSI_ITU656_ERR BIT(4)
> +#define CSI_ITU656_F BIT(3)
> +#define CSI_SCD_DONE BIT(2)
> +#define CSI_BUSY_S BIT(1)
> +#define CSI_EN_S BIT(0)
Could you align the macros values consistently in the whole file ? It
would be easier to read. For the lines above, that would be
#define ISP_RAW_DATA_BITS 12
#define SCALER_RATIO_MAX 1
#define STF_ISP_REG_OFFSET_MAX 0x0fff
#define STF_ISP_REG_DELAY_MAX 100
/* isp registers */
#define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
#define CSI_SCD_ERR BIT(6)
#define CSI_ITU656_ERR BIT(4)
#define CSI_ITU656_F BIT(3)
#define CSI_SCD_DONE BIT(2)
#define CSI_BUSY_S BIT(1)
#define CSI_EN_S BIT(0)
(possibly with additional indentation if some of the lines below require
that)
You could also move the register definitions to a stf_isp_regs.h header
if desired, as they're only needed by stf_isp_hw_ops.c, not by the other
files that include stf_isp.h. up to you.
> +
> +#define ISP_REG_CSIINTS 0x008
> +#define CSI_INTS(n) ((n) << 16)
> +#define CSI_SHA_M(n) ((n) << 0)
> +#define CSI_INTS_MASK GENMASK(17, 16)
> +
> +#define ISP_REG_CSI_MODULE_CFG 0x010
> +#define CSI_DUMP_EN BIT(19)
> +#define CSI_VS_EN BIT(18)
> +#define CSI_SC_EN BIT(17)
> +#define CSI_OBA_EN BIT(16)
> +#define CSI_AWB_EN BIT(7)
> +#define CSI_LCCF_EN BIT(6)
> +#define CSI_OECFHM_EN BIT(5)
> +#define CSI_OECF_EN BIT(4)
> +#define CSI_LCBQ_EN BIT(3)
> +#define CSI_OBC_EN BIT(2)
> +#define CSI_DEC_EN BIT(1)
> +#define CSI_DC_EN BIT(0)
> +
> +#define ISP_REG_SENSOR 0x014
> +#define DVP_SYNC_POL(n) ((n) << 2)
> +#define ITU656_EN(n) ((n) << 1)
> +#define IMAGER_SEL(n) ((n) << 0)
> +
> +#define ISP_REG_RAW_FORMAT_CFG 0x018
> +#define SMY13(n) ((n) << 14)
> +#define SMY12(n) ((n) << 12)
> +#define SMY11(n) ((n) << 10)
> +#define SMY10(n) ((n) << 8)
> +#define SMY3(n) ((n) << 6)
> +#define SMY2(n) ((n) << 4)
> +#define SMY1(n) ((n) << 2)
> +#define SMY0(n) ((n) << 0)
> +
> +#define ISP_REG_PIC_CAPTURE_START_CFG 0x01c
> +#define VSTART_CAP(n) ((n) << 16)
> +#define HSTART_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_PIC_CAPTURE_END_CFG 0x020
> +#define VEND_CAP(n) ((n) << 16)
> +#define HEND_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_DUMP_CFG_0 0x024
> +#define ISP_REG_DUMP_CFG_1 0x028
> +#define DUMP_ID(n) ((n) << 24)
> +#define DUMP_SHT(n) ((n) << 20)
> +#define DUMP_BURST_LEN(n) ((n) << 16)
> +#define DUMP_SD(n) ((n) << 0)
> +#define DUMP_BURST_LEN_MASK GENMASK(17, 16)
> +#define DUMP_SD_MASK GENMASK(15, 0)
> +
> +#define ISP_REG_DEC_CFG 0x030
> +#define DEC_V_KEEP(n) ((n) << 24)
> +#define DEC_V_PERIOD(n) ((n) << 16)
> +#define DEC_H_KEEP(n) ((n) << 8)
> +#define DEC_H_PERIOD(n) ((n) << 0)
> +
> +#define ISP_REG_OBC_CFG 0x034
> +#define OBC_W_H(y) ((y) << 4)
> +#define OBC_W_W(x) ((x) << 0)
> +
> +#define ISP_REG_DC_CFG_1 0x044
> +#define DC_AXI_ID(n) ((n) << 0)
> +
> +#define ISP_REG_LCCF_CFG_0 0x050
> +#define Y_DISTANCE(y) ((y) << 16)
> +#define X_DISTANCE(x) ((x) << 0)
> +
> +#define ISP_REG_LCCF_CFG_1 0x058
> +#define LCCF_MAX_DIS(n) ((n) << 0)
> +
> +#define ISP_REG_LCBQ_CFG_0 0x074
> +#define H_LCBQ(y) ((y) << 12)
> +#define W_LCBQ(x) ((x) << 8)
> +
> +#define ISP_REG_LCBQ_CFG_1 0x07c
> +#define Y_COOR(y) ((y) << 16)
> +#define X_COOR(x) ((x) << 0)
> +
> +#define ISP_REG_LCCF_CFG_2 0x0e0
> +#define ISP_REG_LCCF_CFG_3 0x0e4
> +#define ISP_REG_LCCF_CFG_4 0x0e8
> +#define ISP_REG_LCCF_CFG_5 0x0ec
> +#define LCCF_F2_PAR(n) ((n) << 16)
> +#define LCCF_F1_PAR(n) ((n) << 0)
> +
> +#define ISP_REG_OECF_X0_CFG0 0x100
> +#define ISP_REG_OECF_X0_CFG1 0x104
> +#define ISP_REG_OECF_X0_CFG2 0x108
> +#define ISP_REG_OECF_X0_CFG3 0x10c
> +#define ISP_REG_OECF_X0_CFG4 0x110
> +#define ISP_REG_OECF_X0_CFG5 0x114
> +#define ISP_REG_OECF_X0_CFG6 0x118
> +#define ISP_REG_OECF_X0_CFG7 0x11c
> +
> +#define ISP_REG_OECF_Y3_CFG0 0x1e0
> +#define ISP_REG_OECF_Y3_CFG1 0x1e4
> +#define ISP_REG_OECF_Y3_CFG2 0x1e8
> +#define ISP_REG_OECF_Y3_CFG3 0x1ec
> +#define ISP_REG_OECF_Y3_CFG4 0x1f0
> +#define ISP_REG_OECF_Y3_CFG5 0x1f4
> +#define ISP_REG_OECF_Y3_CFG6 0x1f8
> +#define ISP_REG_OECF_Y3_CFG7 0x1fc
> +
> +#define ISP_REG_OECF_S0_CFG0 0x200
> +#define ISP_REG_OECF_S3_CFG7 0x27c
> +#define OCEF_PAR_H(n) ((n) << 16)
> +#define OCEF_PAR_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_X0_CFG_0 0x280
> +#define ISP_REG_AWB_X0_CFG_1 0x284
> +#define ISP_REG_AWB_X1_CFG_0 0x288
> +#define ISP_REG_AWB_X1_CFG_1 0x28c
> +#define ISP_REG_AWB_X2_CFG_0 0x290
> +#define ISP_REG_AWB_X2_CFG_1 0x294
> +#define ISP_REG_AWB_X3_CFG_0 0x298
> +#define ISP_REG_AWB_X3_CFG_1 0x29c
> +#define AWB_X_SYMBOL_H(n) ((n) << 16)
> +#define AWB_X_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_Y0_CFG_0 0x2a0
> +#define ISP_REG_AWB_Y0_CFG_1 0x2a4
> +#define ISP_REG_AWB_Y1_CFG_0 0x2a8
> +#define ISP_REG_AWB_Y1_CFG_1 0x2ac
> +#define ISP_REG_AWB_Y2_CFG_0 0x2b0
> +#define ISP_REG_AWB_Y2_CFG_1 0x2b4
> +#define ISP_REG_AWB_Y3_CFG_0 0x2b8
> +#define ISP_REG_AWB_Y3_CFG_1 0x2bc
> +#define AWB_Y_SYMBOL_H(n) ((n) << 16)
> +#define AWB_Y_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_AWB_S0_CFG_0 0x2c0
> +#define ISP_REG_AWB_S0_CFG_1 0x2c4
> +#define ISP_REG_AWB_S1_CFG_0 0x2c8
> +#define ISP_REG_AWB_S1_CFG_1 0x2cc
> +#define ISP_REG_AWB_S2_CFG_0 0x2d0
> +#define ISP_REG_AWB_S2_CFG_1 0x2d4
> +#define ISP_REG_AWB_S3_CFG_0 0x2d8
> +#define ISP_REG_AWB_S3_CFG_1 0x2dc
> +#define AWB_S_SYMBOL_H(n) ((n) << 16)
> +#define AWB_S_SYMBOL_L(n) ((n) << 0)
> +
> +#define ISP_REG_OBCG_CFG_0 0x2e0
> +#define ISP_REG_OBCG_CFG_1 0x2e4
> +#define ISP_REG_OBCG_CFG_2 0x2e8
> +#define ISP_REG_OBCG_CFG_3 0x2ec
> +#define ISP_REG_OBCO_CFG_0 0x2f0
> +#define ISP_REG_OBCO_CFG_1 0x2f4
> +#define ISP_REG_OBCO_CFG_2 0x2f8
> +#define ISP_REG_OBCO_CFG_3 0x2fc
> +#define GAIN_D_POINT(x) ((x) << 24)
> +#define GAIN_C_POINT(x) ((x) << 16)
> +#define GAIN_B_POINT(x) ((x) << 8)
> +#define GAIN_A_POINT(x) ((x) << 0)
> +#define OFFSET_D_POINT(x) ((x) << 24)
> +#define OFFSET_C_POINT(x) ((x) << 16)
> +#define OFFSET_B_POINT(x) ((x) << 8)
> +#define OFFSET_A_POINT(x) ((x) << 0)
> +
> +#define ISP_REG_ISP_CTRL_0 0xa00
> +#define ISPC_SCFEINT BIT(27)
> +#define ISPC_VSFWINT BIT(26)
> +#define ISPC_VSINT BIT(25)
> +#define ISPC_INTS BIT(24)
> +#define ISPC_ENUO BIT(20)
> +#define ISPC_ENLS BIT(17)
> +#define ISPC_ENSS1 BIT(12)
> +#define ISPC_ENSS0 BIT(11)
> +#define ISPC_RST BIT(1)
> +#define ISPC_EN BIT(0)
> +#define ISPC_RST_MASK BIT(1)
> +
> +#define ISP_REG_ISP_CTRL_1 0xa08
> +#define CTRL_SAT(n) ((n) << 28)
> +#define CTRL_DBC BIT(22)
> +#define CTRL_CTC BIT(21)
> +#define CTRL_YHIST BIT(20)
> +#define CTRL_YCURVE BIT(19)
> +#define CTRL_CTM BIT(18)
> +#define CTRL_BIYUV BIT(17)
> +#define CTRL_SCE BIT(8)
> +#define CTRL_EE BIT(7)
> +#define CTRL_CCE BIT(5)
> +#define CTRL_RGE BIT(4)
> +#define CTRL_CME BIT(3)
> +#define CTRL_AE BIT(2)
> +#define CTRL_CE BIT(1)
> +#define CTRL_SAT_MASK GENMASK(31, 28)
> +
> +#define ISP_REG_PIPELINE_XY_SIZE 0xa0c
> +#define H_ACT_CAP(n) ((n) << 16)
> +#define W_ACT_CAP(n) ((n) << 0)
> +
> +#define ISP_REG_ICTC 0xa10
> +#define GF_MODE(n) ((n) << 30)
> +#define MAXGT(n) ((n) << 16)
> +#define MINGT(n) ((n) << 0)
> +
> +#define ISP_REG_IDBC 0xa14
> +#define BADGT(n) ((n) << 16)
> +#define BADXT(n) ((n) << 0)
> +
> +#define ISP_REG_ICFAM 0xa1c
> +#define CROSS_COV(n) ((n) << 4)
> +#define HV_W(n) ((n) << 0)
> +
> +#define ISP_REG_CS_GAIN 0xa30
> +#define CMAD(n) ((n) << 16)
> +#define CMAB(n) ((n) << 0)
> +
> +#define ISP_REG_CS_THRESHOLD 0xa34
> +#define CMD(n) ((n) << 16)
> +#define CMB(n) ((n) << 0)
> +
> +#define ISP_REG_CS_OFFSET 0xa38
> +#define VOFF(n) ((n) << 16)
> +#define UOFF(n) ((n) << 0)
> +
> +#define ISP_REG_CS_HUE_F 0xa3c
> +#define SIN(n) ((n) << 16)
> +#define COS(n) ((n) << 0)
> +
> +#define ISP_REG_CS_SCALE 0xa40
> +#define CMSF(n) ((n) << 0)
> +
> +#define ISP_REG_IESHD 0xa50
> +#define SHAD_UP_M BIT(1)
> +#define SHAD_UP_EN BIT(0)
> +
> +#define ISP_REG_YADJ0 0xa54
> +#define YOIR(n) ((n) << 16)
> +#define YIMIN(n) ((n) << 0)
> +
> +#define ISP_REG_YADJ1 0xa58
> +#define YOMAX(n) ((n) << 16)
> +#define YOMIN(n) ((n) << 0)
> +
> +#define ISP_REG_Y_PLANE_START_ADDR 0xa80
> +#define ISP_REG_UV_PLANE_START_ADDR 0xa84
> +
> +#define ISP_REG_STRIDE 0xa88
> +#define IMG_STR(n) ((n) << 0)
For registers that contain a single field at offset 0, you don't need to
define a macro, you can pass the value directly to the register write
function (IMG_STR is actually not used :-)). Same above and/or below
where applicable.
> +
> +#define ISP_REG_ITIIWSR 0xb20
> +#define ITI_HSIZE(n) ((n) << 16)
> +#define ITI_WSIZE(n) ((n) << 0)
> +
> +#define ISP_REG_ITIDWLSR 0xb24
> +#define ITI_WSTRIDE(n) ((n) << 0)
> +
> +#define ISP_REG_ITIPDFR 0xb38
> +#define ITI_PACKAGE_FMT(n) ((n) << 0)
> +
> +#define ISP_REG_ITIDRLSR 0xb3C
> +#define ITI_STRIDE_L(n) ((n) << 0)
> +
> +#define ISP_REG_DNYUV_YSWR0 0xc00
> +#define ISP_REG_DNYUV_YSWR1 0xc04
> +#define ISP_REG_DNYUV_CSWR0 0xc08
> +#define ISP_REG_DNYUV_CSWR1 0xc0c
> +#define YUVSW5(n) ((n) << 20)
> +#define YUVSW4(n) ((n) << 16)
> +#define YUVSW3(n) ((n) << 12)
> +#define YUVSW2(n) ((n) << 8)
> +#define YUVSW1(n) ((n) << 4)
> +#define YUVSW0(n) ((n) << 0)
> +
> +#define ISP_REG_DNYUV_YDR0 0xc10
> +#define ISP_REG_DNYUV_YDR1 0xc14
> +#define ISP_REG_DNYUV_YDR2 0xc18
> +#define ISP_REG_DNYUV_CDR0 0xc1c
> +#define ISP_REG_DNYUV_CDR1 0xc20
> +#define ISP_REG_DNYUV_CDR2 0xc24
> +#define CURVE_D_H(n) ((n) << 16)
> +#define CURVE_D_L(n) ((n) << 0)
> +
> +#define ISP_REG_ICAMD_0 0xc40
> +#define DNRM_F(n) ((n) << 16)
> +#define ISP_REG_ICAMD_12 0xc70
> +#define ISP_REG_ICAMD_20 0xc90
> +#define ISP_REG_ICAMD_24 0xca0
> +#define ISP_REG_ICAMD_25 0xca4
> +#define CCM_M_DAT(n) ((n) << 0)
> +
> +#define ISP_REG_GAMMA_VAL0 0xe00
> +#define ISP_REG_GAMMA_VAL1 0xe04
> +#define ISP_REG_GAMMA_VAL2 0xe08
> +#define ISP_REG_GAMMA_VAL3 0xe0c
> +#define ISP_REG_GAMMA_VAL4 0xe10
> +#define ISP_REG_GAMMA_VAL5 0xe14
> +#define ISP_REG_GAMMA_VAL6 0xe18
> +#define ISP_REG_GAMMA_VAL7 0xe1c
> +#define ISP_REG_GAMMA_VAL8 0xe20
> +#define ISP_REG_GAMMA_VAL9 0xe24
> +#define ISP_REG_GAMMA_VAL10 0xe28
> +#define ISP_REG_GAMMA_VAL11 0xe2c
> +#define ISP_REG_GAMMA_VAL12 0xe30
> +#define ISP_REG_GAMMA_VAL13 0xe34
> +#define ISP_REG_GAMMA_VAL14 0xe38
> +#define GAMMA_S_VAL(n) ((n) << 16)
> +#define GAMMA_VAL(n) ((n) << 0)
> +
> +#define ISP_REG_R2Y_0 0xe40
> +#define ISP_REG_R2Y_1 0xe44
> +#define ISP_REG_R2Y_2 0xe48
> +#define ISP_REG_R2Y_3 0xe4c
> +#define ISP_REG_R2Y_4 0xe50
> +#define ISP_REG_R2Y_5 0xe54
> +#define ISP_REG_R2Y_6 0xe58
> +#define ISP_REG_R2Y_7 0xe5c
> +#define ISP_REG_R2Y_8 0xe60
> +#define CSC_M(n) ((n) << 0)
> +
> +#define ISP_REG_SHARPEN0 0xe80
> +#define ISP_REG_SHARPEN1 0xe84
> +#define ISP_REG_SHARPEN2 0xe88
> +#define ISP_REG_SHARPEN3 0xe8c
> +#define ISP_REG_SHARPEN4 0xe90
> +#define ISP_REG_SHARPEN5 0xe94
> +#define ISP_REG_SHARPEN6 0xe98
> +#define ISP_REG_SHARPEN7 0xe9c
> +#define ISP_REG_SHARPEN8 0xea0
> +#define ISP_REG_SHARPEN9 0xea4
> +#define ISP_REG_SHARPEN10 0xea8
> +#define ISP_REG_SHARPEN11 0xeac
> +#define ISP_REG_SHARPEN12 0xeb0
> +#define ISP_REG_SHARPEN13 0xeb4
> +#define ISP_REG_SHARPEN14 0xeb8
> +#define S_DELTA(n) ((n) << 16)
> +#define S_WEIGHT(n) ((n) << 8)
> +
> +#define ISP_REG_SHARPEN_FS0 0xebc
> +#define ISP_REG_SHARPEN_FS1 0xec0
> +#define ISP_REG_SHARPEN_FS2 0xec4
> +#define ISP_REG_SHARPEN_FS3 0xec8
> +#define ISP_REG_SHARPEN_FS4 0xecc
> +#define ISP_REG_SHARPEN_FS5 0xed0
> +#define S_FACTOR(n) ((n) << 24)
> +#define S_SLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_SHARPEN_WN 0xed4
> +#define PDIRF(n) ((n) << 28)
> +#define NDIRF(n) ((n) << 24)
> +#define WSUM(n) ((n) << 0)
> +
> +#define ISP_REG_IUVS1 0xed8
> +#define UVDIFF2(n) ((n) << 16)
> +#define UVDIFF1(n) ((n) << 0)
> +
> +#define ISP_REG_IUVS2 0xedc
> +#define UVF(n) ((n) << 24)
> +#define UVSLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_IUVCKS1 0xee0
> +#define UVCKDIFF2(n) ((n) << 16)
> +#define UVCKDIFF1(n) ((n) << 0)
> +
> +#define ISP_REG_IUVCKS2 0xee4
> +#define UVCKSLOPE(n) ((n) << 0)
> +
> +#define ISP_REG_ISHRPET 0xee8
> +#define TH(n) ((n) << 8)
> +#define EN(n) ((n) << 0)
> +
> +#define ISP_REG_YCURVE_0 0xf00
> +#define ISP_REG_YCURVE_63 0xffc
> +#define L_PARAM(n) ((n) << 0)
> +
> +#define IMAGE_MAX_WIDTH 1920
> +#define IMAGE_MAX_HEIGH 1080
> +
> +/* The output line of ISP */
What is an ISP "line" ?
> +enum isp_line_id {
> + STF_ISP_LINE_INVALID = -1,
> + STF_ISP_LINE_SRC = 1,
> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
> +};
> +
> +/* pad id for media framework */
> +enum isp_pad_id {
> + STF_ISP_PAD_SINK = 0,
> + STF_ISP_PAD_SRC,
> + STF_ISP_PAD_MAX
> +};
> +
> +enum {
> + EN_INT_NONE = 0,
> + EN_INT_ISP_DONE = (0x1 << 24),
> + EN_INT_CSI_DONE = (0x1 << 25),
> + EN_INT_SC_DONE = (0x1 << 26),
> + EN_INT_LINE_INT = (0x1 << 27),
> + EN_INT_ALL = (0xF << 24),
Please use lowercase for hex constants.
But I'm wondering if this is really needed, it seems to duplicate the
ISPC_* bits defined above. I would use those directly.
> +};
> +
> +enum {
> + INTERFACE_DVP = 0,
> + INTERFACE_CSI,
> +};
This isn't used.
> +
> +struct isp_format {
> + u32 code;
> + u8 bpp;
> +};
> +
> +struct isp_format_table {
> + const struct isp_format *fmts;
> + int nfmts;
> +};
> +
> +struct regval_t {
> + u32 addr;
> + u32 val;
> + u32 delay_ms;
> +};
> +
> +struct reg_table {
> + const struct regval_t *regval;
> + int regval_num;
> +};
Those two structures are not used.
> +
> +struct stf_isp_crop {
> + struct v4l2_rect rect;
> + u32 bpp;
> +};
> +
> +struct stf_isp_dev {
> + struct stfcamss *stfcamss;
> + struct v4l2_subdev subdev;
> + struct media_pad pads[STF_ISP_PAD_MAX];
> + struct v4l2_mbus_framefmt fmt[STF_ISP_PAD_MAX];
> + struct stf_isp_crop rect[STF_ISP_PAD_MAX];
> + const struct isp_format_table *formats;
> + unsigned int nformats;
> + struct mutex stream_lock; /* serialize stream control */
You should explicitly list the fields protected by the mutex.
> + int stream_count;
> +};
> +
> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev);
> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev);
> +int stf_isp_reset(struct stf_isp_dev *isp_dev);
> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev);
> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
> + struct stf_isp_crop *crop_array, u32 mcode);
> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev);
> +int stf_isp_subdev_init(struct stfcamss *stfcamss);
> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev);
> +int stf_isp_unregister(struct stf_isp_dev *isp_dev);
> +
> +#endif /* STF_ISP_H */
> diff --git a/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> new file mode 100644
> index 000000000000..2088b6bd0d61
> --- /dev/null
> +++ b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> @@ -0,0 +1,468 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf_isp_hw_ops.c
> + *
> + * Register interface file for StarFive ISP driver
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + *
> + */
> +
> +#include "stf_camss.h"
> +
> +static void stf_isp_config_obc(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG, OBC_W_H(11) | OBC_W_W(11));
> +
> + reg_val = GAIN_D_POINT(0x40) | GAIN_C_POINT(0x40) |
> + GAIN_B_POINT(0x40) | GAIN_A_POINT(0x40);
> + for (reg_add = ISP_REG_OBCG_CFG_0; reg_add <= ISP_REG_OBCG_CFG_3;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + reg_val = OFFSET_D_POINT(0) | OFFSET_C_POINT(0) |
> + OFFSET_B_POINT(0) | OFFSET_A_POINT(0);
> + for (reg_add = ISP_REG_OBCO_CFG_0; reg_add <= ISP_REG_OBCO_CFG_3;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_oecf(struct stfcamss *stfcamss)
> +{
> + u32 reg_add, par_val;
> + u16 par_h, par_l;
> +
> + par_h = 0x10; par_l = 0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG0; reg_add <= ISP_REG_OECF_Y3_CFG0;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x40; par_l = 0x20;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG1; reg_add <= ISP_REG_OECF_Y3_CFG1;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x80; par_l = 0x60;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG2; reg_add <= ISP_REG_OECF_Y3_CFG2;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0xc0; par_l = 0xa0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG3; reg_add <= ISP_REG_OECF_Y3_CFG3;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x100; par_l = 0xe0;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG4; reg_add <= ISP_REG_OECF_Y3_CFG4;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x200; par_l = 0x180;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG5; reg_add <= ISP_REG_OECF_Y3_CFG5;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x300; par_l = 0x280;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG6; reg_add <= ISP_REG_OECF_Y3_CFG6;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x3fe; par_l = 0x380;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_X0_CFG7; reg_add <= ISP_REG_OECF_Y3_CFG7;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 0x20;
> + }
> +
> + par_h = 0x80; par_l = 0x80;
> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
> + for (reg_add = ISP_REG_OECF_S0_CFG0; reg_add <= ISP_REG_OECF_S3_CFG7;) {
> + stf_isp_reg_write(stfcamss, reg_add, par_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_lccf(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
> + Y_DISTANCE(0x21C) | X_DISTANCE(0x3C0));
> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1, LCCF_MAX_DIS(0xb));
> +
> + for (reg_add = ISP_REG_LCCF_CFG_2; reg_add <= ISP_REG_LCCF_CFG_5;) {
> + stf_isp_reg_write(stfcamss, reg_add,
> + LCCF_F2_PAR(0x0) | LCCF_F1_PAR(0x0));
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_awb(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> + u16 symbol_h, symbol_l;
> +
> + symbol_h = 0x0; symbol_l = 0x0;
> + reg_val = AWB_X_SYMBOL_H(symbol_h) | AWB_X_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_X0_CFG_0; reg_add <= ISP_REG_AWB_X3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + symbol_h = 0x0, symbol_l = 0x0;
> + reg_val = AWB_Y_SYMBOL_H(symbol_h) | AWB_Y_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_Y0_CFG_0; reg_add <= ISP_REG_AWB_Y3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +
> + symbol_h = 0x80, symbol_l = 0x80;
> + reg_val = AWB_S_SYMBOL_H(symbol_h) | AWB_S_SYMBOL_L(symbol_l);
> +
> + for (reg_add = ISP_REG_AWB_S0_CFG_0; reg_add <= ISP_REG_AWB_S3_CFG_1;) {
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + }
> +}
> +
> +static void stf_isp_config_grgb(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
> + GF_MODE(1) | MAXGT(0x140) | MINGT(0x40));
> + stf_isp_reg_write(stfcamss, ISP_REG_IDBC, BADGT(0x200) | BADXT(0x200));
> +}
> +
> +static void stf_isp_config_cfa(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG,
> + SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) | SMY3(2) |
> + SMY2(3) | SMY1(2) | SMY0(3));
> + stf_isp_reg_write(stfcamss, ISP_REG_ICFAM, CROSS_COV(3) | HV_W(2));
> +}
> +
> +static void stf_isp_config_ccm(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6) | CCM_M_DAT(0));
> +
> + for (reg_add = ISP_REG_ICAMD_12; reg_add <= ISP_REG_ICAMD_20;) {
> + stf_isp_reg_write(stfcamss, reg_add, CCM_M_DAT(0x80));
> + reg_add += 0x10;
> + }
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_24, CCM_M_DAT(0x700));
> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_25, CCM_M_DAT(0x200));
> +}
> +
> +static void stf_isp_config_gamma(struct stfcamss *stfcamss)
> +{
> + u32 reg_val, reg_add;
> + u16 gamma_slope_v, gamma_v;
> +
> + gamma_slope_v = 0x2400; gamma_v = 0x0;
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL0, reg_val);
> +
> + gamma_slope_v = 0x800; gamma_v = 0x20;
> + for (reg_add = ISP_REG_GAMMA_VAL1; reg_add <= ISP_REG_GAMMA_VAL7;) {
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + gamma_v += 0x20;
> + }
> +
> + gamma_v = 0x100;
> + for (reg_add = ISP_REG_GAMMA_VAL8; reg_add <= ISP_REG_GAMMA_VAL13;) {
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
> + reg_add += 4;
> + gamma_v += 0x80;
> + }
> +
> + gamma_v = 0x3fe;
> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL14, reg_val);
> +}
> +
> +static void stf_isp_config_r2y(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_0, CSC_M(0x4C));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_1, CSC_M(0x97));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_2, CSC_M(0x1d));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_3, CSC_M(0x1d5));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_4, CSC_M(0x1ac));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_5, CSC_M(0x80));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_6, CSC_M(0x80));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_7, CSC_M(0x194));
> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_8, CSC_M(0x1ec));
> +}
> +
> +static void stf_isp_config_y_curve(struct stfcamss *stfcamss)
> +{
> + u32 reg_add;
> + u16 y_curve;
> +
> + y_curve = 0x0;
> + for (reg_add = ISP_REG_YCURVE_0; reg_add <= ISP_REG_YCURVE_63;) {
> + stf_isp_reg_write(stfcamss, reg_add, y_curve);
> + reg_add += 4;
> + y_curve += 0x10;
> + }
> +}
> +
> +static void stf_isp_config_sharpen(struct stfcamss *sc)
> +{
> + u32 reg_add;
> +
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN0, S_DELTA(0x7) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN1, S_DELTA(0x18) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN2, S_DELTA(0x80) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN3, S_DELTA(0x100) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN4, S_DELTA(0x10) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN5, S_DELTA(0x60) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN6, S_DELTA(0x100) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN7, S_DELTA(0x190) | S_WEIGHT(0xf));
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN8, S_DELTA(0x0) | S_WEIGHT(0xf));
> +
> + for (reg_add = ISP_REG_SHARPEN9; reg_add <= ISP_REG_SHARPEN14;) {
> + stf_isp_reg_write(sc, reg_add, S_WEIGHT(0xf));
> + reg_add += 4;
> + }
> +
> + for (reg_add = ISP_REG_SHARPEN_FS0; reg_add <= ISP_REG_SHARPEN_FS5;) {
> + stf_isp_reg_write(sc, reg_add, S_FACTOR(0x10) | S_SLOPE(0x0));
> + reg_add += 4;
> + }
> +
> + stf_isp_reg_write(sc, ISP_REG_SHARPEN_WN,
> + PDIRF(0x8) | NDIRF(0x8) | WSUM(0xd7c));
> + stf_isp_reg_write(sc, ISP_REG_IUVS1, UVDIFF2(0xC0) | UVDIFF1(0x40));
> + stf_isp_reg_write(sc, ISP_REG_IUVS2, UVF(0xff) | UVSLOPE(0x0));
> + stf_isp_reg_write(sc, ISP_REG_IUVCKS1,
> + UVCKDIFF2(0xa0) | UVCKDIFF1(0x40));
> +}
> +
> +static void stf_isp_config_dnyuv(struct stfcamss *stfcamss)
> +{
> + u32 reg_val;
> +
> + reg_val = YUVSW5(7) | YUVSW4(7) | YUVSW3(7) | YUVSW2(7) |
> + YUVSW1(7) | YUVSW0(7);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0, reg_val);
> +
> + reg_val = YUVSW3(7) | YUVSW2(7) | YUVSW1(7) | YUVSW0(7);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1, reg_val);
> +
> + reg_val = CURVE_D_H(0x60) | CURVE_D_L(0x40);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0, reg_val);
> +
> + reg_val = CURVE_D_H(0xd8) | CURVE_D_L(0x90);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1, reg_val);
> +
> + reg_val = CURVE_D_H(0x1e6) | CURVE_D_L(0x144);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2, reg_val);
> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2, reg_val);
> +}
> +
> +static void stf_isp_config_sat(struct stfcamss *stfcamss)
> +{
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN, CMAD(0x0) | CMAB(0x100));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD, CMD(0x1f) | CMB(0x1));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET, VOFF(0x0) | UOFF(0x0));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F, SIN(0x0) | COS(0x100));
> + stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, CMSF(0x8));
> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ0, YOIR(0x401) | YIMIN(0x1));
> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
> +}
> +
> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + clk_prepare_enable(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
Missing error handling.
Is there a specific hardware requirement to split the clock enabling and
reset deassertion in two, with some of them handled in the runtime PM
resume handler and the rest here ? If not, moving all clock and reset
control to the runtime PM handlers would simplify the code.
> +
> + return 0;
> +}
> +
> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
> +
> + return 0;
> +}
> +
> +int stf_isp_reset(struct stf_isp_dev *isp_dev)
> +{
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_RST_MASK, ISPC_RST);
> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_RST_MASK, 0);
> +
> + return 0;
> +}
> +
> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
> +{
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
> + DEC_V_KEEP(0x0) |
> + DEC_V_PERIOD(0x0) |
> + DEC_H_KEEP(0x0) |
> + DEC_H_PERIOD(0x0));
> +
> + stf_isp_config_obc(isp_dev->stfcamss);
> + stf_isp_config_oecf(isp_dev->stfcamss);
> + stf_isp_config_lccf(isp_dev->stfcamss);
> + stf_isp_config_awb(isp_dev->stfcamss);
> + stf_isp_config_grgb(isp_dev->stfcamss);
> + stf_isp_config_cfa(isp_dev->stfcamss);
> + stf_isp_config_ccm(isp_dev->stfcamss);
> + stf_isp_config_gamma(isp_dev->stfcamss);
> + stf_isp_config_r2y(isp_dev->stfcamss);
> + stf_isp_config_y_curve(isp_dev->stfcamss);
> + stf_isp_config_sharpen(isp_dev->stfcamss);
> + stf_isp_config_dnyuv(isp_dev->stfcamss);
> + stf_isp_config_sat(isp_dev->stfcamss);
All these parameters are hardcoded, why are they not exposed to
userspace ?
> +
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
> +}
> +
> +static void stf_isp_config_crop(struct stfcamss *stfcamss,
> + struct stf_isp_crop *crop)
> +{
> + struct v4l2_rect *rect = &crop[STF_ISP_PAD_SRC].rect;
> + u32 bpp = crop[STF_ISP_PAD_SRC].bpp;
> + u32 val;
> +
> + val = VSTART_CAP(rect->top) | HSTART_CAP(rect->left);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_START_CFG, val);
> +
> + val = VEND_CAP(rect->height + rect->top - 1) |
> + HEND_CAP(rect->width + rect->left - 1);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_END_CFG, val);
> +
> + val = H_ACT_CAP(rect->height) | W_ACT_CAP(rect->width);
> + stf_isp_reg_write(stfcamss, ISP_REG_PIPELINE_XY_SIZE, val);
> +
> + val = ALIGN(rect->width * bpp / 8, STFCAMSS_FRAME_WIDTH_ALIGN_8);
> + stf_isp_reg_write(stfcamss, ISP_REG_STRIDE, val);
> +}
> +
> +static void stf_isp_config_raw_fmt(struct stfcamss *stfcamss, u32 mcode)
> +{
> + u32 val, val1;
> +
> + switch (mcode) {
> + case MEDIA_BUS_FMT_SRGGB10_1X10:
> + case MEDIA_BUS_FMT_SRGGB8_1X8:
> + /* 3 2 3 2 1 0 1 0 B Gb B Gb Gr R Gr R */
> + val = SMY13(3) | SMY12(2) | SMY11(3) | SMY10(2) |
> + SMY3(1) | SMY2(0) | SMY1(1) | SMY0(0);
> + val1 = CTRL_SAT(0x0);
> + break;
> + case MEDIA_BUS_FMT_SGRBG10_1X10:
> + case MEDIA_BUS_FMT_SGRBG8_1X8:
> + /* 2 3 2 3 0 1 0 1, Gb B Gb B R Gr R Gr */
> + val = SMY13(2) | SMY12(3) | SMY11(2) | SMY10(3) |
> + SMY3(0) | SMY2(1) | SMY1(0) | SMY0(1);
> + val1 = CTRL_SAT(0x2);
> + break;
> + case MEDIA_BUS_FMT_SGBRG10_1X10:
> + case MEDIA_BUS_FMT_SGBRG8_1X8:
> + /* 1 0 1 0 3 2 3 2, Gr R Gr R B Gb B Gb */
> + val = SMY13(1) | SMY12(0) | SMY11(1) | SMY10(0) |
> + SMY3(3) | SMY2(2) | SMY1(3) | SMY0(2);
> + val1 = CTRL_SAT(0x3);
> + break;
> + case MEDIA_BUS_FMT_SBGGR10_1X10:
> + case MEDIA_BUS_FMT_SBGGR8_1X8:
> + /* 0 1 0 1 2 3 2 3 R Gr R Gr Gb B Gb B */
> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
> + val1 = CTRL_SAT(0x1);
> + break;
> + default:
> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
> + val1 = CTRL_SAT(0x1);
> + break;
> + }
> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG, val);
> + stf_isp_reg_set_bit(stfcamss, ISP_REG_ISP_CTRL_1, CTRL_SAT_MASK, val1);
> +}
> +
> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
> + struct stf_isp_crop *crop, u32 mcode)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + stf_isp_config_crop(stfcamss, crop);
> + stf_isp_config_raw_fmt(stfcamss, mcode);
> +
> + stf_isp_reg_set_bit(stfcamss, ISP_REG_DUMP_CFG_1,
> + DUMP_BURST_LEN_MASK | DUMP_SD_MASK,
> + DUMP_BURST_LEN(3));
> +
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIIWSR,
> + ITI_HSIZE(IMAGE_MAX_HEIGH) |
> + ITI_WSIZE(IMAGE_MAX_WIDTH));
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDWLSR, ITI_WSTRIDE(0x960));
> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDRLSR, ITI_STRIDE_L(0x960));
> + stf_isp_reg_write(stfcamss, ISP_REG_SENSOR, 0x1);
Should 0x1 be IMAGER_SEL(1) or are those unrelated ?
> +}
> +
> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev)
> +{
> + struct stfcamss *stfcamss = isp_dev->stfcamss;
> +
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS | ISPC_RST, 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS, 10);
> + stf_isp_reg_write(stfcamss, ISP_REG_IESHD, SHAD_UP_M);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
> + ISPC_ENUO | ISPC_ENLS | ISPC_EN, 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
> + CSI_INTS(1) | CSI_SHA_M(4), 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
> + CSI_INTS(2) | CSI_SHA_M(4), 10);
> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSI_INPUT_EN_AND_STATUS,
> + CSI_EN_S, 10);
> +}
--
Regards,
Laurent Pinchart
Hi Laurent,
Thank you for your comments.
On 2023/7/27 18:23, Laurent Pinchart wrote:
> On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote:
>> On 19/06/2023 12:28, Jack Zhu wrote:
>> > Add starfive_camss.rst file that documents the Starfive Camera
>> > Subsystem driver which is used for handing image sensor data.
>> >
>> > Signed-off-by: Jack Zhu <[email protected]>
>> > ---
>> > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
>> > .../media/starfive_camss_graph.dot | 16 ++++++
>> > .../admin-guide/media/v4l-drivers.rst | 1 +
>> > MAINTAINERS | 1 +
>> > 4 files changed, 75 insertions(+)
>> > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
>> > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
>> >
>> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
>> > new file mode 100644
>> > index 000000000000..a6378849384f
>> > --- /dev/null
>> > +++ b/Documentation/admin-guide/media/starfive_camss.rst
>> > @@ -0,0 +1,57 @@
>> > +.. SPDX-License-Identifier: GPL-2.0
>> > +
>> > +.. include:: <isonum.txt>
>> > +
>> > +================================
>> > +Starfive Camera Subsystem driver
>> > +================================
>> > +
>> > +Introduction
>> > +------------
>> > +
>> > +This file documents the driver for the Starfive Camera Subsystem found on
>> > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/
>> > +starfive.
>> > +
>> > +The driver implements V4L2, Media controller and v4l2_subdev interfaces.
>> > +Camera sensor using V4L2 subdev interface in the kernel is supported.
>> > +
>> > +The driver has been successfully used on the Gstreamer 1.18.5 with
>> > +v4l2src plugin.
>> > +
>> > +
>> > +Starfive Camera Subsystem hardware
>> > +----------------------------------
>> > +
>> > +The Starfive Camera Subsystem hardware consists of:
>> > +
>> > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
>>
>> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd
>> be consistent with one - recommend MIPI throughout your documentation.
>
> I'd drop the "mipi" and just write "data", not "MIPI data".
>
Ok, I will modify it.
>> > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
>
> It's "CSI-2", not "CSI2".
Ok, I will modify it.
>
>> > + based camera sensor data stream.
>> > +- ISP: handles the image data streams from the MIPI CSIRx Controller.
>>
>> Maybe you've done this elsewhere but, it would be worthwhile describing
>> what the ISP does. Debayering ? 3As ? Just raw dumps ?
>>
>> > +- VIN(Video In): a top-level module, is responsible for controlling power
>> > + and clocks to other modules, dumps the input data to memory or transfers the
>> > + input data to ISP.
>
> A block diagram could be nice. Let me give it a try, based on my
> understanding:
>
> Starfive Camera Subsystem hardware
> ----------------------------------
>
> The Starfive Camera Subsystem hardware consists of::
>
> +---------------------------------------------+
> | VIN |
> +--------+ | +-------+ +-------+ +-----+ |
> | Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | |
> | Sensor | | | RX | | RX | | +-----+ |
> +--------+ | +-------+ +-------+ | |
> | | +-----+ +-----+ |
> | \->| ISP |-->| DMA | |
> | +-----+ +-----+ |
> +---------------------------------------------+
>
> - VIN (Video IN): The top-level module, responsible for controlling
> power and clocks, and routing data between modules.
>
> - D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2
> camera sensor.
>
> - CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding
> the CSI-2 protocol.
>
> - ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and
> producing YUV frames.
>
>
> I haven't tried compiling this, so please fix formatting if anything is
> wrong (and of course feel free to adapt the diagram and text).
>
Ok, I will add a block diagram.
>> > +
>> > +
>> > +Topology
>> > +--------
>> > +
>> > +The media controller pipeline graph is as follows:
>> > +
>> > +.. _starfive_camss_graph:
>> > +
>> > +.. kernel-figure:: starfive_camss_graph.dot
>> > + :alt: starfive_camss_graph.dot
>> > + :align: center
>> > +
>> > +The driver has 2 video devices:
>> > +
>> > +- stf_vin0_wr_video0: capture device for images directly from the VIN module.
>>
>> OK so you can get the raw images.
>>
>> > +- stf_vin0_isp0_video1: capture device for images without scaling.
>> > +
>> > +The driver has 3 subdevices:
>> > +
>> > +- stf_isp0: is responsible for all the isp operations.
>> > +- stf_vin0_wr: used to dump RAW images to memory.
>> > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
>>
>> But what is being output here ? RGB, jpeg, YUV ?
>> It would be worth adding a few bits of text to describe that so you
>> don't have to dive into code to understand it.
>
--
Regards,
Jack Zhu
Hi Laurent,
Thank you for your comments.
On 2023/7/27 19:33, Laurent Pinchart wrote:
> Hi Jack,
>
> Thank you for the patch.
>
> On Mon, Jun 19, 2023 at 07:28:35PM +0800, Jack Zhu wrote:
>> Add basic platform driver for StarFive Camera Subsystem.
>>
>> Reviewed-by: Bryan O'Donoghue <[email protected]>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> MAINTAINERS | 1 +
>> drivers/media/platform/Kconfig | 1 +
>> drivers/media/platform/Makefile | 1 +
>> drivers/media/platform/starfive/Kconfig | 5 +
>> drivers/media/platform/starfive/Makefile | 2 +
>> drivers/media/platform/starfive/camss/Kconfig | 16 +
>> .../media/platform/starfive/camss/Makefile | 8 +
>> .../media/platform/starfive/camss/stf_camss.c | 338 ++++++++++++++++++
>> .../media/platform/starfive/camss/stf_camss.h | 146 ++++++++
>> 9 files changed, 518 insertions(+)
>> create mode 100644 drivers/media/platform/starfive/Kconfig
>> create mode 100644 drivers/media/platform/starfive/Makefile
>> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
>> create mode 100644 drivers/media/platform/starfive/camss/Makefile
>> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 71291dc58671..62f9e3593eab 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -20077,6 +20077,7 @@ L: [email protected]
>> S: Maintained
>> F: Documentation/admin-guide/media/starfive_camss.rst
>> F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
>> +F: drivers/media/platform/starfive/camss
>>
>> STARFIVE DEVICETREES
>> M: Emil Renner Berthing <[email protected]>
>> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
>> index ee579916f874..627eaa0ab3ee 100644
>> --- a/drivers/media/platform/Kconfig
>> +++ b/drivers/media/platform/Kconfig
>> @@ -80,6 +80,7 @@ source "drivers/media/platform/renesas/Kconfig"
>> source "drivers/media/platform/rockchip/Kconfig"
>> source "drivers/media/platform/samsung/Kconfig"
>> source "drivers/media/platform/st/Kconfig"
>> +source "drivers/media/platform/starfive/Kconfig"
>> source "drivers/media/platform/sunxi/Kconfig"
>> source "drivers/media/platform/ti/Kconfig"
>> source "drivers/media/platform/verisilicon/Kconfig"
>> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
>> index 5453bb868e67..5a038498a370 100644
>> --- a/drivers/media/platform/Makefile
>> +++ b/drivers/media/platform/Makefile
>> @@ -23,6 +23,7 @@ obj-y += renesas/
>> obj-y += rockchip/
>> obj-y += samsung/
>> obj-y += st/
>> +obj-y += starfive/
>> obj-y += sunxi/
>> obj-y += ti/
>> obj-y += verisilicon/
>> diff --git a/drivers/media/platform/starfive/Kconfig b/drivers/media/platform/starfive/Kconfig
>> new file mode 100644
>> index 000000000000..7955c2a0a4a3
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/Kconfig
>> @@ -0,0 +1,5 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +
>> +comment "StarFive media platform drivers"
>> +
>> +source "drivers/media/platform/starfive/camss/Kconfig"
>> diff --git a/drivers/media/platform/starfive/Makefile b/drivers/media/platform/starfive/Makefile
>> new file mode 100644
>> index 000000000000..4639fa1bca32
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/Makefile
>> @@ -0,0 +1,2 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +obj-y += camss/
>> diff --git a/drivers/media/platform/starfive/camss/Kconfig b/drivers/media/platform/starfive/camss/Kconfig
>> new file mode 100644
>> index 000000000000..dafe1d24324b
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/Kconfig
>> @@ -0,0 +1,16 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +config VIDEO_STARFIVE_CAMSS
>> + tristate "Starfive Camera Subsystem driver"
>> + depends on V4L_PLATFORM_DRIVERS
>> + depends on VIDEO_DEV && OF
>> + depends on HAS_DMA
>
> You need to depend on PM, otherwise the runtime PM operations will be
> no-ops and the driver won't work as clocks won't be enabled.
>
OK, I will add dependency.
>> + select MEDIA_CONTROLLER
>> + select VIDEO_V4L2_SUBDEV_API
>> + select VIDEOBUF2_DMA_CONTIG
>> + select V4L2_FWNODE
>> + help
>> + Enable this to support for the Starfive Camera subsystem
>> + found on Starfive JH7110 SoC.
>> +
>> + To compile this driver as a module, choose M here: the
>> + module will be called stf-camss.
>> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> new file mode 100644
>> index 000000000000..d56ddd078a71
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/Makefile
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +#
>> +# Makefile for StarFive Camera Subsystem driver
>> +#
>> +
>> +starfive-camss-objs += stf_camss.o
>> +
>> +obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> new file mode 100644
>> index 000000000000..dc2b5dba7bd4
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> @@ -0,0 +1,338 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_camss.c
>> + *
>> + * Starfive Camera Subsystem driver
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +#include <linux/module.h>
>> +#include <linux/of.h>
>> +#include <linux/of_graph.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/videodev2.h>
>> +#include <media/v4l2-fwnode.h>
>> +#include <media/v4l2-mc.h>
>> +
>> +#include "stf_camss.h"
>> +
>> +static const char * const stfcamss_clocks[] = {
>> + "clk_apb_func",
>> + "clk_wrapper_clk_c",
>> + "clk_dvp_inv",
>> + "clk_axiwr",
>> + "clk_mipi_rx0_pxl",
>> + "clk_ispcore_2x",
>> + "clk_isp_axi",
>> +};
>> +
>> +static const char * const stfcamss_resets[] = {
>> + "rst_wrapper_p",
>> + "rst_wrapper_c",
>> + "rst_axird",
>> + "rst_axiwr",
>> + "rst_isp_top_n",
>> + "rst_isp_top_axi",
>> +};
>> +
>> +static int stfcamss_get_mem_res(struct platform_device *pdev,
>> + struct stfcamss *stfcamss)
>> +{
>> + stfcamss->syscon_base =
>> + devm_platform_ioremap_resource_byname(pdev, "syscon");
>> + if (IS_ERR(stfcamss->syscon_base))
>> + return PTR_ERR(stfcamss->syscon_base);
>> +
>> + stfcamss->isp_base =
>> + devm_platform_ioremap_resource_byname(pdev, "isp");
>
> This hols on a single line:
>
> stfcamss->isp_base = devm_platform_ioremap_resource_byname(pdev, "isp");
>
> I would do the same above too.
>
OK, I will modify.
>> + if (IS_ERR(stfcamss->isp_base))
>> + return PTR_ERR(stfcamss->isp_base);
>> +
>> + return 0;
>> +}
>> +
>> +/*
>> + * stfcamss_of_parse_endpoint_node - Parse port endpoint node
>> + * @dev: Device
>> + * @node: Device node to be parsed
>> + * @csd: Parsed data from port endpoint node
>> + *
>> + * Return 0 on success or a negative error code on failure
>> + */
>> +static int stfcamss_of_parse_endpoint_node(struct device *dev,
>
> For consistency, pass the stfcamss pointer to this function, not the
> struct device.
>
OK, will fix it.
>> + struct device_node *node,
>> + struct stfcamss_async_subdev *csd)
>> +{
>> + struct v4l2_fwnode_endpoint vep = { { 0 } };
>> +
>> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(node), &vep);
>> + dev_dbg(dev, "vep.base.port = %d, id = %d\n",
>> + vep.base.port, vep.base.id);
>
> There are debug messages in v4l2_fwnode_endpoint_parse() already, you
> can drop this one.
>
OK, will drop it.
>> +
>> + csd->port = vep.base.port;
>> +
>> + return 0;
>> +}
>> +
>> +/*
>> + * stfcamss_of_parse_ports - Parse ports node
>> + * @stfcamss: STFCAMSS device
>> + *
>> + * Return number of "port" nodes found in "ports" node
>> + */
>> +static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
>> +{
>> + struct device *dev = stfcamss->dev;
>> + struct device_node *node = NULL;
>> + int ret, num_subdevs = 0;
>> +
>> + for_each_endpoint_of_node(dev->of_node, node) {
>
> for_each_endpoint_of_node(stfcamss->dev->of_node, node) {
>
> and drop the local dev variable.
>
OK, will modify it.
>> + struct stfcamss_async_subdev *csd;
>> +
>> + if (!of_device_is_available(node))
>> + continue;
>> +
>> + csd = v4l2_async_nf_add_fwnode_remote(&stfcamss->notifier,
>> + of_fwnode_handle(node),
>> + struct stfcamss_async_subdev);
>> + if (IS_ERR(csd)) {
>> + ret = PTR_ERR(csd);
>> + goto err_cleanup;
>> + }
>> +
>> + ret = stfcamss_of_parse_endpoint_node(dev, node, csd);
>> + if (ret < 0)
>> + goto err_cleanup;
>> +
>> + num_subdevs++;
>> + }
>> +
>> + return num_subdevs;
>> +
>> +err_cleanup:
>> + of_node_put(node);
>> + return ret;
>> +}
>> +
>> +static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
>> + struct v4l2_subdev *subdev,
>> + struct v4l2_async_subdev *asd)
>> +{
>> + struct media_pad *pad[STF_PADS_NUM];
>> + unsigned int i, pad_num = 0;
>> +
>> + for (i = 0; i < pad_num; ++i) {
>
> The loop never runs as pad_num is 0. This is modified in subsequent
> patches in the series, but it makes it hard to review the patches
> individually. Is there any way you could restructure the series to avoid
> this ?
>
OK, I will modify the code.
>> + int ret;
>> +
>> + ret = v4l2_create_fwnode_links_to_pad(subdev, pad[i], 0);
>> + if (ret < 0)
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int stfcamss_subdev_notifier_complete(struct v4l2_async_notifier *ntf)
>> +{
>> + struct stfcamss *stfcamss =
>> + container_of(ntf, struct stfcamss, notifier);
>> +
>> + return v4l2_device_register_subdev_nodes(&stfcamss->v4l2_dev);
>> +}
>> +
>> +static const struct v4l2_async_notifier_operations
>> +stfcamss_subdev_notifier_ops = {
>> + .bound = stfcamss_subdev_notifier_bound,
>> + .complete = stfcamss_subdev_notifier_complete,
>> +};
>> +
>> +static const struct media_device_ops stfcamss_media_ops = {
>> + .link_notify = v4l2_pipeline_link_notify,
>
> You can drop this, link management notification only serves to call the
> subdev .s_power() operations, which are a deprecated API.
>
OK, will drop it.
>> +};
>> +
>> +static void stfcamss_mc_init(struct platform_device *pdev,
>> + struct stfcamss *stfcamss)
>> +{
>> + stfcamss->media_dev.dev = stfcamss->dev;
>> + strscpy(stfcamss->media_dev.model, "Starfive Camera Subsystem",
>> + sizeof(stfcamss->media_dev.model));
>> + stfcamss->media_dev.ops = &stfcamss_media_ops;
>> + media_device_init(&stfcamss->media_dev);
>> +
>> + stfcamss->v4l2_dev.mdev = &stfcamss->media_dev;
>> +}
>> +
>> +/*
>> + * stfcamss_probe - Probe STFCAMSS platform device
>> + * @pdev: Pointer to STFCAMSS platform device
>> + *
>> + * Return 0 on success or a negative error code on failure
>> + */
>> +static int stfcamss_probe(struct platform_device *pdev)
>> +{
>> + struct stfcamss *stfcamss;
>> + struct device *dev = &pdev->dev;
>> + int ret, num_subdevs;
>> + unsigned int i;
>> +
>> + stfcamss = devm_kzalloc(dev, sizeof(*stfcamss), GFP_KERNEL);
>> + if (!stfcamss)
>> + return -ENOMEM;
>> +
>> + for (i = 0; i < ARRAY_SIZE(stfcamss->irq); ++i) {
>> + stfcamss->irq[i] = platform_get_irq(pdev, i);
>> + if (stfcamss->irq[i] < 0)
>> + return dev_err_probe(&pdev->dev, stfcamss->irq[i],
>> + "Failed to get irq%d", i);
>> + }
>> +
>> + stfcamss->nclks = ARRAY_SIZE(stfcamss->sys_clk);
>> + for (i = 0; i < stfcamss->nclks; ++i)
>> + stfcamss->sys_clk[i].id = stfcamss_clocks[i];
>> + ret = devm_clk_bulk_get(dev, stfcamss->nclks, stfcamss->sys_clk);
>> + if (ret) {
>> + dev_err(dev, "Failed to get clk controls\n");
>> + return ret;
>> + }
>> +
>> + stfcamss->nrsts = ARRAY_SIZE(stfcamss->sys_rst);
>> + for (i = 0; i < stfcamss->nrsts; ++i)
>> + stfcamss->sys_rst[i].id = stfcamss_resets[i];
>> + ret = devm_reset_control_bulk_get_shared(dev, stfcamss->nrsts,
>> + stfcamss->sys_rst);
>> + if (ret) {
>> + dev_err(dev, "Failed to get reset controls\n");
>> + return ret;
>> + }
>> +
>> + ret = stfcamss_get_mem_res(pdev, stfcamss);
>> + if (ret) {
>> + dev_err(dev, "Could not map registers\n");
>> + return ret;
>> + }
>> +
>> + stfcamss->dev = dev;
>
> Move this right after allocating stfcamss, and drop the pdev argument to
> stfcamss_get_mem_res(). The platform device can be retrieved in the
> function using to_platform_device().
>
OK, I will modify.
>> + platform_set_drvdata(pdev, stfcamss);
>> +
>> + v4l2_async_nf_init(&stfcamss->notifier);
>> +
>> + num_subdevs = stfcamss_of_parse_ports(stfcamss);
>> + if (num_subdevs < 0) {
>> + ret = -ENODEV;
>
> An error message would be useful, silent errors are hard to debug.
>
OK, will add error printing information.
>> + goto err_cleanup_notifier;
>> + }
>> +
>> + stfcamss_mc_init(pdev, stfcamss);
>> +
>> + ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
>> + goto err_cleanup_notifier;
>> + }
>> +
>> + ret = media_device_register(&stfcamss->media_dev);
>> + if (ret) {
>> + dev_err(dev, "Failed to register media device: %d\n", ret);
>> + goto err_unregister_device;
>> + }
>> +
>> + pm_runtime_enable(dev);
>
> Would it be useful to enable autosuspend too, to avoid expensive
> suspend/resume cycles when userspace wants to briefly stop capture and
> restart it immediately ?
>
It seems rare to use autosuspend in the Linux camera system.
>> +
>> + stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
>> + ret = v4l2_async_nf_register(&stfcamss->v4l2_dev, &stfcamss->notifier);
>> + if (ret) {
>> + dev_err(dev, "Failed to register async subdev nodes: %d\n",
>> + ret);
>> + goto err_unregister_media_dev;
>
> You need to disable runtime PM in this error path.
>
OK, will fix it.
>> + }
>> +
>> + return 0;
>> +
>> +err_unregister_media_dev:
>> + media_device_unregister(&stfcamss->media_dev);
>> +err_unregister_device:
>> + v4l2_device_unregister(&stfcamss->v4l2_dev);
>> +err_cleanup_notifier:
>> + v4l2_async_nf_cleanup(&stfcamss->notifier);
>> + return ret;
>> +}
>> +
>> +/*
>> + * stfcamss_remove - Remove STFCAMSS platform device
>> + * @pdev: Pointer to STFCAMSS platform device
>> + *
>> + * Always returns 0.
>> + */
>> +static int stfcamss_remove(struct platform_device *pdev)
>> +{
>> + struct stfcamss *stfcamss = platform_get_drvdata(pdev);
>> +
>> + v4l2_device_unregister(&stfcamss->v4l2_dev);
>> + media_device_cleanup(&stfcamss->media_dev);
>> + pm_runtime_disable(&pdev->dev);
>> +
>> + return 0;
>> +}
>> +
>> +static const struct of_device_id stfcamss_of_match[] = {
>> + { .compatible = "starfive,jh7110-camss" },
>> + { /* sentinel */ },
>> +};
>> +
>> +MODULE_DEVICE_TABLE(of, stfcamss_of_match);
>> +
>> +static int __maybe_unused stfcamss_runtime_suspend(struct device *dev)
>> +{
>> + struct stfcamss *stfcamss = dev_get_drvdata(dev);
>> +
>> + reset_control_assert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
>> + reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
>> + reset_control_assert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
>
> The reset bulk API (reset_control_bulk_assert() and
> reset_control_bulk_deassert() for below) would help.
OK, I will use the bulk API.
>
>> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
>> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
>
> Same for clocks, possibly.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int __maybe_unused stfcamss_runtime_resume(struct device *dev)
>> +{
>> + struct stfcamss *stfcamss = dev_get_drvdata(dev);
>> + int ret;
>> +
>> + ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISPCORE_2X].clk);
>> + if (ret)
>> + return ret;
>> +
>> + ret = clk_prepare_enable(stfcamss->sys_clk[STF_CLK_ISP_AXI].clk);
>> + if (ret)
>
> You need to disable the STF_CLK_ISPCORE_2X clock here.
>
OK, I will fix it.
>> + return ret;
>> +
>> + reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_N].rstc);
>> + reset_control_deassert(stfcamss->sys_rst[STF_RST_ISP_TOP_AXI].rstc);
>> + reset_control_deassert(stfcamss->sys_rst[STF_RST_AXIWR].rstc);
>
> These calls can fail too.
>
> Error handling would be easier with the reset and clk bulk APIs.
>
OK, I will modify it.
>> +
>> + return 0;
>> +}
>> +
>> +static const struct dev_pm_ops stfcamss_pm_ops = {
>> + SET_RUNTIME_PM_OPS(stfcamss_runtime_suspend,
>> + stfcamss_runtime_resume,
>> + NULL)
>> +};
>> +
>> +static struct platform_driver stfcamss_driver = {
>> + .probe = stfcamss_probe,
>> + .remove = stfcamss_remove,
>> + .driver = {
>> + .name = "starfive-camss",
>> + .pm = &stfcamss_pm_ops,
>> + .of_match_table = of_match_ptr(stfcamss_of_match),
>
> The driver depends on OF, so let's drop the of_match_ptr().
>
OK, will drop it.
>> + },
>> +};
>> +
>> +module_platform_driver(stfcamss_driver);
>> +
>> +MODULE_AUTHOR("StarFive Corporation");
>> +MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
>> +MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
>> new file mode 100644
>> index 000000000000..15c4f34b9054
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
>> @@ -0,0 +1,146 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * stf_camss.h
>> + *
>> + * Starfive Camera Subsystem driver
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#ifndef STF_CAMSS_H
>> +#define STF_CAMSS_H
>> +
>> +#include <linux/clk.h>
>> +#include <linux/delay.h>
>> +#include <linux/reset.h>
>> +#include <media/media-device.h>
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-async.h>
>> +#include <media/v4l2-device.h>
>> +
>> +#define STF_DVP_NAME "stf_dvp"
>> +#define STF_CSI_NAME "cdns_csi2rx"
>> +#define STF_ISP_NAME "stf_isp"
>> +#define STF_VIN_NAME "stf_vin"
>> +
>> +#define STF_PAD_SINK 0
>> +#define STF_PAD_SRC 1
>> +#define STF_PADS_NUM 2
>> +
>> +enum stf_port_num {
>> + STF_PORT_DVP = 0,
>> + STF_PORT_CSI2RX
>> +};
>> +
>> +enum stf_clk {
>> + STF_CLK_APB_FUNC = 0,
>> + STF_CLK_WRAPPER_CLK_C,
>> + STF_CLK_DVP_INV,
>> + STF_CLK_AXIWR,
>> + STF_CLK_MIPI_RX0_PXL,
>> + STF_CLK_ISPCORE_2X,
>> + STF_CLK_ISP_AXI,
>> + STF_CLK_NUM
>> +};
>> +
>> +enum stf_rst {
>> + STF_RST_WRAPPER_P = 0,
>> + STF_RST_WRAPPER_C,
>> + STF_RST_AXIRD,
>> + STF_RST_AXIWR,
>> + STF_RST_ISP_TOP_N,
>> + STF_RST_ISP_TOP_AXI,
>> + STF_RST_NUM
>> +};
>> +
>> +enum stf_irq {
>> + STF_IRQ_VINWR = 0,
>> + STF_IRQ_ISP,
>> + STF_IRQ_ISPCSIL,
>> + STF_IRQ_NUM
>> +};
>> +
>> +struct stfcamss {
>> + struct v4l2_device v4l2_dev;
>> + struct media_device media_dev;
>> + struct media_pipeline pipe;
>> + struct device *dev;
>> + struct v4l2_async_notifier notifier;
>> + void __iomem *syscon_base;
>> + void __iomem *isp_base;
>> + int irq[STF_IRQ_NUM];
>> + struct clk_bulk_data sys_clk[STF_CLK_NUM];
>> + int nclks;
>> + struct reset_control_bulk_data sys_rst[STF_RST_NUM];
>> + int nrsts;
>> +};
>> +
>> +struct stfcamss_async_subdev {
>> + struct v4l2_async_subdev asd; /* must be first */
>> + enum stf_port_num port;
>> +};
>> +
>> +static inline u32 stf_isp_reg_read(struct stfcamss *stfcamss, u32 reg)
>> +{
>> + return ioread32(stfcamss->isp_base + reg);
>> +}
>> +
>> +static inline void stf_isp_reg_write(struct stfcamss *stfcamss,
>> + u32 reg, u32 val)
>> +{
>> + iowrite32(val, stfcamss->isp_base + reg);
>> +}
>> +
>> +static inline void stf_isp_reg_write_delay(struct stfcamss *stfcamss,
>> + u32 reg, u32 val, u32 delay)
>> +{
>> + iowrite32(val, stfcamss->isp_base + reg);
>> + usleep_range(1000 * delay, 1000 * delay + 100);
>> +}
>> +
>> +static inline void stf_isp_reg_set_bit(struct stfcamss *stfcamss,
>> + u32 reg, u32 mask, u32 val)
>> +{
>> + u32 value;
>> +
>> + value = ioread32(stfcamss->isp_base + reg) & ~mask;
>> + val &= mask;
>> + val |= value;
>> + iowrite32(val, stfcamss->isp_base + reg);
>> +}
>> +
>> +static inline void stf_isp_reg_set(struct stfcamss *stfcamss, u32 reg, u32 mask)
>> +{
>> + iowrite32(ioread32(stfcamss->isp_base + reg) | mask,
>> + stfcamss->isp_base + reg);
>> +}
>> +
>> +static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32 reg)
>> +{
>> + return ioread32(stfcamss->syscon_base + reg);
>> +}
>> +
>> +static inline void stf_syscon_reg_write(struct stfcamss *stfcamss,
>> + u32 reg, u32 val)
>> +{
>> + iowrite32(val, stfcamss->syscon_base + reg);
>> +}
>> +
>> +static inline void stf_syscon_reg_set_bit(struct stfcamss *stfcamss,
>> + u32 reg, u32 bit_mask)
>> +{
>> + u32 value;
>> +
>> + value = ioread32(stfcamss->syscon_base + reg);
>> + iowrite32(value | bit_mask, stfcamss->syscon_base + reg);
>> +}
>> +
>> +static inline void stf_syscon_reg_clear_bit(struct stfcamss *stfcamss,
>> + u32 reg, u32 bit_mask)
>> +{
>> + u32 value;
>> +
>> + value = ioread32(stfcamss->syscon_base + reg);
>> + iowrite32(value & ~bit_mask, stfcamss->syscon_base + reg);
>> +}
>> +#endif /* STF_CAMSS_H */
>
--
Regards,
Jack Zhu
Hi Hans,
Thank you for your comments.
On 2023/7/27 16:49, Hans Verkuil wrote:
> On 19/06/2023 13:28, Jack Zhu wrote:
>> Add video driver for StarFive Camera Subsystem.
>>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../media/platform/starfive/camss/Makefile | 4 +-
>> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
>> .../media/platform/starfive/camss/stf_video.h | 92 +++
>> 3 files changed, 819 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
>>
>> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> index d56ddd078a71..eb457917a914 100644
>> --- a/drivers/media/platform/starfive/camss/Makefile
>> +++ b/drivers/media/platform/starfive/camss/Makefile
>> @@ -3,6 +3,8 @@
>> # Makefile for StarFive Camera Subsystem driver
>> #
>>
>> -starfive-camss-objs += stf_camss.o
>> +starfive-camss-objs += \
>> + stf_camss.o \
>> + stf_video.o
>>
>> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
>> new file mode 100644
>> index 000000000000..2e6472fe51c6
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_video.c
>> @@ -0,0 +1,724 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_video.c
>> + *
>> + * StarFive Camera Subsystem - V4L2 device node
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#include <linux/pm_runtime.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-mc.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "stf_camss.h"
>> +#include "stf_video.h"
>> +
>> +static const struct stfcamss_format_info formats_pix_wr[] = {
>> + {
>> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SRGGB10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SGRBG10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SGBRG10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SBGGR10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> +};
>> +
>> +static const struct stfcamss_format_info formats_pix_isp[] = {
>> + {
>> + .code = MEDIA_BUS_FMT_Y12_1X12,
>> + .pixelformat = V4L2_PIX_FMT_NV12,
>> + .planes = 2,
>> + .vsub = { 1, 2 },
>> + .bpp = 8,
>> + },
>> +};
>> +
>> +/* -----------------------------------------------------------------------------
>> + * Helper functions
>> + */
>> +
>> +static int video_find_format(u32 code, u32 pixelformat,
>> + struct stfcamss_video *video)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < video->nformats; ++i) {
>> + if (video->formats[i].code == code &&
>> + video->formats[i].pixelformat == pixelformat)
>> + return i;
>> + }
>> +
>> + for (i = 0; i < video->nformats; ++i)
>> + if (video->formats[i].code == code)
>> + return i;
>> +
>> + for (i = 0; i < video->nformats; ++i)
>> + if (video->formats[i].pixelformat == pixelformat)
>> + return i;
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
>> +{
>> + struct v4l2_pix_format *pix;
>> + const struct stfcamss_format_info *fi;
>> + u32 width, height;
>> + u32 bpl;
>> + unsigned int i;
>> +
>> + pix = &f->fmt.pix;
>> +
>> + for (i = 0; i < video->nformats; i++)
>> + if (pix->pixelformat == video->formats[i].pixelformat)
>> + break;
>> +
>> + if (i == video->nformats)
>> + i = 0; /* default format */
>> +
>> + fi = &video->formats[i];
>> + width = pix->width;
>> + height = pix->height;
>> +
>> + memset(pix, 0, sizeof(*pix));
>> +
>> + pix->pixelformat = fi->pixelformat;
>> + pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
>> + STFCAMSS_FRAME_MAX_WIDTH);
>> + pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
>> + STFCAMSS_FRAME_MAX_HEIGHT);
>> + bpl = pix->width * fi->bpp / 8;
>> + bpl = ALIGN(bpl, video->bpl_alignment);
>> + pix->bytesperline = bpl;
>> +
>> + for (i = 0; i < fi->planes; ++i)
>> + pix->sizeimage += bpl * pix->height / fi->vsub[i];
>> +
>> + pix->field = V4L2_FIELD_NONE;
>> + pix->colorspace = V4L2_COLORSPACE_SRGB;
>> + pix->flags = 0;
>> + pix->ycbcr_enc =
>> + V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
>> + pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
>> + pix->colorspace,
>> + pix->ycbcr_enc);
>> + pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
>> +
>> + return 0;
>> +}
>> +
>> +static int stf_video_init_format(struct stfcamss_video *video)
>> +{
>> + int ret;
>> + struct v4l2_format format = {
>> + .type = video->type,
>> + .fmt.pix = {
>> + .width = 1920,
>> + .height = 1080,
>> + .pixelformat = V4L2_PIX_FMT_RGB565,
>> + },
>> + };
>> +
>> + ret = __video_try_fmt(video, &format);
>> +
>> + if (ret < 0)
>> + return ret;
>> +
>> + video->active_fmt = format;
>> +
>> + return 0;
>> +}
>> +
>> +/* -----------------------------------------------------------------------------
>> + * Video queue operations
>> + */
>> +
>> +static int video_queue_setup(struct vb2_queue *q,
>> + unsigned int *num_buffers,
>> + unsigned int *num_planes,
>> + unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + struct stfcamss_video *video = vb2_get_drv_priv(q);
>> + const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
>> +
>> + if (*num_planes) {
>> + if (*num_planes != 1)
>> + return -EINVAL;
>> +
>> + if (sizes[0] < format->sizeimage)
>> + return -EINVAL;
>> + }
>> +
>> + *num_planes = 1;
>> + sizes[0] = format->sizeimage;
>> + if (!sizes[0])
>> + dev_err(video->stfcamss->dev,
>> + "%s: error size is zero!!!\n", __func__);
>> +
>> + dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
>> + *num_planes, sizes[0]);
>> +
>> + return 0;
>> +}
>> +
>> +static int video_buf_init(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + struct stfcamss_buffer *buffer =
>> + container_of(vbuf, struct stfcamss_buffer, vb);
>> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
>> + dma_addr_t *paddr;
>> +
>> + paddr = vb2_plane_cookie(vb, 0);
>> + buffer->addr[0] = *paddr;
>> +
>> + if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV61)
>> + buffer->addr[1] =
>> + buffer->addr[0] + fmt->bytesperline * fmt->height;
>> +
>> + return 0;
>> +}
>> +
>> +static int video_buf_prepare(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
>> +
>> + if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
>> + dev_err(video->stfcamss->dev,
>> + "sizeimage = %d, plane size = %d\n",
>> + fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
>> + return -EINVAL;
>> + }
>> + vb2_set_plane_payload(vb, 0, fmt->sizeimage);
>> +
>> + vbuf->field = V4L2_FIELD_NONE;
>> +
>> + return 0;
>> +}
>> +
>> +static void video_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + struct stfcamss_buffer *buffer =
>> + container_of(vbuf, struct stfcamss_buffer, vb);
>> +
>> + video->ops->queue_buffer(video, buffer);
>> +}
>> +
>> +/*
>> + * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
>> + * @mbus: v4l2_mbus_framefmt format (input)
>> + * @pix: v4l2_pix_format_mplane format (output)
>> + * @f: a pointer to formats array element to be used for the conversion
>> + * @alignment: bytesperline alignment value
>> + *
>> + * Fill the output pix structure with information from the input mbus format.
>> + *
>> + * Return 0 on success or a negative error code otherwise
>> + */
>> +static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
>> + struct v4l2_pix_format *pix,
>> + const struct stfcamss_format_info *f,
>> + unsigned int alignment)
>> +{
>> + u32 bytesperline;
>> + unsigned int i;
>> +
>> + memset(pix, 0, sizeof(*pix));
>> + v4l2_fill_pix_format(pix, mbus);
>> + pix->pixelformat = f->pixelformat;
>> + bytesperline = pix->width * f->bpp / 8;
>> + bytesperline = ALIGN(bytesperline, alignment);
>> + pix->bytesperline = bytesperline;
>> +
>> + for (i = 0; i < f->planes; ++i)
>> + pix->sizeimage += bytesperline * pix->height / f->vsub[i];
>> +
>> + return 0;
>> +}
>> +
>> +static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
>> + u32 *pad)
>> +{
>> + struct media_pad *remote;
>> +
>> + remote = media_pad_remote_pad_first(&video->pad);
>> +
>> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
>> + return NULL;
>> +
>> + if (pad)
>> + *pad = remote->index;
>> +
>> + return media_entity_to_v4l2_subdev(remote->entity);
>> +}
>> +
>> +static int video_get_subdev_format(struct stfcamss_video *video,
>> + struct v4l2_format *format)
>> +{
>> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
>> + struct v4l2_subdev_format fmt;
>> + struct v4l2_subdev *subdev;
>> + u32 pixelformat;
>> + u32 pad;
>> + int ret;
>> +
>> + subdev = video_remote_subdev(video, &pad);
>> + if (!subdev)
>> + return -EPIPE;
>> +
>> + fmt.pad = pad;
>> + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>> +
>> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
>> + if (ret)
>> + return ret;
>> +
>> + pixelformat = pix->pixelformat;
>> + ret = video_find_format(fmt.format.code, pixelformat, video);
>> + if (ret < 0)
>> + return ret;
>> +
>> + format->type = video->type;
>> +
>> + return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
>> + &video->formats[ret], video->bpl_alignment);
>> +}
>> +
>> +static int video_check_format(struct stfcamss_video *video)
>> +{
>> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
>> + struct v4l2_format format;
>> + struct v4l2_pix_format *sd_pix = &format.fmt.pix;
>> + int ret;
>> +
>> + sd_pix->pixelformat = pix->pixelformat;
>> + ret = video_get_subdev_format(video, &format);
>> + if (ret < 0)
>> + return ret;
>> +
>> + if (pix->pixelformat != sd_pix->pixelformat ||
>> + pix->height != sd_pix->height ||
>> + pix->width != sd_pix->width ||
>> + pix->field != format.fmt.pix.field) {
>> + dev_err(video->stfcamss->dev,
>> + "not match:\n"
>> + "pixelformat: 0x%x <-> 0x%x\n"
>> + "height: %d <-> %d\n"
>> + "field: %d <-> %d\n",
>> + pix->pixelformat, sd_pix->pixelformat,
>> + pix->height, sd_pix->height,
>> + pix->field, format.fmt.pix.field);
>> + return -EPIPE;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int video_start_streaming(struct vb2_queue *q, unsigned int count)
>> +{
>> + struct stfcamss_video *video = vb2_get_drv_priv(q);
>> + struct video_device *vdev = &video->vdev;
>> + struct media_entity *entity;
>> + struct media_pad *pad;
>> + struct v4l2_subdev *subdev;
>> + int ret;
>> +
>> + ret = video_device_pipeline_start(vdev, &video->stfcamss->pipe);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev,
>> + "Failed to media_pipeline_start: %d\n", ret);
>
> This doesn't call flush_buffers to return the buffers back to the QUEUED state.
>
> You do that correctly elsewhere in this function, but you missed this case.
>
OK, I will fix it.
>> + return ret;
>> + }
>> +
>> + ret = video_check_format(video);
>> + if (ret < 0)
>> + goto error;
>> +
>> + ret = pm_runtime_resume_and_get(video->stfcamss->dev);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev, "power up failed %d\n", ret);
>> + goto error;
>> + }
>> +
>> + entity = &vdev->entity;
>> + while (1) {
>> + pad = &entity->pads[0];
>> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
>> + break;
>> +
>> + pad = media_pad_remote_pad_first(pad);
>> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
>> + break;
>> +
>> + entity = pad->entity;
>> + subdev = media_entity_to_v4l2_subdev(entity);
>> +
>> + ret = v4l2_subdev_call(subdev, video, s_stream, 1);
>> + if (ret < 0 && ret != -ENOIOCTLCMD)
>> + goto err_pm_put;
>> + }
>> + return 0;
>> +
>> +err_pm_put:
>> + pm_runtime_put(video->stfcamss->dev);
>> +error:
>> + video_device_pipeline_stop(vdev);
>> + video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
>> + return ret;
>> +}
>> +
>> +static void video_stop_streaming(struct vb2_queue *q)
>> +{
>> + struct stfcamss_video *video = vb2_get_drv_priv(q);
>> + struct video_device *vdev = &video->vdev;
>> + struct media_entity *entity;
>> + struct media_pad *pad;
>> + struct v4l2_subdev *subdev;
>> + int ret;
>> +
>> + entity = &vdev->entity;
>> + while (1) {
>> + pad = &entity->pads[0];
>> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
>> + break;
>> +
>> + pad = media_pad_remote_pad_first(pad);
>> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
>> + break;
>> +
>> + entity = pad->entity;
>> + subdev = media_entity_to_v4l2_subdev(entity);
>> +
>> + v4l2_subdev_call(subdev, video, s_stream, 0);
>> + }
>> +
>> + ret = pm_runtime_put(video->stfcamss->dev);
>> + if (ret < 0)
>> + dev_err(video->stfcamss->dev, "power down failed:%d\n", ret);
>> +
>> + video_device_pipeline_stop(vdev);
>> + video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
>> +}
>> +
>> +static const struct vb2_ops stf_video_vb2_q_ops = {
>> + .queue_setup = video_queue_setup,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> + .buf_init = video_buf_init,
>> + .buf_prepare = video_buf_prepare,
>> + .buf_queue = video_buf_queue,
>> + .start_streaming = video_start_streaming,
>> + .stop_streaming = video_stop_streaming,
>> +};
>> +
>> +/* -----------------------------------------------------------------------------
>> + * V4L2 ioctls
>> + */
>> +
>> +static int video_querycap(struct file *file, void *fh,
>> + struct v4l2_capability *cap)
>> +{
>> + strscpy(cap->driver, "stf camss", sizeof(cap->driver));
>> + strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
>> +
>> + return 0;
>> +}
>> +
>> +static int video_get_pfmt_by_index(struct stfcamss_video *video, int ndx)
>> +{
>> + int i, j, k;
>> +
>> + /* find index "i" of "k"th unique pixelformat in formats array */
>> + k = -1;
>> + for (i = 0; i < video->nformats; i++) {
>> + for (j = 0; j < i; j++) {
>> + if (video->formats[i].pixelformat ==
>> + video->formats[j].pixelformat)
>> + break;
>> + }
>> +
>> + if (j == i)
>> + k++;
>> +
>> + if (k == ndx)
>> + return i;
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int video_get_pfmt_by_mcode(struct stfcamss_video *video, u32 mcode)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < video->nformats; i++) {
>> + if (video->formats[i].code == mcode)
>> + return i;
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int video_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
>> +{
>> + struct stfcamss_video *video = video_drvdata(file);
>> + int i;
>> +
>> + if (f->type != video->type)
>> + return -EINVAL;
>> + if (f->index >= video->nformats)
>> + return -EINVAL;
>> +
>> + if (f->mbus_code) {
>> + /* Each entry in formats[] table has unique mbus_code */
>> + if (f->index > 0)
>> + return -EINVAL;
>> +
>> + i = video_get_pfmt_by_mcode(video, f->mbus_code);
>> + } else {
>> + i = video_get_pfmt_by_index(video, f->index);
>> + }
>> +
>> + if (i < 0)
>> + return -EINVAL;
>> +
>> + f->pixelformat = video->formats[i].pixelformat;
>> +
>> + return 0;
>> +}
>> +
>> +static int video_enum_framesizes(struct file *file, void *fh,
>> + struct v4l2_frmsizeenum *fsize)
>> +{
>> + struct stfcamss_video *video = video_drvdata(file);
>> + int i;
>> +
>> + if (fsize->index)
>> + return -EINVAL;
>> +
>> + for (i = 0; i < video->nformats; i++) {
>> + if (video->formats[i].pixelformat == fsize->pixel_format)
>> + break;
>> + }
>> +
>> + if (i == video->nformats)
>> + return -EINVAL;
>> +
>> + fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
>> + fsize->stepwise.min_width = STFCAMSS_FRAME_MIN_WIDTH;
>> + fsize->stepwise.max_width = STFCAMSS_FRAME_MAX_WIDTH;
>> + fsize->stepwise.min_height = STFCAMSS_FRAME_MIN_HEIGHT;
>> + fsize->stepwise.max_height = STFCAMSS_FRAME_MAX_HEIGHT;
>> + fsize->stepwise.step_width = 1;
>> + fsize->stepwise.step_height = 1;
>> +
>> + return 0;
>> +}
>> +
>> +static int video_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
>> +{
>> + struct stfcamss_video *video = video_drvdata(file);
>> +
>> + *f = video->active_fmt;
>> +
>> + return 0;
>> +}
>> +
>> +static int video_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
>> +{
>> + struct stfcamss_video *video = video_drvdata(file);
>> + int ret;
>> +
>> + if (vb2_is_busy(&video->vb2_q))
>> + return -EBUSY;
>> +
>> + ret = __video_try_fmt(video, f);
>> + if (ret < 0)
>> + return ret;
>> +
>> + video->active_fmt = *f;
>> +
>> + return 0;
>> +}
>> +
>> +static int video_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
>> +{
>> + struct stfcamss_video *video = video_drvdata(file);
>> +
>> + return __video_try_fmt(video, f);
>> +}
>> +
>> +static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
>> + .vidioc_querycap = video_querycap,
>> + .vidioc_enum_fmt_vid_cap = video_enum_fmt,
>> + .vidioc_enum_fmt_vid_out = video_enum_fmt,
>> + .vidioc_enum_framesizes = video_enum_framesizes,
>> + .vidioc_g_fmt_vid_cap = video_g_fmt,
>> + .vidioc_s_fmt_vid_cap = video_s_fmt,
>> + .vidioc_try_fmt_vid_cap = video_try_fmt,
>> + .vidioc_g_fmt_vid_out = video_g_fmt,
>> + .vidioc_s_fmt_vid_out = video_s_fmt,
>> + .vidioc_try_fmt_vid_out = video_try_fmt,
>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> + .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>> +};
>> +
>> +/* -----------------------------------------------------------------------------
>> + * V4L2 file operations
>> + */
>> +
>> +static const struct v4l2_file_operations stf_vid_fops = {
>> + .owner = THIS_MODULE,
>> + .unlocked_ioctl = video_ioctl2,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .poll = vb2_fop_poll,
>> + .mmap = vb2_fop_mmap,
>> + .read = vb2_fop_read,
>> +};
>> +
>> +/* -----------------------------------------------------------------------------
>> + * STFCAMSS video core
>> + */
>> +
>> +static void stf_video_release(struct video_device *vdev)
>> +{
>> + struct stfcamss_video *video = video_get_drvdata(vdev);
>> +
>> + media_entity_cleanup(&vdev->entity);
>> +
>> + mutex_destroy(&video->q_lock);
>> + mutex_destroy(&video->lock);
>> +}
>> +
>> +int stf_video_register(struct stfcamss_video *video,
>> + struct v4l2_device *v4l2_dev, const char *name)
>> +{
>> + struct video_device *vdev;
>> + struct vb2_queue *q;
>> + struct media_pad *pad = &video->pad;
>> + int ret;
>> +
>> + vdev = &video->vdev;
>> +
>> + mutex_init(&video->q_lock);
>> +
>> + q = &video->vb2_q;
>> + q->drv_priv = video;
>> + q->mem_ops = &vb2_dma_contig_memops;
>> + q->ops = &stf_video_vb2_q_ops;
>> + q->type = video->type;
>> + q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
>
> VB2_READ support does not generally make sense for uncompressed video since
> read() always requires a memcpy, and that makes it very inefficient.
>
> It doesn't hurt though, so it is up to you whether or not you want this.
>
Yes, we would like to retain this feature to meet some possible special needs.
>> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + q->buf_struct_size = sizeof(struct stfcamss_buffer);
>> + q->dev = video->stfcamss->dev;
>> + q->lock = &video->q_lock;
>> + q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
>> + ret = vb2_queue_init(q);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev,
>> + "Failed to init vb2 queue: %d\n", ret);
>> + goto err_vb2_init;
>> + }
>> +
>> + pad->flags = MEDIA_PAD_FL_SINK;
>> + ret = media_entity_pads_init(&vdev->entity, 1, pad);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev,
>> + "Failed to init video entity: %d\n", ret);
>> + goto err_vb2_init;
>> + }
>> +
>> + mutex_init(&video->lock);
>> +
>> + if (video->id == STF_V_LINE_WR) {
>> + video->formats = formats_pix_wr;
>> + video->nformats = ARRAY_SIZE(formats_pix_wr);
>> + video->bpl_alignment = 8;
>> + } else {
>> + video->formats = formats_pix_isp;
>> + video->nformats = ARRAY_SIZE(formats_pix_isp);
>> + video->bpl_alignment = 1;
>> + }
>> +
>> + ret = stf_video_init_format(video);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev,
>> + "Failed to init format: %d\n", ret);
>> + goto err_vid_init_format;
>> + }
>> +
>> + vdev->fops = &stf_vid_fops;
>> + vdev->ioctl_ops = &stf_vid_ioctl_ops;
>> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
>> + vdev->vfl_dir = VFL_DIR_RX;
>> + vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
>> + vdev->release = stf_video_release;
>> + vdev->v4l2_dev = v4l2_dev;
>> + vdev->queue = &video->vb2_q;
>> + vdev->lock = &video->lock;
>> + strscpy(vdev->name, name, sizeof(vdev->name));
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
>> + if (ret < 0) {
>> + dev_err(video->stfcamss->dev,
>> + "Failed to register video device: %d\n", ret);
>> + goto err_vid_reg;
>> + }
>> +
>> + video_set_drvdata(vdev, video);
>> + return 0;
>> +
>> +err_vid_reg:
>> +err_vid_init_format:
>> + media_entity_cleanup(&vdev->entity);
>> + mutex_destroy(&video->lock);
>> +err_vb2_init:
>> + mutex_destroy(&video->q_lock);
>> + return ret;
>> +}
>> +
>> +void stf_video_unregister(struct stfcamss_video *video)
>> +{
>> + vb2_video_unregister_device(&video->vdev);
>> +}
>> diff --git a/drivers/media/platform/starfive/camss/stf_video.h b/drivers/media/platform/starfive/camss/stf_video.h
>> new file mode 100644
>> index 000000000000..60323c23a40c
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_video.h
>> @@ -0,0 +1,92 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * stf_video.h
>> + *
>> + * StarFive Camera Subsystem - V4L2 device node
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#ifndef STF_VIDEO_H
>> +#define STF_VIDEO_H
>> +
>> +#include <linux/list.h>
>> +#include <linux/mutex.h>
>> +#include <linux/videodev2.h>
>> +#include <media/v4l2-dev.h>
>> +#include <media/v4l2-fh.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#define STFCAMSS_FRAME_MIN_WIDTH 64
>> +#define STFCAMSS_FRAME_MAX_WIDTH 1920
>> +#define STFCAMSS_FRAME_MIN_HEIGHT 64
>> +#define STFCAMSS_FRAME_MAX_HEIGHT 1080
>> +#define STFCAMSS_FRAME_WIDTH_ALIGN_8 8
>> +#define STFCAMSS_FRAME_WIDTH_ALIGN_128 128
>> +#define STFCAMSS_MIN_BUFFERS 2
>> +
>> +#define STFCAMSS_MAX_ENTITY_NAME_LEN 27
>> +
>> +enum stf_v_line_id {
>> + STF_V_LINE_WR = 0,
>> + STF_V_LINE_ISP,
>> + STF_V_LINE_MAX,
>> +};
>> +
>> +struct stfcamss_buffer {
>> + struct vb2_v4l2_buffer vb;
>> + dma_addr_t addr[3];
>> + struct list_head queue;
>> +};
>> +
>> +struct fract {
>> + u8 numerator;
>> + u8 denominator;
>> +};
>> +
>> +/*
>> + * struct stfcamss_format_info - ISP media bus format information
>> + * @code: V4L2 media bus format code
>> + * @pixelformat: V4L2 pixel format FCC identifier
>> + * @planes: Number of planes
>> + * @vsub: Vertical subsampling (for each plane)
>> + * @bpp: Bits per pixel when stored in memory (for each plane)
>> + */
>> +struct stfcamss_format_info {
>> + u32 code;
>> + u32 pixelformat;
>> + u8 planes;
>> + u8 vsub[3];
>> + u8 bpp;
>> +};
>> +
>> +struct stfcamss_video {
>> + struct stfcamss *stfcamss;
>> + u8 id;
>> + struct vb2_queue vb2_q;
>> + struct video_device vdev;
>> + struct media_pad pad;
>> + struct v4l2_format active_fmt;
>> + enum v4l2_buf_type type;
>> + const struct stfcamss_video_ops *ops;
>> + struct mutex lock; /* serialize device access */
>> + struct mutex q_lock; /* protects the queue */
>> + unsigned int bpl_alignment;
>> + const struct stfcamss_format_info *formats;
>> + unsigned int nformats;
>> +};
>> +
>> +struct stfcamss_video_ops {
>> + int (*queue_buffer)(struct stfcamss_video *vid,
>> + struct stfcamss_buffer *buf);
>> + int (*flush_buffers)(struct stfcamss_video *vid,
>> + enum vb2_buffer_state state);
>> +};
>> +
>> +int stf_video_register(struct stfcamss_video *video,
>> + struct v4l2_device *v4l2_dev, const char *name);
>> +
>> +void stf_video_unregister(struct stfcamss_video *video);
>> +
>> +#endif /* STF_VIDEO_H */
>
> Regards,
>
> Hans
--
Regards,
Jack Zhu
Hi Jack,
On Tue, Aug 01, 2023 at 11:24:22AM +0800, Jack Zhu wrote:
> On 2023/7/27 19:33, Laurent Pinchart wrote:
> > On Mon, Jun 19, 2023 at 07:28:35PM +0800, Jack Zhu wrote:
> >> Add basic platform driver for StarFive Camera Subsystem.
> >>
> >> Reviewed-by: Bryan O'Donoghue <[email protected]>
> >> Signed-off-by: Jack Zhu <[email protected]>
> >> ---
> >> MAINTAINERS | 1 +
> >> drivers/media/platform/Kconfig | 1 +
> >> drivers/media/platform/Makefile | 1 +
> >> drivers/media/platform/starfive/Kconfig | 5 +
> >> drivers/media/platform/starfive/Makefile | 2 +
> >> drivers/media/platform/starfive/camss/Kconfig | 16 +
> >> .../media/platform/starfive/camss/Makefile | 8 +
> >> .../media/platform/starfive/camss/stf_camss.c | 338 ++++++++++++++++++
> >> .../media/platform/starfive/camss/stf_camss.h | 146 ++++++++
> >> 9 files changed, 518 insertions(+)
> >> create mode 100644 drivers/media/platform/starfive/Kconfig
> >> create mode 100644 drivers/media/platform/starfive/Makefile
> >> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
> >> create mode 100644 drivers/media/platform/starfive/camss/Makefile
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
[snip]
> >> diff --git a/drivers/media/platform/starfive/camss/Kconfig b/drivers/media/platform/starfive/camss/Kconfig
> >> new file mode 100644
> >> index 000000000000..dafe1d24324b
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/Kconfig
> >> @@ -0,0 +1,16 @@
> >> +# SPDX-License-Identifier: GPL-2.0-only
> >> +config VIDEO_STARFIVE_CAMSS
> >> + tristate "Starfive Camera Subsystem driver"
> >> + depends on V4L_PLATFORM_DRIVERS
> >> + depends on VIDEO_DEV && OF
> >> + depends on HAS_DMA
> >
> > You need to depend on PM, otherwise the runtime PM operations will be
> > no-ops and the driver won't work as clocks won't be enabled.
>
> OK, I will add dependency.
By the way, if it makes it easier for you, you don't need to acknowledge
every single review comment. You can reply to comments you disagree
with, or comments that you find unclear. Anything that you agree with
and will address in the next version can be left unanswered in your
e-mail replies. It's entirely up to you.
> >> + select MEDIA_CONTROLLER
> >> + select VIDEO_V4L2_SUBDEV_API
> >> + select VIDEOBUF2_DMA_CONTIG
> >> + select V4L2_FWNODE
> >> + help
> >> + Enable this to support for the Starfive Camera subsystem
> >> + found on Starfive JH7110 SoC.
> >> +
> >> + To compile this driver as a module, choose M here: the
> >> + module will be called stf-camss.
[snip]
> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> >> new file mode 100644
> >> index 000000000000..dc2b5dba7bd4
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> >> @@ -0,0 +1,338 @@
[snip]
> >> +/*
> >> + * stfcamss_probe - Probe STFCAMSS platform device
> >> + * @pdev: Pointer to STFCAMSS platform device
> >> + *
> >> + * Return 0 on success or a negative error code on failure
> >> + */
> >> +static int stfcamss_probe(struct platform_device *pdev)
> >> +{
> >> + struct stfcamss *stfcamss;
> >> + struct device *dev = &pdev->dev;
> >> + int ret, num_subdevs;
> >> + unsigned int i;
> >> +
> >> + stfcamss = devm_kzalloc(dev, sizeof(*stfcamss), GFP_KERNEL);
> >> + if (!stfcamss)
> >> + return -ENOMEM;
> >> +
> >> + for (i = 0; i < ARRAY_SIZE(stfcamss->irq); ++i) {
> >> + stfcamss->irq[i] = platform_get_irq(pdev, i);
> >> + if (stfcamss->irq[i] < 0)
> >> + return dev_err_probe(&pdev->dev, stfcamss->irq[i],
> >> + "Failed to get irq%d", i);
> >> + }
> >> +
> >> + stfcamss->nclks = ARRAY_SIZE(stfcamss->sys_clk);
> >> + for (i = 0; i < stfcamss->nclks; ++i)
> >> + stfcamss->sys_clk[i].id = stfcamss_clocks[i];
> >> + ret = devm_clk_bulk_get(dev, stfcamss->nclks, stfcamss->sys_clk);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to get clk controls\n");
> >> + return ret;
> >> + }
> >> +
> >> + stfcamss->nrsts = ARRAY_SIZE(stfcamss->sys_rst);
> >> + for (i = 0; i < stfcamss->nrsts; ++i)
> >> + stfcamss->sys_rst[i].id = stfcamss_resets[i];
> >> + ret = devm_reset_control_bulk_get_shared(dev, stfcamss->nrsts,
> >> + stfcamss->sys_rst);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to get reset controls\n");
> >> + return ret;
> >> + }
> >> +
> >> + ret = stfcamss_get_mem_res(pdev, stfcamss);
> >> + if (ret) {
> >> + dev_err(dev, "Could not map registers\n");
> >> + return ret;
> >> + }
> >> +
> >> + stfcamss->dev = dev;
> >
> > Move this right after allocating stfcamss, and drop the pdev argument to
> > stfcamss_get_mem_res(). The platform device can be retrieved in the
> > function using to_platform_device().
>
> OK, I will modify.
>
> >> + platform_set_drvdata(pdev, stfcamss);
> >> +
> >> + v4l2_async_nf_init(&stfcamss->notifier);
> >> +
> >> + num_subdevs = stfcamss_of_parse_ports(stfcamss);
> >> + if (num_subdevs < 0) {
> >> + ret = -ENODEV;
> >
> > An error message would be useful, silent errors are hard to debug.
>
> OK, will add error printing information.
>
> >> + goto err_cleanup_notifier;
> >> + }
> >> +
> >> + stfcamss_mc_init(pdev, stfcamss);
> >> +
> >> + ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
> >> + if (ret < 0) {
> >> + dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
> >> + goto err_cleanup_notifier;
> >> + }
> >> +
> >> + ret = media_device_register(&stfcamss->media_dev);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to register media device: %d\n", ret);
> >> + goto err_unregister_device;
> >> + }
> >> +
> >> + pm_runtime_enable(dev);
> >
> > Would it be useful to enable autosuspend too, to avoid expensive
> > suspend/resume cycles when userspace wants to briefly stop capture and
> > restart it immediately ?
>
> It seems rare to use autosuspend in the Linux camera system.
It's a relatively recent practice, and is more common in sensor drivers
than ISP drivers, but I think it's a good practice nonetheless. It makes
stop-reconfigure-start cycles much faster.
> >> +
> >> + stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
> >> + ret = v4l2_async_nf_register(&stfcamss->v4l2_dev, &stfcamss->notifier);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to register async subdev nodes: %d\n",
> >> + ret);
> >> + goto err_unregister_media_dev;
> >
> > You need to disable runtime PM in this error path.
>
> OK, will fix it.
>
> >> + }
> >> +
> >> + return 0;
> >> +
> >> +err_unregister_media_dev:
> >> + media_device_unregister(&stfcamss->media_dev);
> >> +err_unregister_device:
> >> + v4l2_device_unregister(&stfcamss->v4l2_dev);
> >> +err_cleanup_notifier:
> >> + v4l2_async_nf_cleanup(&stfcamss->notifier);
> >> + return ret;
> >> +}
[snip]
--
Regards,
Laurent Pinchart
Hi Jack,
On Tue, Aug 01, 2023 at 02:23:07PM +0800, Jack Zhu wrote:
> On 2023/7/27 16:49, Hans Verkuil wrote:
> > On 19/06/2023 13:28, Jack Zhu wrote:
> >> Add video driver for StarFive Camera Subsystem.
> >>
> >> Signed-off-by: Jack Zhu <[email protected]>
> >> ---
> >> .../media/platform/starfive/camss/Makefile | 4 +-
> >> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
> >> .../media/platform/starfive/camss/stf_video.h | 92 +++
> >> 3 files changed, 819 insertions(+), 1 deletion(-)
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
[snip]
> >> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
> >> new file mode 100644
> >> index 000000000000..2e6472fe51c6
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_video.c
> >> @@ -0,0 +1,724 @@
[snip]
> >> +int stf_video_register(struct stfcamss_video *video,
> >> + struct v4l2_device *v4l2_dev, const char *name)
> >> +{
> >> + struct video_device *vdev;
> >> + struct vb2_queue *q;
> >> + struct media_pad *pad = &video->pad;
> >> + int ret;
> >> +
> >> + vdev = &video->vdev;
> >> +
> >> + mutex_init(&video->q_lock);
> >> +
> >> + q = &video->vb2_q;
> >> + q->drv_priv = video;
> >> + q->mem_ops = &vb2_dma_contig_memops;
> >> + q->ops = &stf_video_vb2_q_ops;
> >> + q->type = video->type;
> >> + q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
> >
> > VB2_READ support does not generally make sense for uncompressed video since
> > read() always requires a memcpy, and that makes it very inefficient.
> >
> > It doesn't hurt though, so it is up to you whether or not you want this.
>
> Yes, we would like to retain this feature to meet some possible special needs.
The issue with enabling READ support in drivers is that it encourages
applications to do the wrong thing. If you want to keep it, I'd like to
know what use cases you envision would strongly require it.
> >> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> >> + q->buf_struct_size = sizeof(struct stfcamss_buffer);
> >> + q->dev = video->stfcamss->dev;
> >> + q->lock = &video->q_lock;
> >> + q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
> >> + ret = vb2_queue_init(q);
> >> + if (ret < 0) {
> >> + dev_err(video->stfcamss->dev,
> >> + "Failed to init vb2 queue: %d\n", ret);
> >> + goto err_vb2_init;
> >> + }
> >> +
> >> + pad->flags = MEDIA_PAD_FL_SINK;
> >> + ret = media_entity_pads_init(&vdev->entity, 1, pad);
> >> + if (ret < 0) {
> >> + dev_err(video->stfcamss->dev,
> >> + "Failed to init video entity: %d\n", ret);
> >> + goto err_vb2_init;
> >> + }
> >> +
> >> + mutex_init(&video->lock);
> >> +
> >> + if (video->id == STF_V_LINE_WR) {
> >> + video->formats = formats_pix_wr;
> >> + video->nformats = ARRAY_SIZE(formats_pix_wr);
> >> + video->bpl_alignment = 8;
> >> + } else {
> >> + video->formats = formats_pix_isp;
> >> + video->nformats = ARRAY_SIZE(formats_pix_isp);
> >> + video->bpl_alignment = 1;
> >> + }
> >> +
> >> + ret = stf_video_init_format(video);
> >> + if (ret < 0) {
> >> + dev_err(video->stfcamss->dev,
> >> + "Failed to init format: %d\n", ret);
> >> + goto err_vid_init_format;
> >> + }
> >> +
> >> + vdev->fops = &stf_vid_fops;
> >> + vdev->ioctl_ops = &stf_vid_ioctl_ops;
> >> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> >> + vdev->vfl_dir = VFL_DIR_RX;
> >> + vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
> >> + vdev->release = stf_video_release;
> >> + vdev->v4l2_dev = v4l2_dev;
> >> + vdev->queue = &video->vb2_q;
> >> + vdev->lock = &video->lock;
> >> + strscpy(vdev->name, name, sizeof(vdev->name));
> >> +
> >> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
> >> + if (ret < 0) {
> >> + dev_err(video->stfcamss->dev,
> >> + "Failed to register video device: %d\n", ret);
> >> + goto err_vid_reg;
> >> + }
> >> +
> >> + video_set_drvdata(vdev, video);
> >> + return 0;
> >> +
> >> +err_vid_reg:
> >> +err_vid_init_format:
> >> + media_entity_cleanup(&vdev->entity);
> >> + mutex_destroy(&video->lock);
> >> +err_vb2_init:
> >> + mutex_destroy(&video->q_lock);
> >> + return ret;
> >> +}
[snip]
--
Regards,
Laurent Pinchart
On 2023/8/2 2:47, Laurent Pinchart wrote:
> Hi Jack,
>
> On Tue, Aug 01, 2023 at 02:23:07PM +0800, Jack Zhu wrote:
>> On 2023/7/27 16:49, Hans Verkuil wrote:
>> > On 19/06/2023 13:28, Jack Zhu wrote:
>> >> Add video driver for StarFive Camera Subsystem.
>> >>
>> >> Signed-off-by: Jack Zhu <[email protected]>
>> >> ---
>> >> .../media/platform/starfive/camss/Makefile | 4 +-
>> >> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
>> >> .../media/platform/starfive/camss/stf_video.h | 92 +++
>> >> 3 files changed, 819 insertions(+), 1 deletion(-)
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
>
> [snip]
>
>> >> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
>> >> new file mode 100644
>> >> index 000000000000..2e6472fe51c6
>> >> --- /dev/null
>> >> +++ b/drivers/media/platform/starfive/camss/stf_video.c
>> >> @@ -0,0 +1,724 @@
>
> [snip]
>
>> >> +int stf_video_register(struct stfcamss_video *video,
>> >> + struct v4l2_device *v4l2_dev, const char *name)
>> >> +{
>> >> + struct video_device *vdev;
>> >> + struct vb2_queue *q;
>> >> + struct media_pad *pad = &video->pad;
>> >> + int ret;
>> >> +
>> >> + vdev = &video->vdev;
>> >> +
>> >> + mutex_init(&video->q_lock);
>> >> +
>> >> + q = &video->vb2_q;
>> >> + q->drv_priv = video;
>> >> + q->mem_ops = &vb2_dma_contig_memops;
>> >> + q->ops = &stf_video_vb2_q_ops;
>> >> + q->type = video->type;
>> >> + q->io_modes = VB2_DMABUF | VB2_MMAP | VB2_READ;
>> >
>> > VB2_READ support does not generally make sense for uncompressed video since
>> > read() always requires a memcpy, and that makes it very inefficient.
>> >
>> > It doesn't hurt though, so it is up to you whether or not you want this.
>>
>> Yes, we would like to retain this feature to meet some possible special needs.
>
> The issue with enabling READ support in drivers is that it encourages
> applications to do the wrong thing. If you want to keep it, I'd like to
> know what use cases you envision would strongly require it.
>
OK, I'll drop it. Thank you!
>> >> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> >> + q->buf_struct_size = sizeof(struct stfcamss_buffer);
>> >> + q->dev = video->stfcamss->dev;
>> >> + q->lock = &video->q_lock;
>> >> + q->min_buffers_needed = STFCAMSS_MIN_BUFFERS;
>> >> + ret = vb2_queue_init(q);
>> >> + if (ret < 0) {
>> >> + dev_err(video->stfcamss->dev,
>> >> + "Failed to init vb2 queue: %d\n", ret);
>> >> + goto err_vb2_init;
>> >> + }
>> >> +
>> >> + pad->flags = MEDIA_PAD_FL_SINK;
>> >> + ret = media_entity_pads_init(&vdev->entity, 1, pad);
>> >> + if (ret < 0) {
>> >> + dev_err(video->stfcamss->dev,
>> >> + "Failed to init video entity: %d\n", ret);
>> >> + goto err_vb2_init;
>> >> + }
>> >> +
>> >> + mutex_init(&video->lock);
>> >> +
>> >> + if (video->id == STF_V_LINE_WR) {
>> >> + video->formats = formats_pix_wr;
>> >> + video->nformats = ARRAY_SIZE(formats_pix_wr);
>> >> + video->bpl_alignment = 8;
>> >> + } else {
>> >> + video->formats = formats_pix_isp;
>> >> + video->nformats = ARRAY_SIZE(formats_pix_isp);
>> >> + video->bpl_alignment = 1;
>> >> + }
>> >> +
>> >> + ret = stf_video_init_format(video);
>> >> + if (ret < 0) {
>> >> + dev_err(video->stfcamss->dev,
>> >> + "Failed to init format: %d\n", ret);
>> >> + goto err_vid_init_format;
>> >> + }
>> >> +
>> >> + vdev->fops = &stf_vid_fops;
>> >> + vdev->ioctl_ops = &stf_vid_ioctl_ops;
>> >> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
>> >> + vdev->vfl_dir = VFL_DIR_RX;
>> >> + vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_READWRITE;
>> >> + vdev->release = stf_video_release;
>> >> + vdev->v4l2_dev = v4l2_dev;
>> >> + vdev->queue = &video->vb2_q;
>> >> + vdev->lock = &video->lock;
>> >> + strscpy(vdev->name, name, sizeof(vdev->name));
>> >> +
>> >> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, video->id);
>> >> + if (ret < 0) {
>> >> + dev_err(video->stfcamss->dev,
>> >> + "Failed to register video device: %d\n", ret);
>> >> + goto err_vid_reg;
>> >> + }
>> >> +
>> >> + video_set_drvdata(vdev, video);
>> >> + return 0;
>> >> +
>> >> +err_vid_reg:
>> >> +err_vid_init_format:
>> >> + media_entity_cleanup(&vdev->entity);
>> >> + mutex_destroy(&video->lock);
>> >> +err_vb2_init:
>> >> + mutex_destroy(&video->q_lock);
>> >> + return ret;
>> >> +}
>
> [snip]
>
On 2023/8/2 2:45, Laurent Pinchart wrote:
> Hi Jack,
>
> On Tue, Aug 01, 2023 at 11:24:22AM +0800, Jack Zhu wrote:
>> On 2023/7/27 19:33, Laurent Pinchart wrote:
>> > On Mon, Jun 19, 2023 at 07:28:35PM +0800, Jack Zhu wrote:
>> >> Add basic platform driver for StarFive Camera Subsystem.
>> >>
>> >> Reviewed-by: Bryan O'Donoghue <[email protected]>
>> >> Signed-off-by: Jack Zhu <[email protected]>
>> >> ---
>> >> MAINTAINERS | 1 +
>> >> drivers/media/platform/Kconfig | 1 +
>> >> drivers/media/platform/Makefile | 1 +
>> >> drivers/media/platform/starfive/Kconfig | 5 +
>> >> drivers/media/platform/starfive/Makefile | 2 +
>> >> drivers/media/platform/starfive/camss/Kconfig | 16 +
>> >> .../media/platform/starfive/camss/Makefile | 8 +
>> >> .../media/platform/starfive/camss/stf_camss.c | 338 ++++++++++++++++++
>> >> .../media/platform/starfive/camss/stf_camss.h | 146 ++++++++
>> >> 9 files changed, 518 insertions(+)
>> >> create mode 100644 drivers/media/platform/starfive/Kconfig
>> >> create mode 100644 drivers/media/platform/starfive/Makefile
>> >> create mode 100644 drivers/media/platform/starfive/camss/Kconfig
>> >> create mode 100644 drivers/media/platform/starfive/camss/Makefile
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.c
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_camss.h
>
> [snip]
>
>> >> diff --git a/drivers/media/platform/starfive/camss/Kconfig b/drivers/media/platform/starfive/camss/Kconfig
>> >> new file mode 100644
>> >> index 000000000000..dafe1d24324b
>> >> --- /dev/null
>> >> +++ b/drivers/media/platform/starfive/camss/Kconfig
>> >> @@ -0,0 +1,16 @@
>> >> +# SPDX-License-Identifier: GPL-2.0-only
>> >> +config VIDEO_STARFIVE_CAMSS
>> >> + tristate "Starfive Camera Subsystem driver"
>> >> + depends on V4L_PLATFORM_DRIVERS
>> >> + depends on VIDEO_DEV && OF
>> >> + depends on HAS_DMA
>> >
>> > You need to depend on PM, otherwise the runtime PM operations will be
>> > no-ops and the driver won't work as clocks won't be enabled.
>>
>> OK, I will add dependency.
>
> By the way, if it makes it easier for you, you don't need to acknowledge
> every single review comment. You can reply to comments you disagree
> with, or comments that you find unclear. Anything that you agree with
> and will address in the next version can be left unanswered in your
> e-mail replies. It's entirely up to you.
>
Hi Laurent,
Your suggestion is very useful for me. Thanks!
>> >> + select MEDIA_CONTROLLER
>> >> + select VIDEO_V4L2_SUBDEV_API
>> >> + select VIDEOBUF2_DMA_CONTIG
>> >> + select V4L2_FWNODE
>> >> + help
>> >> + Enable this to support for the Starfive Camera subsystem
>> >> + found on Starfive JH7110 SoC.
>> >> +
>> >> + To compile this driver as a module, choose M here: the
>> >> + module will be called stf-camss.
>
> [snip]
>
>> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> new file mode 100644
>> >> index 000000000000..dc2b5dba7bd4
>> >> --- /dev/null
>> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> @@ -0,0 +1,338 @@
>
> [snip]
>
>> >> +/*
>> >> + * stfcamss_probe - Probe STFCAMSS platform device
>> >> + * @pdev: Pointer to STFCAMSS platform device
>> >> + *
>> >> + * Return 0 on success or a negative error code on failure
>> >> + */
>> >> +static int stfcamss_probe(struct platform_device *pdev)
>> >> +{
>> >> + struct stfcamss *stfcamss;
>> >> + struct device *dev = &pdev->dev;
>> >> + int ret, num_subdevs;
>> >> + unsigned int i;
>> >> +
>> >> + stfcamss = devm_kzalloc(dev, sizeof(*stfcamss), GFP_KERNEL);
>> >> + if (!stfcamss)
>> >> + return -ENOMEM;
>> >> +
>> >> + for (i = 0; i < ARRAY_SIZE(stfcamss->irq); ++i) {
>> >> + stfcamss->irq[i] = platform_get_irq(pdev, i);
>> >> + if (stfcamss->irq[i] < 0)
>> >> + return dev_err_probe(&pdev->dev, stfcamss->irq[i],
>> >> + "Failed to get irq%d", i);
>> >> + }
>> >> +
>> >> + stfcamss->nclks = ARRAY_SIZE(stfcamss->sys_clk);
>> >> + for (i = 0; i < stfcamss->nclks; ++i)
>> >> + stfcamss->sys_clk[i].id = stfcamss_clocks[i];
>> >> + ret = devm_clk_bulk_get(dev, stfcamss->nclks, stfcamss->sys_clk);
>> >> + if (ret) {
>> >> + dev_err(dev, "Failed to get clk controls\n");
>> >> + return ret;
>> >> + }
>> >> +
>> >> + stfcamss->nrsts = ARRAY_SIZE(stfcamss->sys_rst);
>> >> + for (i = 0; i < stfcamss->nrsts; ++i)
>> >> + stfcamss->sys_rst[i].id = stfcamss_resets[i];
>> >> + ret = devm_reset_control_bulk_get_shared(dev, stfcamss->nrsts,
>> >> + stfcamss->sys_rst);
>> >> + if (ret) {
>> >> + dev_err(dev, "Failed to get reset controls\n");
>> >> + return ret;
>> >> + }
>> >> +
>> >> + ret = stfcamss_get_mem_res(pdev, stfcamss);
>> >> + if (ret) {
>> >> + dev_err(dev, "Could not map registers\n");
>> >> + return ret;
>> >> + }
>> >> +
>> >> + stfcamss->dev = dev;
>> >
>> > Move this right after allocating stfcamss, and drop the pdev argument to
>> > stfcamss_get_mem_res(). The platform device can be retrieved in the
>> > function using to_platform_device().
>>
>> OK, I will modify.
>>
>> >> + platform_set_drvdata(pdev, stfcamss);
>> >> +
>> >> + v4l2_async_nf_init(&stfcamss->notifier);
>> >> +
>> >> + num_subdevs = stfcamss_of_parse_ports(stfcamss);
>> >> + if (num_subdevs < 0) {
>> >> + ret = -ENODEV;
>> >
>> > An error message would be useful, silent errors are hard to debug.
>>
>> OK, will add error printing information.
>>
>> >> + goto err_cleanup_notifier;
>> >> + }
>> >> +
>> >> + stfcamss_mc_init(pdev, stfcamss);
>> >> +
>> >> + ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
>> >> + if (ret < 0) {
>> >> + dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
>> >> + goto err_cleanup_notifier;
>> >> + }
>> >> +
>> >> + ret = media_device_register(&stfcamss->media_dev);
>> >> + if (ret) {
>> >> + dev_err(dev, "Failed to register media device: %d\n", ret);
>> >> + goto err_unregister_device;
>> >> + }
>> >> +
>> >> + pm_runtime_enable(dev);
>> >
>> > Would it be useful to enable autosuspend too, to avoid expensive
>> > suspend/resume cycles when userspace wants to briefly stop capture and
>> > restart it immediately ?
>>
>> It seems rare to use autosuspend in the Linux camera system.
>
> It's a relatively recent practice, and is more common in sensor drivers
> than ISP drivers, but I think it's a good practice nonetheless. It makes
> stop-reconfigure-start cycles much faster.
>
Yes, I agree with you, but the existing applications on our platform are
relatively simple, and I want to keep this usage for now.
>> >> +
>> >> + stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
>> >> + ret = v4l2_async_nf_register(&stfcamss->v4l2_dev, &stfcamss->notifier);
>> >> + if (ret) {
>> >> + dev_err(dev, "Failed to register async subdev nodes: %d\n",
>> >> + ret);
>> >> + goto err_unregister_media_dev;
>> >
>> > You need to disable runtime PM in this error path.
>>
>> OK, will fix it.
>>
>> >> + }
>> >> +
>> >> + return 0;
>> >> +
>> >> +err_unregister_media_dev:
>> >> + media_device_unregister(&stfcamss->media_dev);
>> >> +err_unregister_device:
>> >> + v4l2_device_unregister(&stfcamss->v4l2_dev);
>> >> +err_cleanup_notifier:
>> >> + v4l2_async_nf_cleanup(&stfcamss->notifier);
>> >> + return ret;
>> >> +}
>
> [snip]
>
Hi Laurent,
Thank you for your comments.
On 2023/7/27 23:25, Laurent Pinchart wrote:
> Hi Jack,
>
> Thank you for the patch.
>
> On Mon, Jun 19, 2023 at 07:28:36PM +0800, Jack Zhu wrote:
>> Add video driver for StarFive Camera Subsystem.
>>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../media/platform/starfive/camss/Makefile | 4 +-
>> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
>> .../media/platform/starfive/camss/stf_video.h | 92 +++
>> 3 files changed, 819 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
>>
>> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> index d56ddd078a71..eb457917a914 100644
>> --- a/drivers/media/platform/starfive/camss/Makefile
>> +++ b/drivers/media/platform/starfive/camss/Makefile
>> @@ -3,6 +3,8 @@
>> # Makefile for StarFive Camera Subsystem driver
>> #
>>
>> -starfive-camss-objs += stf_camss.o
>> +starfive-camss-objs += \
>> + stf_camss.o \
>> + stf_video.o
>>
>> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
>> new file mode 100644
>> index 000000000000..2e6472fe51c6
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_video.c
>> @@ -0,0 +1,724 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_video.c
>> + *
>> + * StarFive Camera Subsystem - V4L2 device node
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#include <linux/pm_runtime.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-event.h>
>> +#include <media/v4l2-mc.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "stf_camss.h"
>> +#include "stf_video.h"
>> +
>> +static const struct stfcamss_format_info formats_pix_wr[] = {
>> + {
>> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SRGGB10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SGRBG10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SGBRG10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> + {
>> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
>> + .pixelformat = V4L2_PIX_FMT_SBGGR10,
>> + .planes = 1,
>> + .vsub = { 1 },
>> + .bpp = 10,
>> + },
>> +};
>> +
>> +static const struct stfcamss_format_info formats_pix_isp[] = {
>> + {
>> + .code = MEDIA_BUS_FMT_Y12_1X12,
>> + .pixelformat = V4L2_PIX_FMT_NV12,
>> + .planes = 2,
>> + .vsub = { 1, 2 },
>> + .bpp = 8,
>> + },
>> +};
>> +
>> +/* -----------------------------------------------------------------------------
>> + * Helper functions
>> + */
>> +
>> +static int video_find_format(u32 code, u32 pixelformat,
>> + struct stfcamss_video *video)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < video->nformats; ++i) {
>> + if (video->formats[i].code == code &&
>> + video->formats[i].pixelformat == pixelformat)
>> + return i;
>> + }
>> +
>> + for (i = 0; i < video->nformats; ++i)
>> + if (video->formats[i].code == code)
>> + return i;
>> +
>> + for (i = 0; i < video->nformats; ++i)
>> + if (video->formats[i].pixelformat == pixelformat)
>> + return i;
>> +
>
> This looks weird, I don't think it does what you expect below. I think
> you can drop the function, and instead use video_get_pfmt_by_mcode() to
> convert the mbus code to a pixel format, and compare it to the active
> pixel format in video_check_format().
>
>> + return -EINVAL;
>> +}
>> +
>> +static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
>> +{
>> + struct v4l2_pix_format *pix;
>> + const struct stfcamss_format_info *fi;
>> + u32 width, height;
>> + u32 bpl;
>> + unsigned int i;
>> +
>> + pix = &f->fmt.pix;
>
> You can initialize pix when declaring it.
>
>> +
>> + for (i = 0; i < video->nformats; i++)
>> + if (pix->pixelformat == video->formats[i].pixelformat)
>> + break;
>> +
>
> for (i = 0; i < video->nformats; i++) {
> if (pix->pixelformat == video->formats[i].pixelformat)
> break;
> }
>
> But a helper function that looks up a format by pixelformat, similar to
> video_get_pfmt_by_mcode(), would be useful. I think I would make all
> those helpers return a const struct stfcamss_format_info pointer instead
> of an index.
>
>> + if (i == video->nformats)
>> + i = 0; /* default format */
>> +
>> + fi = &video->formats[i];
>> + width = pix->width;
>> + height = pix->height;
>> +
>> + memset(pix, 0, sizeof(*pix));
>> +
>> + pix->pixelformat = fi->pixelformat;
>> + pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
>> + STFCAMSS_FRAME_MAX_WIDTH);
>> + pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
>> + STFCAMSS_FRAME_MAX_HEIGHT);
>> + bpl = pix->width * fi->bpp / 8;
>> + bpl = ALIGN(bpl, video->bpl_alignment);
>> + pix->bytesperline = bpl;
>
> Does the hardware support configuring the stride ?
>
The hardware does not support.
>> +
>> + for (i = 0; i < fi->planes; ++i)
>> + pix->sizeimage += bpl * pix->height / fi->vsub[i];
>> +
>> + pix->field = V4L2_FIELD_NONE;
>> + pix->colorspace = V4L2_COLORSPACE_SRGB;
>> + pix->flags = 0;
>> + pix->ycbcr_enc =
>> + V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
>> + pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
>> + pix->colorspace,
>> + pix->ycbcr_enc);
>> + pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
>
> This doesn't seem right for the processed output.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int stf_video_init_format(struct stfcamss_video *video)
>> +{
>> + int ret;
>> + struct v4l2_format format = {
>> + .type = video->type,
>> + .fmt.pix = {
>> + .width = 1920,
>> + .height = 1080,
>> + .pixelformat = V4L2_PIX_FMT_RGB565,
>
> That format doesn't seem supported, let's pick V4L2_PIX_FMT_NV12.
>
>> + },
>> + };
>> +
>> + ret = __video_try_fmt(video, &format);
>> +
>> + if (ret < 0)
>> + return ret;
>> +
>> + video->active_fmt = format;
>> +
>> + return 0;
>> +}
>> +
>> +/* -----------------------------------------------------------------------------
>> + * Video queue operations
>> + */
>> +
>> +static int video_queue_setup(struct vb2_queue *q,
>> + unsigned int *num_buffers,
>> + unsigned int *num_planes,
>> + unsigned int sizes[],
>> + struct device *alloc_devs[])
>> +{
>> + struct stfcamss_video *video = vb2_get_drv_priv(q);
>> + const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
>> +
>> + if (*num_planes) {
>> + if (*num_planes != 1)
>> + return -EINVAL;
>> +
>> + if (sizes[0] < format->sizeimage)
>> + return -EINVAL;
>> + }
>> +
>> + *num_planes = 1;
>> + sizes[0] = format->sizeimage;
>> + if (!sizes[0])
>> + dev_err(video->stfcamss->dev,
>> + "%s: error size is zero!!!\n", __func__);
>
> Shouldn't you return an error ? Also, use dev_dbg(), printing an error
> message based on a condition that can easily be triggered by
> unpriviledge userspace opens the door to applications flooding the
> kernel log.
>
>> +
>> + dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
>> + *num_planes, sizes[0]);
>> +
>> + return 0;
>> +}
>> +
>> +static int video_buf_init(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + struct stfcamss_buffer *buffer =
>> + container_of(vbuf, struct stfcamss_buffer, vb);
>
> A static inline to_stfcamss_buffer() function that wraps the
> container_of() would be nice. You can use it below too.
>
>> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
>> + dma_addr_t *paddr;
>> +
>> + paddr = vb2_plane_cookie(vb, 0);
>> + buffer->addr[0] = *paddr;
>> +
>> + if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
>> + fmt->pixelformat == V4L2_PIX_FMT_NV61)
>
> Only V4L2_PIX_FMT_NV12 is listed in formats_pix_isp. Does the hardware
> support the other formats ? If so, it would be nice to support them
> already.
>
>> + buffer->addr[1] =
>> + buffer->addr[0] + fmt->bytesperline * fmt->height;
>
> As the hardware supports non-contiguous planes, you should use the
> MPLANE API (V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) and support the NV*M
> formats in addition to the NV* formats.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int video_buf_prepare(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
>> +
>> + if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
>> + dev_err(video->stfcamss->dev,
>
> dev_dbg() here too.
>
>> + "sizeimage = %d, plane size = %d\n",
>> + fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
>
> Both are unsigned, use %u instead of %d.
>
>> + return -EINVAL;
>> + }
>> + vb2_set_plane_payload(vb, 0, fmt->sizeimage);
>> +
>> + vbuf->field = V4L2_FIELD_NONE;
>> +
>> + return 0;
>> +}
>> +
>> +static void video_buf_queue(struct vb2_buffer *vb)
>> +{
>> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
>> + struct stfcamss_buffer *buffer =
>> + container_of(vbuf, struct stfcamss_buffer, vb);
>> +
>> + video->ops->queue_buffer(video, buffer);
>> +}
>> +
>> +/*
>> + * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
>> + * @mbus: v4l2_mbus_framefmt format (input)
>> + * @pix: v4l2_pix_format_mplane format (output)
>> + * @f: a pointer to formats array element to be used for the conversion
>> + * @alignment: bytesperline alignment value
>> + *
>> + * Fill the output pix structure with information from the input mbus format.
>> + *
>> + * Return 0 on success or a negative error code otherwise
>> + */
>> +static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
>> + struct v4l2_pix_format *pix,
>> + const struct stfcamss_format_info *f,
>> + unsigned int alignment)
>> +{
>> + u32 bytesperline;
>> + unsigned int i;
>> +
>> + memset(pix, 0, sizeof(*pix));
>> + v4l2_fill_pix_format(pix, mbus);
>> + pix->pixelformat = f->pixelformat;
>> + bytesperline = pix->width * f->bpp / 8;
>> + bytesperline = ALIGN(bytesperline, alignment);
>> + pix->bytesperline = bytesperline;
>> +
>> + for (i = 0; i < f->planes; ++i)
>> + pix->sizeimage += bytesperline * pix->height / f->vsub[i];
>
> This function is used for validation of the format only, the
> bytesperline and sizeimage values are never used. You can simplify the
> driver by dropping the function and comparing the width, height and
> field of the subdev and video device from the v4l2_mbus_framefmt and
> v4l2_pix_format respectively in video_check_format().
> video_get_subdev_format() will then take a v4l2_mbus_framefmt pointer,
> not a v4l2_pix_format.
>
> The format match check still needs conversion of the
>
> To check the format, you need to convert the mbus code from the subdev
> to a pixel format using the
>
>> +
>> + return 0;
>> +}
>> +
>> +static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
>> + u32 *pad)
>> +{
>> + struct media_pad *remote;
>> +
>> + remote = media_pad_remote_pad_first(&video->pad);
>> +
>> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
>> + return NULL;
>> +
>> + if (pad)
>> + *pad = remote->index;
>> +
>> + return media_entity_to_v4l2_subdev(remote->entity);
>
> As the connected subdev is always the same (the CSI-2 RX for the raw
> capture video device and the ISP for the processed capture video
> device), I would store a pointer to the connected subdev in the
> stfcamss_video structure at registration time. You can pass the pointer
> to the stf_video_register() function.
>
As the hardware also supports the dvp interface, I think the current
function implementation should be flexible and easy to expand later.
>> +}
>> +
>> +static int video_get_subdev_format(struct stfcamss_video *video,
>> + struct v4l2_format *format)
>> +{
>> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
>> + struct v4l2_subdev_format fmt;
>> + struct v4l2_subdev *subdev;
>> + u32 pixelformat;
>> + u32 pad;
>> + int ret;
>> +
>> + subdev = video_remote_subdev(video, &pad);
>> + if (!subdev)
>> + return -EPIPE;
>> +
>> + fmt.pad = pad;
>> + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
>> +
>> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
>
> Use v4l2_subdev_call_state_active() to support the subdev state API.
>
>> + if (ret)
>> + return ret;
>> +
>> + pixelformat = pix->pixelformat;
>> + ret = video_find_format(fmt.format.code, pixelformat, video);
>> + if (ret < 0)
>> + return ret;
>> +
>> + format->type = video->type;
>> +
>> + return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
>> + &video->formats[ret], video->bpl_alignment);
>> +}
>> +
Regards,
Jack Zhu
Hi Jack,
On Wed, Aug 02, 2023 at 10:57:03AM +0800, Jack Zhu wrote:
> On 2023/7/27 23:25, Laurent Pinchart wrote:
> > On Mon, Jun 19, 2023 at 07:28:36PM +0800, Jack Zhu wrote:
> >> Add video driver for StarFive Camera Subsystem.
> >>
> >> Signed-off-by: Jack Zhu <[email protected]>
> >> ---
> >> .../media/platform/starfive/camss/Makefile | 4 +-
> >> .../media/platform/starfive/camss/stf_video.c | 724 ++++++++++++++++++
> >> .../media/platform/starfive/camss/stf_video.h | 92 +++
> >> 3 files changed, 819 insertions(+), 1 deletion(-)
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.c
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_video.h
> >>
> >> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> >> index d56ddd078a71..eb457917a914 100644
> >> --- a/drivers/media/platform/starfive/camss/Makefile
> >> +++ b/drivers/media/platform/starfive/camss/Makefile
> >> @@ -3,6 +3,8 @@
> >> # Makefile for StarFive Camera Subsystem driver
> >> #
> >>
> >> -starfive-camss-objs += stf_camss.o
> >> +starfive-camss-objs += \
> >> + stf_camss.o \
> >> + stf_video.o
> >>
> >> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> >> diff --git a/drivers/media/platform/starfive/camss/stf_video.c b/drivers/media/platform/starfive/camss/stf_video.c
> >> new file mode 100644
> >> index 000000000000..2e6472fe51c6
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_video.c
> >> @@ -0,0 +1,724 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * stf_video.c
> >> + *
> >> + * StarFive Camera Subsystem - V4L2 device node
> >> + *
> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> + */
> >> +
> >> +#include <linux/pm_runtime.h>
> >> +#include <media/v4l2-ctrls.h>
> >> +#include <media/v4l2-event.h>
> >> +#include <media/v4l2-mc.h>
> >> +#include <media/videobuf2-dma-contig.h>
> >> +
> >> +#include "stf_camss.h"
> >> +#include "stf_video.h"
> >> +
> >> +static const struct stfcamss_format_info formats_pix_wr[] = {
> >> + {
> >> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> >> + .pixelformat = V4L2_PIX_FMT_SRGGB10,
> >> + .planes = 1,
> >> + .vsub = { 1 },
> >> + .bpp = 10,
> >> + },
> >> + {
> >> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> >> + .pixelformat = V4L2_PIX_FMT_SGRBG10,
> >> + .planes = 1,
> >> + .vsub = { 1 },
> >> + .bpp = 10,
> >> + },
> >> + {
> >> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> >> + .pixelformat = V4L2_PIX_FMT_SGBRG10,
> >> + .planes = 1,
> >> + .vsub = { 1 },
> >> + .bpp = 10,
> >> + },
> >> + {
> >> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> >> + .pixelformat = V4L2_PIX_FMT_SBGGR10,
> >> + .planes = 1,
> >> + .vsub = { 1 },
> >> + .bpp = 10,
> >> + },
> >> +};
> >> +
> >> +static const struct stfcamss_format_info formats_pix_isp[] = {
> >> + {
> >> + .code = MEDIA_BUS_FMT_Y12_1X12,
> >> + .pixelformat = V4L2_PIX_FMT_NV12,
> >> + .planes = 2,
> >> + .vsub = { 1, 2 },
> >> + .bpp = 8,
> >> + },
> >> +};
> >> +
> >> +/* -----------------------------------------------------------------------------
> >> + * Helper functions
> >> + */
> >> +
> >> +static int video_find_format(u32 code, u32 pixelformat,
> >> + struct stfcamss_video *video)
> >> +{
> >> + unsigned int i;
> >> +
> >> + for (i = 0; i < video->nformats; ++i) {
> >> + if (video->formats[i].code == code &&
> >> + video->formats[i].pixelformat == pixelformat)
> >> + return i;
> >> + }
> >> +
> >> + for (i = 0; i < video->nformats; ++i)
> >> + if (video->formats[i].code == code)
> >> + return i;
> >> +
> >> + for (i = 0; i < video->nformats; ++i)
> >> + if (video->formats[i].pixelformat == pixelformat)
> >> + return i;
> >> +
> >
> > This looks weird, I don't think it does what you expect below. I think
> > you can drop the function, and instead use video_get_pfmt_by_mcode() to
> > convert the mbus code to a pixel format, and compare it to the active
> > pixel format in video_check_format().
> >
> >> + return -EINVAL;
> >> +}
> >> +
> >> +static int __video_try_fmt(struct stfcamss_video *video, struct v4l2_format *f)
> >> +{
> >> + struct v4l2_pix_format *pix;
> >> + const struct stfcamss_format_info *fi;
> >> + u32 width, height;
> >> + u32 bpl;
> >> + unsigned int i;
> >> +
> >> + pix = &f->fmt.pix;
> >
> > You can initialize pix when declaring it.
> >
> >> +
> >> + for (i = 0; i < video->nformats; i++)
> >> + if (pix->pixelformat == video->formats[i].pixelformat)
> >> + break;
> >> +
> >
> > for (i = 0; i < video->nformats; i++) {
> > if (pix->pixelformat == video->formats[i].pixelformat)
> > break;
> > }
> >
> > But a helper function that looks up a format by pixelformat, similar to
> > video_get_pfmt_by_mcode(), would be useful. I think I would make all
> > those helpers return a const struct stfcamss_format_info pointer instead
> > of an index.
> >
> >> + if (i == video->nformats)
> >> + i = 0; /* default format */
> >> +
> >> + fi = &video->formats[i];
> >> + width = pix->width;
> >> + height = pix->height;
> >> +
> >> + memset(pix, 0, sizeof(*pix));
> >> +
> >> + pix->pixelformat = fi->pixelformat;
> >> + pix->width = clamp_t(u32, width, STFCAMSS_FRAME_MIN_WIDTH,
> >> + STFCAMSS_FRAME_MAX_WIDTH);
> >> + pix->height = clamp_t(u32, height, STFCAMSS_FRAME_MIN_HEIGHT,
> >> + STFCAMSS_FRAME_MAX_HEIGHT);
> >> + bpl = pix->width * fi->bpp / 8;
> >> + bpl = ALIGN(bpl, video->bpl_alignment);
> >> + pix->bytesperline = bpl;
> >
> > Does the hardware support configuring the stride ?
>
> The hardware does not support.
>
> >> +
> >> + for (i = 0; i < fi->planes; ++i)
> >> + pix->sizeimage += bpl * pix->height / fi->vsub[i];
> >> +
> >> + pix->field = V4L2_FIELD_NONE;
> >> + pix->colorspace = V4L2_COLORSPACE_SRGB;
> >> + pix->flags = 0;
> >> + pix->ycbcr_enc =
> >> + V4L2_MAP_YCBCR_ENC_DEFAULT(pix->colorspace);
> >> + pix->quantization = V4L2_MAP_QUANTIZATION_DEFAULT(true,
> >> + pix->colorspace,
> >> + pix->ycbcr_enc);
> >> + pix->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(pix->colorspace);
> >
> > This doesn't seem right for the processed output.
> >
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int stf_video_init_format(struct stfcamss_video *video)
> >> +{
> >> + int ret;
> >> + struct v4l2_format format = {
> >> + .type = video->type,
> >> + .fmt.pix = {
> >> + .width = 1920,
> >> + .height = 1080,
> >> + .pixelformat = V4L2_PIX_FMT_RGB565,
> >
> > That format doesn't seem supported, let's pick V4L2_PIX_FMT_NV12.
> >
> >> + },
> >> + };
> >> +
> >> + ret = __video_try_fmt(video, &format);
> >> +
> >> + if (ret < 0)
> >> + return ret;
> >> +
> >> + video->active_fmt = format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +/* -----------------------------------------------------------------------------
> >> + * Video queue operations
> >> + */
> >> +
> >> +static int video_queue_setup(struct vb2_queue *q,
> >> + unsigned int *num_buffers,
> >> + unsigned int *num_planes,
> >> + unsigned int sizes[],
> >> + struct device *alloc_devs[])
> >> +{
> >> + struct stfcamss_video *video = vb2_get_drv_priv(q);
> >> + const struct v4l2_pix_format *format = &video->active_fmt.fmt.pix;
> >> +
> >> + if (*num_planes) {
> >> + if (*num_planes != 1)
> >> + return -EINVAL;
> >> +
> >> + if (sizes[0] < format->sizeimage)
> >> + return -EINVAL;
> >> + }
> >> +
> >> + *num_planes = 1;
> >> + sizes[0] = format->sizeimage;
> >> + if (!sizes[0])
> >> + dev_err(video->stfcamss->dev,
> >> + "%s: error size is zero!!!\n", __func__);
> >
> > Shouldn't you return an error ? Also, use dev_dbg(), printing an error
> > message based on a condition that can easily be triggered by
> > unpriviledge userspace opens the door to applications flooding the
> > kernel log.
> >
> >> +
> >> + dev_dbg(video->stfcamss->dev, "planes = %d, size = %d\n",
> >> + *num_planes, sizes[0]);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int video_buf_init(struct vb2_buffer *vb)
> >> +{
> >> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> >> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> >> + struct stfcamss_buffer *buffer =
> >> + container_of(vbuf, struct stfcamss_buffer, vb);
> >
> > A static inline to_stfcamss_buffer() function that wraps the
> > container_of() would be nice. You can use it below too.
> >
> >> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> >> + dma_addr_t *paddr;
> >> +
> >> + paddr = vb2_plane_cookie(vb, 0);
> >> + buffer->addr[0] = *paddr;
> >> +
> >> + if (fmt->pixelformat == V4L2_PIX_FMT_NV12 ||
> >> + fmt->pixelformat == V4L2_PIX_FMT_NV21 ||
> >> + fmt->pixelformat == V4L2_PIX_FMT_NV16 ||
> >> + fmt->pixelformat == V4L2_PIX_FMT_NV61)
> >
> > Only V4L2_PIX_FMT_NV12 is listed in formats_pix_isp. Does the hardware
> > support the other formats ? If so, it would be nice to support them
> > already.
> >
> >> + buffer->addr[1] =
> >> + buffer->addr[0] + fmt->bytesperline * fmt->height;
> >
> > As the hardware supports non-contiguous planes, you should use the
> > MPLANE API (V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) and support the NV*M
> > formats in addition to the NV* formats.
> >
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int video_buf_prepare(struct vb2_buffer *vb)
> >> +{
> >> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> >> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> >> + const struct v4l2_pix_format *fmt = &video->active_fmt.fmt.pix;
> >> +
> >> + if (fmt->sizeimage > vb2_plane_size(vb, 0)) {
> >> + dev_err(video->stfcamss->dev,
> >
> > dev_dbg() here too.
> >
> >> + "sizeimage = %d, plane size = %d\n",
> >> + fmt->sizeimage, (unsigned int)vb2_plane_size(vb, 0));
> >
> > Both are unsigned, use %u instead of %d.
> >
> >> + return -EINVAL;
> >> + }
> >> + vb2_set_plane_payload(vb, 0, fmt->sizeimage);
> >> +
> >> + vbuf->field = V4L2_FIELD_NONE;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static void video_buf_queue(struct vb2_buffer *vb)
> >> +{
> >> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> >> + struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> >> + struct stfcamss_buffer *buffer =
> >> + container_of(vbuf, struct stfcamss_buffer, vb);
> >> +
> >> + video->ops->queue_buffer(video, buffer);
> >> +}
> >> +
> >> +/*
> >> + * video_mbus_to_pix - Convert v4l2_mbus_framefmt to v4l2_pix_format
> >> + * @mbus: v4l2_mbus_framefmt format (input)
> >> + * @pix: v4l2_pix_format_mplane format (output)
> >> + * @f: a pointer to formats array element to be used for the conversion
> >> + * @alignment: bytesperline alignment value
> >> + *
> >> + * Fill the output pix structure with information from the input mbus format.
> >> + *
> >> + * Return 0 on success or a negative error code otherwise
> >> + */
> >> +static int video_mbus_to_pix(const struct v4l2_mbus_framefmt *mbus,
> >> + struct v4l2_pix_format *pix,
> >> + const struct stfcamss_format_info *f,
> >> + unsigned int alignment)
> >> +{
> >> + u32 bytesperline;
> >> + unsigned int i;
> >> +
> >> + memset(pix, 0, sizeof(*pix));
> >> + v4l2_fill_pix_format(pix, mbus);
> >> + pix->pixelformat = f->pixelformat;
> >> + bytesperline = pix->width * f->bpp / 8;
> >> + bytesperline = ALIGN(bytesperline, alignment);
> >> + pix->bytesperline = bytesperline;
> >> +
> >> + for (i = 0; i < f->planes; ++i)
> >> + pix->sizeimage += bytesperline * pix->height / f->vsub[i];
> >
> > This function is used for validation of the format only, the
> > bytesperline and sizeimage values are never used. You can simplify the
> > driver by dropping the function and comparing the width, height and
> > field of the subdev and video device from the v4l2_mbus_framefmt and
> > v4l2_pix_format respectively in video_check_format().
> > video_get_subdev_format() will then take a v4l2_mbus_framefmt pointer,
> > not a v4l2_pix_format.
> >
> > The format match check still needs conversion of the
> >
> > To check the format, you need to convert the mbus code from the subdev
> > to a pixel format using the
> >
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static struct v4l2_subdev *video_remote_subdev(struct stfcamss_video *video,
> >> + u32 *pad)
> >> +{
> >> + struct media_pad *remote;
> >> +
> >> + remote = media_pad_remote_pad_first(&video->pad);
> >> +
> >> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
> >> + return NULL;
> >> +
> >> + if (pad)
> >> + *pad = remote->index;
> >> +
> >> + return media_entity_to_v4l2_subdev(remote->entity);
> >
> > As the connected subdev is always the same (the CSI-2 RX for the raw
> > capture video device and the ISP for the processed capture video
> > device), I would store a pointer to the connected subdev in the
> > stfcamss_video structure at registration time. You can pass the pointer
> > to the stf_video_register() function.
>
> As the hardware also supports the dvp interface, I think the current
> function implementation should be flexible and easy to expand later.
Fair enough, I'm fine with that.
> >> +}
> >> +
> >> +static int video_get_subdev_format(struct stfcamss_video *video,
> >> + struct v4l2_format *format)
> >> +{
> >> + struct v4l2_pix_format *pix = &video->active_fmt.fmt.pix;
> >> + struct v4l2_subdev_format fmt;
> >> + struct v4l2_subdev *subdev;
> >> + u32 pixelformat;
> >> + u32 pad;
> >> + int ret;
> >> +
> >> + subdev = video_remote_subdev(video, &pad);
> >> + if (!subdev)
> >> + return -EPIPE;
> >> +
> >> + fmt.pad = pad;
> >> + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> >> +
> >> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &fmt);
> >
> > Use v4l2_subdev_call_state_active() to support the subdev state API.
> >
> >> + if (ret)
> >> + return ret;
> >> +
> >> + pixelformat = pix->pixelformat;
> >> + ret = video_find_format(fmt.format.code, pixelformat, video);
> >> + if (ret < 0)
> >> + return ret;
> >> +
> >> + format->type = video->type;
> >> +
> >> + return video_mbus_to_pix(&fmt.format, &format->fmt.pix,
> >> + &video->formats[ret], video->bpl_alignment);
> >> +}
> >> +
--
Regards,
Laurent Pinchart
Hi Laurent,
Thank you for your comments.
On 2023/7/28 4:49, Laurent Pinchart wrote:
> Hi Jack,
>
> Thank you for the patch.
>
> On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
>> Add Video In Controller driver for StarFive Camera Subsystem.
>
> I haven't reviewed this patch in details, as I have a high-level
> question: why do you need VIN subdevs ? They don't seem to represent any
> particular piece of hardware, their input and output formats are always
> identical, and they're not used to configure registers. The contents of
> this patch seems to belong to the video device, I think you can drop the
> VIN subdevs.
>
The VIN module corresponds to a hardware module, which is mainly responsible
for data routing and partial interrupt management (when the image data does
not pass through the isp but directly goes to the ddr), the relevant registers
need to be configured.
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../media/platform/starfive/camss/Makefile | 4 +-
>> .../media/platform/starfive/camss/stf_camss.c | 42 +-
>> .../media/platform/starfive/camss/stf_camss.h | 2 +
>> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
>> .../media/platform/starfive/camss/stf_vin.h | 173 +++
>> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
>> 6 files changed, 1478 insertions(+), 4 deletions(-)
>> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
>> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>>
>> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> index cdf57e8c9546..ef574e01ca47 100644
>> --- a/drivers/media/platform/starfive/camss/Makefile
>> +++ b/drivers/media/platform/starfive/camss/Makefile
>> @@ -7,6 +7,8 @@ starfive-camss-objs += \
>> stf_camss.o \
>> stf_isp.o \
>> stf_isp_hw_ops.o \
>> - stf_video.o
>> + stf_video.o \
>> + stf_vin.o \
>> + stf_vin_hw_ops.o
>>
>> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> index 6f56b45f57db..834ea63eb833 100644
>> --- a/drivers/media/platform/starfive/camss/stf_camss.c
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
>> return ret;
>> }
>>
>> + ret = stf_vin_subdev_init(stfcamss);
>> + if (ret < 0) {
>> + dev_err(stfcamss->dev, "Failed to init vin subdev: %d\n", ret);
>> + return ret;
>> + }
>> return ret;
>> }
>>
>> static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
>> {
>> int ret;
>> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
>> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>>
>> ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
>> if (ret < 0) {
>> dev_err(stfcamss->dev,
>> "Failed to register stf isp%d entity: %d\n", 0, ret);
>> - return ret;
>> + goto err_reg_isp;
>> + }
>> +
>> + ret = stf_vin_register(vin_dev, &stfcamss->v4l2_dev);
>> + if (ret < 0) {
>> + dev_err(stfcamss->dev,
>> + "Failed to register vin entity: %d\n", ret);
>> + goto err_reg_vin;
>> }
>>
>> + ret = media_create_pad_link(&isp_dev->subdev.entity,
>> + STF_ISP_PAD_SRC,
>> + &vin_dev->line[VIN_LINE_ISP].subdev.entity,
>> + STF_VIN_PAD_SINK,
>> + 0);
>> + if (ret < 0) {
>> + dev_err(stfcamss->dev,
>> + "Failed to link %s->%s entities: %d\n",
>> + isp_dev->subdev.entity.name,
>> + vin_dev->line[VIN_LINE_ISP].subdev.entity.name, ret);
>> + goto err_link;
>> + }
>> +
>> + return ret;
>> +
>> +err_link:
>> + stf_vin_unregister(&stfcamss->vin_dev);
>> +err_reg_vin:
>> + stf_isp_unregister(&stfcamss->isp_dev);
>> +err_reg_isp:
>> return ret;
>> }
>>
>> static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
>> {
>> stf_isp_unregister(&stfcamss->isp_dev);
>> + stf_vin_unregister(&stfcamss->vin_dev);
>> }
>>
>> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
>> @@ -164,12 +198,14 @@ static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
>> container_of(asd, struct stfcamss_async_subdev, asd);
>> enum stf_port_num port = csd->port;
>> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
>> struct media_pad *pad[STF_PADS_NUM];
>> unsigned int i, pad_num;
>>
>> if (port == STF_PORT_CSI2RX) {
>> - pad[0] = &isp_dev->pads[STF_PAD_SINK];
>> - pad_num = 1;
>> + pad[0] = &vin_dev->line[VIN_LINE_WR].pads[STF_PAD_SINK];
>> + pad[1] = &isp_dev->pads[STF_PAD_SINK];
>> + pad_num = 2;
>> } else if (port == STF_PORT_DVP) {
>> dev_err(stfcamss->dev, "Not support DVP sensor\n");
>> return -EPERM;
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
>> index 9482081288fa..a14f22bc0742 100644
>> --- a/drivers/media/platform/starfive/camss/stf_camss.h
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
>> @@ -19,6 +19,7 @@
>> #include <media/v4l2-device.h>
>>
>> #include "stf_isp.h"
>> +#include "stf_vin.h"
>>
>> #define STF_DVP_NAME "stf_dvp"
>> #define STF_CSI_NAME "cdns_csi2rx"
>> @@ -67,6 +68,7 @@ struct stfcamss {
>> struct media_device media_dev;
>> struct media_pipeline pipe;
>> struct device *dev;
>> + struct stf_vin_dev vin_dev;
>> struct stf_isp_dev isp_dev;
>> struct v4l2_async_notifier notifier;
>> void __iomem *syscon_base;
>> diff --git a/drivers/media/platform/starfive/camss/stf_vin.c b/drivers/media/platform/starfive/camss/stf_vin.c
>> new file mode 100644
>> index 000000000000..0efa4bbb079c
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_vin.c
>> @@ -0,0 +1,1069 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_vin.c
>> + *
>> + * StarFive Camera Subsystem - VIN Module
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#include <linux/dma-mapping.h>
>> +#include <linux/pm_runtime.h>
>> +
>> +#include "stf_camss.h"
>> +
>> +#define vin_line_array(ptr_line) \
>> + ((const struct vin_line (*)[]) &(ptr_line)[-((ptr_line)->id)])
>> +
>> +#define line_to_vin_dev(ptr_line) \
>> + container_of(vin_line_array(ptr_line), struct stf_vin_dev, line)
>> +
>> +#define VIN_FRAME_DROP_MIN_VAL 4
>> +
>> +/* ISP ctrl need 1 sec to let frames become stable. */
>> +#define VIN_FRAME_DROP_SEC_FOR_ISP_CTRL 1
>> +
>> +static const struct vin_format vin_formats_wr[] = {
>> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10},
>> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10},
>> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10},
>> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10},
>> +};
>> +
>> +static const struct vin_format vin_formats_uo[] = {
>> + { MEDIA_BUS_FMT_Y12_1X12, 8},
>> +};
>> +
>> +static const struct vin_format_table vin_formats_table[] = {
>> + /* VIN_LINE_WR */
>> + { vin_formats_wr, ARRAY_SIZE(vin_formats_wr) },
>> + /* VIN_LINE_ISP */
>> + { vin_formats_uo, ARRAY_SIZE(vin_formats_uo) },
>> +};
>> +
>> +static void vin_buffer_done(struct vin_line *line);
>> +static void vin_change_buffer(struct vin_line *line);
>> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output);
>> +static void vin_output_init_addrs(struct vin_line *line);
>> +static struct v4l2_mbus_framefmt *
>> +__vin_get_format(struct vin_line *line,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + enum v4l2_subdev_format_whence which);
>> +
>> +static char *vin_get_line_subdevname(int line_id)
>> +{
>> + char *name = NULL;
>> +
>> + switch (line_id) {
>> + case VIN_LINE_WR:
>> + name = "wr";
>> + break;
>> + case VIN_LINE_ISP:
>> + name = "isp0";
>> + break;
>> + default:
>> + name = "unknown";
>> + break;
>> + }
>> + return name;
>> +}
>> +
>> +static enum isp_line_id vin_map_isp_line(enum vin_line_id line)
>> +{
>> + enum isp_line_id line_id;
>> +
>> + if (line > VIN_LINE_WR && line < VIN_LINE_MAX)
>> + line_id = STF_ISP_LINE_SRC;
>> + else
>> + line_id = STF_ISP_LINE_INVALID;
>> +
>> + return line_id;
>> +}
>> +
>> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def)
>> +{
>> + enum isp_pad_id pad_id;
>> +
>> + if (line == VIN_LINE_WR)
>> + pad_id = STF_ISP_PAD_SINK;
>> + else if ((line > VIN_LINE_WR) && (line < VIN_LINE_MAX))
>> + pad_id = (enum isp_pad_id)vin_map_isp_line(line);
>> + else
>> + pad_id = def;
>> +
>> + return pad_id;
>> +}
>> +
>> +int stf_vin_subdev_init(struct stfcamss *stfcamss)
>> +{
>> + struct device *dev = stfcamss->dev;
>> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
>> + int i, ret = 0;
>> +
>> + vin_dev->stfcamss = stfcamss;
>> +
>> + vin_dev->isr_ops = devm_kzalloc(dev, sizeof(*vin_dev->isr_ops),
>> + GFP_KERNEL);
>> + if (!vin_dev->isr_ops)
>> + return -ENOMEM;
>> + vin_dev->isr_ops->isr_buffer_done = vin_buffer_done;
>> + vin_dev->isr_ops->isr_change_buffer = vin_change_buffer;
>> +
>> + atomic_set(&vin_dev->ref_count, 0);
>> +
>> + ret = devm_request_irq(dev,
>> + stfcamss->irq[STF_IRQ_VINWR],
>> + stf_vin_wr_irq_handler,
>> + 0, "vin_axiwr_irq", vin_dev);
>> + if (ret) {
>> + dev_err(dev, "Failed to request irq\n");
>> + goto out;
>> + }
>> +
>> + ret = devm_request_irq(dev,
>> + stfcamss->irq[STF_IRQ_ISP],
>> + stf_vin_isp_irq_handler,
>> + 0, "vin_isp_irq", vin_dev);
>> + if (ret) {
>> + dev_err(dev, "Failed to request isp irq\n");
>> + goto out;
>> + }
>> +
>> + ret = devm_request_irq(dev,
>> + stfcamss->irq[STF_IRQ_ISPCSIL],
>> + stf_vin_isp_irq_csiline_handler,
>> + 0, "vin_isp_irq_csiline", vin_dev);
>> + if (ret) {
>> + dev_err(dev, "failed to request isp irq csiline\n");
>> + goto out;
>> + }
>> +
>> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++) {
>> + struct dummy_buffer *dummy_buffer = &vin_dev->dummy_buffer[i];
>> +
>> + mutex_init(&dummy_buffer->stream_lock);
>> + dummy_buffer->nums =
>> + i == 0 ? VIN_DUMMY_BUFFER_NUMS : ISP_DUMMY_BUFFER_NUMS;
>> + dummy_buffer->stream_count = 0;
>> + dummy_buffer->buffer =
>> + devm_kzalloc(dev,
>> + dummy_buffer->nums * sizeof(struct vin_dummy_buffer),
>> + GFP_KERNEL);
>> + atomic_set(&dummy_buffer->frame_skip, 0);
>> + }
>> +
>> + for (i = VIN_LINE_WR; i < STF_ISP_LINE_MAX + 1; i++) {
>> + struct vin_line *l = &vin_dev->line[i];
>> +
>> + l->video_out.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>> + l->video_out.stfcamss = stfcamss;
>> + l->id = i;
>> + l->formats = vin_formats_table[i].fmts;
>> + l->nformats = vin_formats_table[i].nfmts;
>> + spin_lock_init(&l->output_lock);
>> +
>> + mutex_init(&l->stream_lock);
>> + l->stream_count = 0;
>> + }
>> +
>> + return 0;
>> +out:
>> + return ret;
>> +}
>> +
>> +static enum link vin_get_link(struct media_entity *entity)
>> +{
>> + struct v4l2_subdev *subdev;
>> + struct media_pad *pad;
>> + bool isp = false;
>> +
>> + while (1) {
>> + pad = &entity->pads[0];
>> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
>> + return LINK_ERROR;
>> +
>> + pad = media_pad_remote_pad_first(pad);
>> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
>> + return LINK_ERROR;
>> +
>> + entity = pad->entity;
>> + subdev = media_entity_to_v4l2_subdev(entity);
>> +
>> + if (!strncmp(subdev->name, STF_CSI_NAME,
>> + strlen(STF_CSI_NAME))) {
>> + if (isp)
>> + return LINK_CSI_TO_ISP;
>> + else
>> + return LINK_CSI_TO_WR;
>> + } else if (!strncmp(subdev->name, STF_DVP_NAME,
>> + strlen(STF_DVP_NAME))) {
>> + if (isp)
>> + return LINK_DVP_TO_ISP;
>> + else
>> + return LINK_DVP_TO_WR;
>> + } else if (!strncmp(subdev->name, STF_ISP_NAME,
>> + strlen(STF_ISP_NAME))) {
>> + isp = true;
>> + } else {
>> + return LINK_ERROR;
>> + }
>> + }
>> +}
>> +
>> +static int vin_enable_output(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + output->state = VIN_OUTPUT_IDLE;
>> +
>> + output->buf[0] = vin_buf_get_pending(output);
>> +
>> + if (!output->buf[0] && output->buf[1]) {
>> + output->buf[0] = output->buf[1];
>> + output->buf[1] = NULL;
>> + }
>> +
>> + if (output->buf[0])
>> + output->state = VIN_OUTPUT_SINGLE;
>> +
>> + output->sequence = 0;
>> +
>> + vin_output_init_addrs(line);
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> + return 0;
>> +}
>> +
>> +static int vin_disable_output(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + output->state = VIN_OUTPUT_OFF;
>> +
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> + return 0;
>> +}
>> +
>> +static u32 vin_line_to_dummy_module(struct vin_line *line)
>> +{
>> + u32 dummy_module = 0;
>> +
>> + switch (line->id) {
>> + case VIN_LINE_WR:
>> + dummy_module = STF_DUMMY_VIN;
>> + break;
>> + case VIN_LINE_ISP:
>> + dummy_module = STF_DUMMY_ISP;
>> + break;
>> + default:
>> + dummy_module = STF_DUMMY_VIN;
>> + break;
>> + }
>> +
>> + return dummy_module;
>> +}
>> +
>> +static int vin_alloc_dummy_buffer(struct stf_vin_dev *vin_dev,
>> + struct v4l2_mbus_framefmt *fmt,
>> + int dummy_module)
>> +{
>> + struct device *dev = vin_dev->stfcamss->dev;
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[dummy_module];
>> + struct vin_dummy_buffer *buffer = NULL;
>> + int ret = 0, i;
>> + u32 aligns;
>> +
>> + for (i = 0; i < dummy_buffer->nums; i++) {
>> + buffer = &vin_dev->dummy_buffer[dummy_module].buffer[i];
>> + buffer->width = fmt->width;
>> + buffer->height = fmt->height;
>> + buffer->mcode = fmt->code;
>> + if (i == STF_VIN_PAD_SINK) {
>> + aligns = ALIGN(fmt->width * 4,
>> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
>> + buffer->buffer_size = PAGE_ALIGN(aligns * fmt->height);
>> + } else if (i == STF_ISP_PAD_SRC) {
>> + aligns = ALIGN(fmt->width,
>> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
>> + buffer->buffer_size =
>> + PAGE_ALIGN(aligns * fmt->height * 3 / 2);
>> + } else {
>> + continue;
>> + }
>> +
>> + buffer->vaddr = dma_alloc_coherent(dev,
>> + buffer->buffer_size,
>> + &buffer->paddr[0],
>> + GFP_KERNEL | __GFP_NOWARN);
>> +
>> + if (buffer->vaddr) {
>> + if (i == STF_ISP_PAD_SRC)
>> + buffer->paddr[1] =
>> + (dma_addr_t)(buffer->paddr[0] + aligns * fmt->height);
>> + else
>> + dev_dbg(dev, "signal plane\n");
>> + }
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static void vin_free_dummy_buffer(struct stf_vin_dev *vin_dev, int dummy_module)
>> +{
>> + struct device *dev = vin_dev->stfcamss->dev;
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[dummy_module];
>> + struct vin_dummy_buffer *buffer = NULL;
>> + int i;
>> +
>> + for (i = 0; i < dummy_buffer->nums; i++) {
>> + buffer = &dummy_buffer->buffer[i];
>> + if (buffer->vaddr)
>> + dma_free_coherent(dev, buffer->buffer_size,
>> + buffer->vaddr, buffer->paddr[0]);
>> + memset(buffer, 0, sizeof(struct vin_dummy_buffer));
>> + }
>> +}
>> +
>> +static void vin_set_dummy_buffer(struct vin_line *line, u32 pad)
>> +{
>> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
>> + int dummy_module = vin_line_to_dummy_module(line);
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[dummy_module];
>> + struct vin_dummy_buffer *buffer = NULL;
>> +
>> + switch (pad) {
>> + case STF_VIN_PAD_SINK:
>> + if (line->id == VIN_LINE_WR) {
>> + buffer = &dummy_buffer->buffer[STF_VIN_PAD_SINK];
>> + stf_vin_wr_set_ping_addr(vin_dev, buffer->paddr[0]);
>> + stf_vin_wr_set_pong_addr(vin_dev, buffer->paddr[0]);
>> + } else {
>> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
>> + stf_vin_isp_set_yuv_addr(vin_dev,
>> + buffer->paddr[0],
>> + buffer->paddr[1]);
>> + }
>> + break;
>> + case STF_ISP_PAD_SRC:
>> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
>> + stf_vin_isp_set_yuv_addr(vin_dev,
>> + buffer->paddr[0],
>> + buffer->paddr[1]);
>> + break;
>> + default:
>> + break;
>> + }
>> +}
>> +
>> +static int vin_set_stream(struct v4l2_subdev *sd, int enable)
>> +{
>> + struct vin_line *line = v4l2_get_subdevdata(sd);
>> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
>> + int dummy_module = vin_line_to_dummy_module(line);
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[dummy_module];
>> + struct v4l2_mbus_framefmt *fmt;
>> + enum link link;
>> +
>> + fmt = __vin_get_format(line, NULL,
>> + STF_VIN_PAD_SINK, V4L2_SUBDEV_FORMAT_ACTIVE);
>> + mutex_lock(&dummy_buffer->stream_lock);
>> + if (enable) {
>> + if (dummy_buffer->stream_count == 0) {
>> + vin_alloc_dummy_buffer(vin_dev, fmt, dummy_module);
>> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
>> + atomic_set(&dummy_buffer->frame_skip,
>> + VIN_FRAME_DROP_MIN_VAL);
>> + }
>> + dummy_buffer->stream_count++;
>> + } else {
>> + if (dummy_buffer->stream_count == 1) {
>> + vin_free_dummy_buffer(vin_dev, dummy_module);
>> + /* set buffer addr to zero */
>> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
>> + } else {
>> + vin_set_dummy_buffer(line,
>> + stf_vin_map_isp_pad(line->id, STF_ISP_PAD_SINK));
>> + }
>> +
>> + dummy_buffer->stream_count--;
>> + }
>> + mutex_unlock(&dummy_buffer->stream_lock);
>> +
>> + mutex_lock(&line->stream_lock);
>> + link = vin_get_link(&sd->entity);
>> + if (link == LINK_ERROR)
>> + goto exit;
>> +
>> + if (enable) {
>> + if (line->stream_count == 0) {
>> + stf_vin_stream_set(vin_dev, link);
>> + if (line->id == VIN_LINE_WR) {
>> + stf_vin_wr_irq_enable(vin_dev, 1);
>> + stf_vin_wr_stream_set(vin_dev);
>> + }
>> + }
>> + line->stream_count++;
>> + } else {
>> + if (line->stream_count == 1 && line->id == VIN_LINE_WR)
>> + stf_vin_wr_irq_enable(vin_dev, 0);
>> + line->stream_count--;
>> + }
>> +exit:
>> + mutex_unlock(&line->stream_lock);
>> +
>> + if (enable)
>> + vin_enable_output(line);
>> + else
>> + vin_disable_output(line);
>> +
>> + return 0;
>> +}
>> +
>> +static struct v4l2_mbus_framefmt *
>> +__vin_get_format(struct vin_line *line,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + if (which == V4L2_SUBDEV_FORMAT_TRY)
>> + return v4l2_subdev_get_try_format(&line->subdev, state, pad);
>> + return &line->fmt[pad];
>> +}
>> +
>> +static void vin_try_format(struct vin_line *line,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + struct v4l2_mbus_framefmt *fmt,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + unsigned int i;
>> +
>> + switch (pad) {
>> + case STF_VIN_PAD_SINK:
>> + /* Set format on sink pad */
>> + for (i = 0; i < line->nformats; i++)
>> + if (fmt->code == line->formats[i].code)
>> + break;
>> +
>> + /* If not found, use UYVY as default */
>> + if (i >= line->nformats)
>> + fmt->code = line->formats[0].code;
>> +
>> + fmt->width = clamp_t(u32, fmt->width,
>> + STFCAMSS_FRAME_MIN_WIDTH,
>> + STFCAMSS_FRAME_MAX_WIDTH);
>> + fmt->height = clamp_t(u32, fmt->height,
>> + STFCAMSS_FRAME_MIN_HEIGHT,
>> + STFCAMSS_FRAME_MAX_HEIGHT);
>> +
>> + fmt->field = V4L2_FIELD_NONE;
>> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> + fmt->flags = 0;
>> + break;
>> +
>> + case STF_VIN_PAD_SRC:
>> + /* Set and return a format same as sink pad */
>> + *fmt = *__vin_get_format(line, state, STF_VIN_PAD_SINK, which);
>> + break;
>> + }
>> +
>> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> +}
>> +
>> +static int vin_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + struct vin_line *line = v4l2_get_subdevdata(sd);
>> +
>> + if (code->index >= line->nformats)
>> + return -EINVAL;
>> + if (code->pad == STF_VIN_PAD_SINK) {
>> + code->code = line->formats[code->index].code;
>> + } else {
>> + struct v4l2_mbus_framefmt *sink_fmt;
>> +
>> + sink_fmt = __vin_get_format(line, state, STF_VIN_PAD_SINK,
>> + code->which);
>> +
>> + code->code = sink_fmt->code;
>> + if (!code->code)
>> + return -EINVAL;
>> + }
>> + code->flags = 0;
>> +
>> + return 0;
>> +}
>> +
>> +static int vin_enum_frame_size(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + struct vin_line *line = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt format;
>> +
>> + if (fse->index != 0)
>> + return -EINVAL;
>> +
>> + format.code = fse->code;
>> + format.width = 1;
>> + format.height = 1;
>> + vin_try_format(line, state, fse->pad, &format, fse->which);
>> + fse->min_width = format.width;
>> + fse->min_height = format.height;
>> +
>> + if (format.code != fse->code)
>> + return -EINVAL;
>> +
>> + format.code = fse->code;
>> + format.width = -1;
>> + format.height = -1;
>> + vin_try_format(line, state, fse->pad, &format, fse->which);
>> + fse->max_width = format.width;
>> + fse->max_height = format.height;
>> +
>> + return 0;
>> +}
>> +
>> +static int vin_get_format(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct vin_line *line = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt *format;
>> +
>> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
>> + if (!format)
>> + return -EINVAL;
>> +
>> + fmt->format = *format;
>> +
>> + return 0;
>> +}
>> +
>> +static int vin_set_format(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct vin_line *line = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt *format;
>> +
>> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
>> + if (!format)
>> + return -EINVAL;
>> +
>> + mutex_lock(&line->stream_lock);
>> + if (line->stream_count) {
>> + fmt->format = *format;
>> + mutex_unlock(&line->stream_lock);
>> + goto out;
>> + } else {
>> + vin_try_format(line, state, fmt->pad, &fmt->format, fmt->which);
>> + *format = fmt->format;
>> + }
>> + mutex_unlock(&line->stream_lock);
>> +
>> + if (fmt->pad == STF_VIN_PAD_SINK) {
>> + /* Propagate the format from sink to source */
>> + format = __vin_get_format(line, state, STF_VIN_PAD_SRC,
>> + fmt->which);
>> +
>> + *format = fmt->format;
>> + vin_try_format(line, state, STF_VIN_PAD_SRC, format,
>> + fmt->which);
>> + }
>> +
>> +out:
>> + return 0;
>> +}
>> +
>> +static int vin_init_formats(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_fh *fh)
>> +{
>> + struct v4l2_subdev_format format = {
>> + .pad = STF_VIN_PAD_SINK,
>> + .which = fh ?
>> + V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
>> + .format = {
>> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
>> + .width = 1920,
>> + .height = 1080
>> + }
>> + };
>> +
>> + return vin_set_format(sd, fh ? fh->state : NULL, &format);
>> +}
>> +
>> +static void vin_output_init_addrs(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
>> + dma_addr_t ping_addr;
>> + dma_addr_t y_addr, uv_addr;
>> +
>> + output->active_buf = 0;
>> +
>> + if (output->buf[0]) {
>> + ping_addr = output->buf[0]->addr[0];
>> + y_addr = output->buf[0]->addr[0];
>> + uv_addr = output->buf[0]->addr[1];
>> + } else {
>> + return;
>> + }
>> +
>> + switch (vin_map_isp_line(line->id)) {
>> + case STF_ISP_LINE_SRC:
>> + stf_vin_isp_set_yuv_addr(vin_dev, y_addr, uv_addr);
>> + break;
>> + default:
>> + if (line->id == VIN_LINE_WR) {
>> + stf_vin_wr_set_ping_addr(vin_dev, ping_addr);
>> + stf_vin_wr_set_pong_addr(vin_dev, ping_addr);
>> + }
>> + break;
>> + }
>> +}
>> +
>> +static void vin_init_outputs(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> +
>> + output->state = VIN_OUTPUT_OFF;
>> + output->buf[0] = NULL;
>> + output->buf[1] = NULL;
>> + output->active_buf = 0;
>> + INIT_LIST_HEAD(&output->pending_bufs);
>> + INIT_LIST_HEAD(&output->ready_bufs);
>> +}
>> +
>> +static void vin_buf_add_ready(struct vin_output *output,
>> + struct stfcamss_buffer *buffer)
>> +{
>> + INIT_LIST_HEAD(&buffer->queue);
>> + list_add_tail(&buffer->queue, &output->ready_bufs);
>> +}
>> +
>> +static struct stfcamss_buffer *vin_buf_get_ready(struct vin_output *output)
>> +{
>> + struct stfcamss_buffer *buffer = NULL;
>> +
>> + if (!list_empty(&output->ready_bufs)) {
>> + buffer = list_first_entry(&output->ready_bufs,
>> + struct stfcamss_buffer,
>> + queue);
>> + list_del(&buffer->queue);
>> + }
>> +
>> + return buffer;
>> +}
>> +
>> +static void vin_buf_add_pending(struct vin_output *output,
>> + struct stfcamss_buffer *buffer)
>> +{
>> + INIT_LIST_HEAD(&buffer->queue);
>> + list_add_tail(&buffer->queue, &output->pending_bufs);
>> +}
>> +
>> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output)
>> +{
>> + struct stfcamss_buffer *buffer = NULL;
>> +
>> + if (!list_empty(&output->pending_bufs)) {
>> + buffer = list_first_entry(&output->pending_bufs,
>> + struct stfcamss_buffer,
>> + queue);
>> + list_del(&buffer->queue);
>> + }
>> +
>> + return buffer;
>> +}
>> +
>> +static void vin_buf_update_on_last(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> +
>> + switch (output->state) {
>> + case VIN_OUTPUT_CONTINUOUS:
>> + output->state = VIN_OUTPUT_SINGLE;
>> + output->active_buf = !output->active_buf;
>> + break;
>> + case VIN_OUTPUT_SINGLE:
>> + output->state = VIN_OUTPUT_STOPPING;
>> + break;
>> + default:
>> + break;
>> + }
>> +}
>> +
>> +static void vin_buf_update_on_next(struct vin_line *line)
>> +{
>> + struct vin_output *output = &line->output;
>> +
>> + switch (output->state) {
>> + case VIN_OUTPUT_CONTINUOUS:
>> + output->active_buf = !output->active_buf;
>> + break;
>> + case VIN_OUTPUT_SINGLE:
>> + default:
>> + break;
>> + }
>> +}
>> +
>> +static void vin_buf_update_on_new(struct vin_line *line,
>> + struct vin_output *output,
>> + struct stfcamss_buffer *new_buf)
>> +{
>> + switch (output->state) {
>> + case VIN_OUTPUT_SINGLE:
>> + vin_buf_add_pending(output, new_buf);
>> + break;
>> + case VIN_OUTPUT_IDLE:
>> + if (!output->buf[0]) {
>> + output->buf[0] = new_buf;
>> + vin_output_init_addrs(line);
>> + output->state = VIN_OUTPUT_SINGLE;
>> + } else {
>> + vin_buf_add_pending(output, new_buf);
>> + }
>> + break;
>> + case VIN_OUTPUT_STOPPING:
>> + if (output->last_buffer) {
>> + output->buf[output->active_buf] = output->last_buffer;
>> + output->last_buffer = NULL;
>> + }
>> +
>> + output->state = VIN_OUTPUT_SINGLE;
>> + vin_buf_add_pending(output, new_buf);
>> + break;
>> + case VIN_OUTPUT_CONTINUOUS:
>> + default:
>> + vin_buf_add_pending(output, new_buf);
>> + break;
>> + }
>> +}
>> +
>> +static void vin_buf_flush(struct vin_output *output,
>> + enum vb2_buffer_state state)
>> +{
>> + struct stfcamss_buffer *buf;
>> + struct stfcamss_buffer *t;
>> +
>> + list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
>> + vb2_buffer_done(&buf->vb.vb2_buf, state);
>> + list_del(&buf->queue);
>> + }
>> + list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
>> + vb2_buffer_done(&buf->vb.vb2_buf, state);
>> + list_del(&buf->queue);
>> + }
>> +}
>> +
>> +static void vin_buffer_done(struct vin_line *line)
>> +{
>> + struct stfcamss_buffer *ready_buf;
>> + struct vin_output *output = &line->output;
>> + unsigned long flags;
>> + u64 ts = ktime_get_ns();
>> +
>> + if (output->state == VIN_OUTPUT_OFF ||
>> + output->state == VIN_OUTPUT_RESERVED)
>> + return;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + while ((ready_buf = vin_buf_get_ready(output))) {
>> + ready_buf->vb.vb2_buf.timestamp = ts;
>> + ready_buf->vb.sequence = output->sequence++;
>> +
>> + vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>> + }
>> +
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> +}
>> +
>> +static void vin_change_buffer(struct vin_line *line)
>> +{
>> + struct stfcamss_buffer *ready_buf;
>> + struct vin_output *output = &line->output;
>> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
>> + dma_addr_t *new_addr;
>> + unsigned long flags;
>> + u32 active_index;
>> +
>> + if (output->state == VIN_OUTPUT_OFF ||
>> + output->state == VIN_OUTPUT_STOPPING ||
>> + output->state == VIN_OUTPUT_RESERVED ||
>> + output->state == VIN_OUTPUT_IDLE)
>> + return;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + active_index = output->active_buf;
>> +
>> + ready_buf = output->buf[active_index];
>> + if (!ready_buf) {
>> + dev_warn(vin_dev->stfcamss->dev, "Missing ready buf %d %d!\n",
>> + active_index, output->state);
>> + active_index = !active_index;
>> + ready_buf = output->buf[active_index];
>> + if (!ready_buf) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Missing ready buf2 %d %d!\n",
>> + active_index, output->state);
>> + goto out_unlock;
>> + }
>> + }
>> +
>> + /* Get next buffer */
>> + output->buf[active_index] = vin_buf_get_pending(output);
>> + if (!output->buf[active_index]) {
>> + /* No next buffer - set same address */
>> + new_addr = ready_buf->addr;
>> + vin_buf_update_on_last(line);
>> + } else {
>> + new_addr = output->buf[active_index]->addr;
>> + vin_buf_update_on_next(line);
>> + }
>> +
>> + if (output->state == VIN_OUTPUT_STOPPING) {
>> + output->last_buffer = ready_buf;
>> + } else {
>> + switch (vin_map_isp_line(line->id)) {
>> + case STF_ISP_LINE_SRC:
>> + stf_vin_isp_set_yuv_addr(vin_dev,
>> + new_addr[0],
>> + new_addr[1]);
>> + break;
>> + default:
>> + if (line->id == VIN_LINE_WR) {
>> + stf_vin_wr_set_ping_addr(vin_dev, new_addr[0]);
>> + stf_vin_wr_set_pong_addr(vin_dev, new_addr[0]);
>> + }
>> + break;
>> + }
>> +
>> + vin_buf_add_ready(output, ready_buf);
>> + }
>> +
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> + return;
>> +
>> +out_unlock:
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> +}
>> +
>> +static int vin_queue_buffer(struct stfcamss_video *vid,
>> + struct stfcamss_buffer *buf)
>> +{
>> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
>> + struct vin_output *output;
>> + unsigned long flags;
>> +
>> + output = &line->output;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + vin_buf_update_on_new(line, output, buf);
>> +
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static int vin_flush_buffers(struct stfcamss_video *vid,
>> + enum vb2_buffer_state state)
>> +{
>> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
>> + struct vin_output *output = &line->output;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&line->output_lock, flags);
>> +
>> + vin_buf_flush(output, state);
>> + if (output->buf[0])
>> + vb2_buffer_done(&output->buf[0]->vb.vb2_buf, state);
>> +
>> + if (output->buf[1])
>> + vb2_buffer_done(&output->buf[1]->vb.vb2_buf, state);
>> +
>> + if (output->last_buffer) {
>> + vb2_buffer_done(&output->last_buffer->vb.vb2_buf, state);
>> + output->last_buffer = NULL;
>> + }
>> + output->buf[0] = NULL;
>> + output->buf[1] = NULL;
>> +
>> + spin_unlock_irqrestore(&line->output_lock, flags);
>> + return 0;
>> +}
>> +
>> +static int vin_link_setup(struct media_entity *entity,
>> + const struct media_pad *local,
>> + const struct media_pad *remote, u32 flags)
>> +{
>> + if (flags & MEDIA_LNK_FL_ENABLED)
>> + if (media_pad_remote_pad_first(local))
>> + return -EBUSY;
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_video_ops vin_video_ops = {
>> + .s_stream = vin_set_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_pad_ops vin_pad_ops = {
>> + .enum_mbus_code = vin_enum_mbus_code,
>> + .enum_frame_size = vin_enum_frame_size,
>> + .get_fmt = vin_get_format,
>> + .set_fmt = vin_set_format,
>> +};
>> +
>> +static const struct v4l2_subdev_ops vin_v4l2_ops = {
>> + .video = &vin_video_ops,
>> + .pad = &vin_pad_ops,
>> +};
>> +
>> +static const struct v4l2_subdev_internal_ops vin_v4l2_internal_ops = {
>> + .open = vin_init_formats,
>> +};
>> +
>> +static const struct stfcamss_video_ops stfcamss_vin_video_ops = {
>> + .queue_buffer = vin_queue_buffer,
>> + .flush_buffers = vin_flush_buffers,
>> +};
>> +
>> +static const struct media_entity_operations vin_media_ops = {
>> + .link_setup = vin_link_setup,
>> + .link_validate = v4l2_subdev_link_validate,
>> +};
>> +
>> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev)
>> +{
>> + struct v4l2_subdev *sd;
>> + struct stfcamss_video *video_out;
>> + struct media_pad *pads;
>> + int ret;
>> + int i;
>> +
>> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
>> + char name[32];
>> + char *sub_name = vin_get_line_subdevname(i);
>> +
>> + sd = &vin_dev->line[i].subdev;
>> + pads = vin_dev->line[i].pads;
>> + video_out = &vin_dev->line[i].video_out;
>> + video_out->id = i;
>> +
>> + vin_init_outputs(&vin_dev->line[i]);
>> +
>> + v4l2_subdev_init(sd, &vin_v4l2_ops);
>> + sd->internal_ops = &vin_v4l2_internal_ops;
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d_%s",
>> + STF_VIN_NAME, 0, sub_name);
>> + v4l2_set_subdevdata(sd, &vin_dev->line[i]);
>> +
>> + ret = vin_init_formats(sd, NULL);
>> + if (ret < 0) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Failed to init format: %d\n", ret);
>> + goto err_init;
>> + }
>> +
>> + pads[STF_VIN_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> + pads[STF_VIN_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + sd->entity.function =
>> + MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>> + sd->entity.ops = &vin_media_ops;
>> + ret = media_entity_pads_init(&sd->entity,
>> + STF_VIN_PADS_NUM, pads);
>> + if (ret < 0) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Failed to init media entity: %d\n",
>> + ret);
>> + goto err_init;
>> + }
>> +
>> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
>> + if (ret < 0) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Failed to register subdev: %d\n", ret);
>> + goto err_reg_subdev;
>> + }
>> +
>> + video_out->ops = &stfcamss_vin_video_ops;
>> + video_out->bpl_alignment = 16 * 8;
>> +
>> + snprintf(name, ARRAY_SIZE(name), "%s_%s%d",
>> + sd->name, "video", i);
>> + ret = stf_video_register(video_out, v4l2_dev, name);
>> + if (ret < 0) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Failed to register video node: %d\n", ret);
>> + goto err_vid_reg;
>> + }
>> +
>> + ret = media_create_pad_link(
>> + &sd->entity, STF_VIN_PAD_SRC,
>> + &video_out->vdev.entity, 0,
>> + MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
>> + if (ret < 0) {
>> + dev_err(vin_dev->stfcamss->dev,
>> + "Failed to link %s->%s entities: %d\n",
>> + sd->entity.name, video_out->vdev.entity.name,
>> + ret);
>> + goto err_create_link;
>> + }
>> + }
>> +
>> + return 0;
>> +
>> +err_create_link:
>> + stf_video_unregister(video_out);
>> +err_vid_reg:
>> + v4l2_device_unregister_subdev(sd);
>> +err_reg_subdev:
>> + media_entity_cleanup(&sd->entity);
>> +err_init:
>> + for (i--; i >= 0; i--) {
>> + sd = &vin_dev->line[i].subdev;
>> + video_out = &vin_dev->line[i].video_out;
>> +
>> + stf_video_unregister(video_out);
>> + v4l2_device_unregister_subdev(sd);
>> + media_entity_cleanup(&sd->entity);
>> + }
>> + return ret;
>> +}
>> +
>> +int stf_vin_unregister(struct stf_vin_dev *vin_dev)
>> +{
>> + struct v4l2_subdev *sd;
>> + struct stfcamss_video *video_out;
>> + int i;
>> +
>> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++)
>> + mutex_destroy(&vin_dev->dummy_buffer[i].stream_lock);
>> +
>> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
>> + sd = &vin_dev->line[i].subdev;
>> + video_out = &vin_dev->line[i].video_out;
>> +
>> + stf_video_unregister(video_out);
>> + v4l2_device_unregister_subdev(sd);
>> + media_entity_cleanup(&sd->entity);
>> + mutex_destroy(&vin_dev->line[i].stream_lock);
>> + }
>> + return 0;
>> +}
>> diff --git a/drivers/media/platform/starfive/camss/stf_vin.h b/drivers/media/platform/starfive/camss/stf_vin.h
>> new file mode 100644
>> index 000000000000..28572eb6abe4
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_vin.h
>> @@ -0,0 +1,173 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * stf_vin.h
>> + *
>> + * StarFive Camera Subsystem - VIN Module
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#ifndef STF_VIN_H
>> +#define STF_VIN_H
>> +
>> +#include <linux/interrupt.h>
>> +#include <linux/spinlock_types.h>
>> +#include <media/v4l2-subdev.h>
>> +
>> +#include "stf_video.h"
>> +
>> +#define SYSCONSAIF_SYSCFG(x) (x)
>> +
>> +/* syscon offset 0 */
>> +#define U0_VIN_CNFG_AXI_DVP_EN BIT(2)
>> +
>> +/* syscon offset 20 */
>> +#define U0_VIN_CHANNEL_SEL_MASK GENMASK(3, 0)
>> +#define U0_VIN_AXIWR0_EN BIT(4)
>> +#define CHANNEL(x) ((x) << 0)
>> +
>> +/* syscon offset 32 */
>> +#define U0_VIN_INTR_CLEAN BIT(0)
>> +#define U0_VIN_INTR_M BIT(1)
>> +#define U0_VIN_PIX_CNT_END_MASK GENMASK(12, 2)
>> +#define U0_VIN_PIX_CT_MASK GENMASK(14, 13)
>> +#define U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS GENMASK(16, 15)
>> +
>> +#define PIX_CNT_END(x) ((x) << 2)
>> +#define PIX_CT(x) ((x) << 13)
>> +#define PIXEL_HEIGH_BIT_SEL(x) ((x) << 15)
>> +
>> +/* syscon offset 36 */
>> +#define U0_VIN_CNFG_DVP_HS_POS BIT(1)
>> +#define U0_VIN_CNFG_DVP_SWAP_EN BIT(2)
>> +#define U0_VIN_CNFG_DVP_VS_POS BIT(3)
>> +#define U0_VIN_CNFG_GEN_EN_AXIRD BIT(4)
>> +#define U0_VIN_CNFG_ISP_DVP_EN0 BIT(5)
>> +#define U0_VIN_MIPI_BYTE_EN_ISP0(n) ((n) << 6)
>> +#define U0_VIN_MIPI_CHANNEL_SEL0(n) ((n) << 8)
>> +#define U0_VIN_P_I_MIPI_HAEDER_EN0(n) ((n) << 12)
>> +#define U0_VIN_PIX_NUM(n) ((n) << 13)
>> +#define U0_VIN_MIPI_BYTE_EN_ISP0_MASK GENMASK(7, 6)
>> +#define U0_VIN_MIPI_CHANNEL_SEL0_MASK GENMASK(11, 8)
>> +#define U0_VIN_P_I_MIPI_HAEDER_EN0_MASK BIT(12)
>> +#define U0_VIN_PIX_NUM_MASK GENMASK(16, 13)
>> +
>> +#define STF_VIN_PAD_SINK 0
>> +#define STF_VIN_PAD_SRC 1
>> +#define STF_VIN_PADS_NUM 2
>> +
>> +#define ISP_DUMMY_BUFFER_NUMS STF_ISP_PAD_MAX
>> +#define VIN_DUMMY_BUFFER_NUMS 1
>> +
>> +enum {
>> + STF_DUMMY_VIN,
>> + STF_DUMMY_ISP,
>> + STF_DUMMY_MODULE_NUMS,
>> +};
>> +
>> +enum link {
>> + LINK_ERROR = -1,
>> + LINK_DVP_TO_WR,
>> + LINK_DVP_TO_ISP,
>> + LINK_CSI_TO_WR,
>> + LINK_CSI_TO_ISP,
>> +};
>> +
>> +struct vin_format {
>> + u32 code;
>> + u8 bpp;
>> +};
>> +
>> +struct vin_format_table {
>> + const struct vin_format *fmts;
>> + int nfmts;
>> +};
>> +
>> +enum vin_output_state {
>> + VIN_OUTPUT_OFF,
>> + VIN_OUTPUT_RESERVED,
>> + VIN_OUTPUT_SINGLE,
>> + VIN_OUTPUT_CONTINUOUS,
>> + VIN_OUTPUT_IDLE,
>> + VIN_OUTPUT_STOPPING
>> +};
>> +
>> +struct vin_output {
>> + int active_buf;
>> + struct stfcamss_buffer *buf[2];
>> + struct stfcamss_buffer *last_buffer;
>> + struct list_head pending_bufs;
>> + struct list_head ready_bufs;
>> + enum vin_output_state state;
>> + unsigned int sequence;
>> + unsigned int frame_skip;
>> +};
>> +
>> +/* The vin output lines */
>> +enum vin_line_id {
>> + VIN_LINE_NONE = -1,
>> + VIN_LINE_WR = 0,
>> + VIN_LINE_ISP,
>> + VIN_LINE_MAX,
>> +};
>> +
>> +struct vin_line {
>> + enum vin_line_id id;
>> + struct v4l2_subdev subdev;
>> + struct media_pad pads[STF_VIN_PADS_NUM];
>> + struct v4l2_mbus_framefmt fmt[STF_VIN_PADS_NUM];
>> + struct stfcamss_video video_out;
>> + struct mutex stream_lock; /* serialize stream control */
>> + int stream_count;
>> + struct vin_output output; /* pipeline and stream states */
>> + spinlock_t output_lock;
>> + const struct vin_format *formats;
>> + unsigned int nformats;
>> +};
>> +
>> +struct vin_dummy_buffer {
>> + dma_addr_t paddr[3];
>> + void *vaddr;
>> + u32 buffer_size;
>> + u32 width;
>> + u32 height;
>> + u32 mcode;
>> +};
>> +
>> +struct dummy_buffer {
>> + struct vin_dummy_buffer *buffer;
>> + u32 nums;
>> + struct mutex stream_lock; /* protects buffer data */
>> + int stream_count;
>> + atomic_t frame_skip;
>> +};
>> +
>> +struct vin_isr_ops {
>> + void (*isr_buffer_done)(struct vin_line *line);
>> + void (*isr_change_buffer)(struct vin_line *line);
>> +};
>> +
>> +struct stf_vin_dev {
>> + struct stfcamss *stfcamss;
>> + struct vin_line line[VIN_LINE_MAX];
>> + struct dummy_buffer dummy_buffer[STF_DUMMY_MODULE_NUMS];
>> + struct vin_isr_ops *isr_ops;
>> + atomic_t ref_count;
>> +};
>> +
>> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev);
>> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link);
>> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable);
>> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
>> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
>> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
>> + dma_addr_t y_addr, dma_addr_t uv_addr);
>> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv);
>> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv);
>> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv);
>> +int stf_vin_subdev_init(struct stfcamss *stfcamss);
>> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev);
>> +int stf_vin_unregister(struct stf_vin_dev *vin_dev);
>> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def);
>> +
>> +#endif /* STF_VIN_H */
>> diff --git a/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>> new file mode 100644
>> index 000000000000..7bd3265128d0
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>> @@ -0,0 +1,192 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_vin_hw_ops.c
>> + *
>> + * Register interface file for StarFive VIN module driver
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +#include "stf_camss.h"
>> +
>> +static void vin_intr_clear(struct stfcamss *stfcamss)
>> +{
>> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_CLEAN);
>> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_CLEAN);
>> +}
>> +
>> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv)
>> +{
>> + struct stf_vin_dev *vin_dev = priv;
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[STF_DUMMY_VIN];
>> +
>> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
>> + vin_dev->isr_ops->isr_change_buffer(&vin_dev->line[VIN_LINE_WR]);
>> + vin_dev->isr_ops->isr_buffer_done(&vin_dev->line[VIN_LINE_WR]);
>> + }
>> +
>> + vin_intr_clear(stfcamss);
>> +
>> + return IRQ_HANDLED;
>> +}
>> +
>> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv)
>> +{
>> + struct stf_vin_dev *vin_dev = priv;
>> + u32 int_status;
>> +
>> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
>> +
>> + if (int_status & ISPC_INTS) {
>> + if ((int_status & ISPC_ENUO))
>> + vin_dev->isr_ops->isr_buffer_done(
>> + &vin_dev->line[VIN_LINE_ISP]);
>> +
>> + /* clear interrupt */
>> + stf_isp_reg_write(vin_dev->stfcamss,
>> + ISP_REG_ISP_CTRL_0,
>> + (int_status & ~EN_INT_ALL) |
>> + EN_INT_ISP_DONE |
>> + EN_INT_CSI_DONE |
>> + EN_INT_SC_DONE);
>> + }
>> +
>> + return IRQ_HANDLED;
>> +}
>> +
>> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv)
>> +{
>> + struct stf_vin_dev *vin_dev = priv;
>> + struct stf_isp_dev *isp_dev;
>> + u32 int_status;
>> +
>> + isp_dev = &vin_dev->stfcamss->isp_dev;
>> +
>> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
>> + if (int_status & ISPC_SCFEINT) {
>> + struct dummy_buffer *dummy_buffer =
>> + &vin_dev->dummy_buffer[STF_DUMMY_ISP];
>> +
>> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
>> + if ((int_status & ISPC_ENUO))
>> + vin_dev->isr_ops->isr_change_buffer(
>> + &vin_dev->line[VIN_LINE_ISP]);
>> + }
>> +
>> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_CSIINTS,
>> + CSI_INTS_MASK, CSI_INTS(0x3));
>> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_IESHD,
>> + SHAD_UP_M | SHAD_UP_EN, 0x3);
>> +
>> + /* clear interrupt */
>> + stf_isp_reg_write(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0,
>> + (int_status & ~EN_INT_ALL) | EN_INT_LINE_INT);
>> + }
>> +
>> + return IRQ_HANDLED;
>> +}
>> +
>> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev)
>> +{
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> +
>> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(20),
>> + U0_VIN_AXIWR0_EN);
>> +
>> + return 0;
>> +}
>> +
>> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link)
>> +{
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> + u32 val;
>> +
>> + switch (link) {
>> + case LINK_CSI_TO_WR:
>> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(20));
>> + val &= ~U0_VIN_CHANNEL_SEL_MASK;
>> + val |= CHANNEL(0);
>> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(20), val);
>> +
>> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(28));
>> + val &= ~U0_VIN_PIX_CT_MASK;
>> + val |= PIX_CT(1);
>> +
>> + val &= ~U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS;
>> + val |= PIXEL_HEIGH_BIT_SEL(0);
>> +
>> + val &= ~U0_VIN_PIX_CNT_END_MASK;
>> + val |= PIX_CNT_END(IMAGE_MAX_WIDTH / 4 - 1);
>> +
>> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(28), val);
>> + break;
>> + case LINK_CSI_TO_ISP:
>> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(36));
>> + val &= ~U0_VIN_MIPI_BYTE_EN_ISP0_MASK;
>> + val |= U0_VIN_MIPI_BYTE_EN_ISP0(0);
>> +
>> + val &= ~U0_VIN_MIPI_CHANNEL_SEL0_MASK;
>> + val |= U0_VIN_MIPI_CHANNEL_SEL0(0);
>> +
>> + val &= ~U0_VIN_PIX_NUM_MASK;
>> + val |= U0_VIN_PIX_NUM(0);
>> +
>> + val &= ~U0_VIN_P_I_MIPI_HAEDER_EN0_MASK;
>> + val |= U0_VIN_P_I_MIPI_HAEDER_EN0(1);
>> +
>> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(36), val);
>> + break;
>> + case LINK_DVP_TO_WR:
>> + case LINK_DVP_TO_ISP:
>> + default:
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable)
>> +{
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> +
>> + if (enable) {
>> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_M);
>> + } else {
>> + /* clear vin interrupt */
>> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_CLEAN);
>> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_CLEAN);
>> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
>> + U0_VIN_INTR_M);
>> + }
>> +}
>> +
>> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
>> +{
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> +
>> + /* set the start address */
>> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(32), (long)addr);
>> +}
>> +
>> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
>> +{
>> + struct stfcamss *stfcamss = vin_dev->stfcamss;
>> +
>> + /* set the start address */
>> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(24), (long)addr);
>> +}
>> +
>> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
>> + dma_addr_t y_addr, dma_addr_t uv_addr)
>> +{
>> + stf_isp_reg_write(vin_dev->stfcamss,
>> + ISP_REG_Y_PLANE_START_ADDR, y_addr);
>> + stf_isp_reg_write(vin_dev->stfcamss,
>> + ISP_REG_UV_PLANE_START_ADDR, uv_addr);
>> +}
>
Regards,
Jack Zhu
Hi Laurent,
On 2023/7/28 4:41, Laurent Pinchart wrote:
> Hi Jack,
>
> Thank you for the patch.
>
> On Mon, Jun 19, 2023 at 07:28:37PM +0800, Jack Zhu wrote:
>> Add ISP driver for StarFive Camera Subsystem.
>>
>> Signed-off-by: Jack Zhu <[email protected]>
>> ---
>> .../media/platform/starfive/camss/Makefile | 2 +
>> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
>> .../media/platform/starfive/camss/stf_camss.h | 3 +
>> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
>> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
>> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
>> 6 files changed, 1544 insertions(+), 3 deletions(-)
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
>> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>>
>> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> index eb457917a914..cdf57e8c9546 100644
>> --- a/drivers/media/platform/starfive/camss/Makefile
>> +++ b/drivers/media/platform/starfive/camss/Makefile
>> @@ -5,6 +5,8 @@
>>
>> starfive-camss-objs += \
>> stf_camss.o \
>> + stf_isp.o \
>> + stf_isp_hw_ops.o \
>> stf_video.o
>>
>> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> index dc2b5dba7bd4..6f56b45f57db 100644
>> --- a/drivers/media/platform/starfive/camss/stf_camss.c
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> @@ -115,12 +115,65 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
>> return ret;
>> }
>>
>> +/*
>> + * stfcamss_init_subdevices - Initialize subdev structures and resources
>> + * @stfcamss: STFCAMSS device
>> + *
>> + * Return 0 on success or a negative error code on failure
>> + */
>> +static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
>> +{
>> + int ret;
>> +
>> + ret = stf_isp_subdev_init(stfcamss);
>> + if (ret < 0) {
>> + dev_err(stfcamss->dev, "Failed to init isp subdev: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
>> +{
>> + int ret;
>> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>> +
>> + ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
>> + if (ret < 0) {
>> + dev_err(stfcamss->dev,
>> + "Failed to register stf isp%d entity: %d\n", 0, ret);
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
>> +{
>> + stf_isp_unregister(&stfcamss->isp_dev);
>> +}
>> +
>> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
>> struct v4l2_subdev *subdev,
>> struct v4l2_async_subdev *asd)
>> {
>> + struct stfcamss *stfcamss =
>> + container_of(async, struct stfcamss, notifier);
>> + struct stfcamss_async_subdev *csd =
>> + container_of(asd, struct stfcamss_async_subdev, asd);
>> + enum stf_port_num port = csd->port;
>> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>> struct media_pad *pad[STF_PADS_NUM];
>> - unsigned int i, pad_num = 0;
>> + unsigned int i, pad_num;
>> +
>> + if (port == STF_PORT_CSI2RX) {
>> + pad[0] = &isp_dev->pads[STF_PAD_SINK];
>> + pad_num = 1;
>> + } else if (port == STF_PORT_DVP) {
>> + dev_err(stfcamss->dev, "Not support DVP sensor\n");
>> + return -EPERM;
>> + }
>>
>> for (i = 0; i < pad_num; ++i) {
>> int ret;
>> @@ -223,12 +276,18 @@ static int stfcamss_probe(struct platform_device *pdev)
>> goto err_cleanup_notifier;
>> }
>>
>> + ret = stfcamss_init_subdevices(stfcamss);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to init subdevice: %d\n", ret);
>> + goto err_cleanup_notifier;
>> + }
>> +
>> stfcamss_mc_init(pdev, stfcamss);
>>
>> ret = v4l2_device_register(stfcamss->dev, &stfcamss->v4l2_dev);
>> if (ret < 0) {
>> dev_err(dev, "Failed to register V4L2 device: %d\n", ret);
>> - goto err_cleanup_notifier;
>> + goto err_cleanup_media_device;
>> }
>>
>> ret = media_device_register(&stfcamss->media_dev);
>> @@ -237,6 +296,12 @@ static int stfcamss_probe(struct platform_device *pdev)
>> goto err_unregister_device;
>> }
>>
>> + ret = stfcamss_register_subdevices(stfcamss);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to register subdevice: %d\n", ret);
>> + goto err_unregister_media_dev;
>> + }
>> +
>> pm_runtime_enable(dev);
>>
>> stfcamss->notifier.ops = &stfcamss_subdev_notifier_ops;
>> @@ -244,15 +309,19 @@ static int stfcamss_probe(struct platform_device *pdev)
>> if (ret) {
>> dev_err(dev, "Failed to register async subdev nodes: %d\n",
>> ret);
>> - goto err_unregister_media_dev;
>> + goto err_unregister_subdevs;
>> }
>>
>> return 0;
>>
>> +err_unregister_subdevs:
>> + stfcamss_unregister_subdevices(stfcamss);
>> err_unregister_media_dev:
>> media_device_unregister(&stfcamss->media_dev);
>> err_unregister_device:
>> v4l2_device_unregister(&stfcamss->v4l2_dev);
>> +err_cleanup_media_device:
>> + media_device_cleanup(&stfcamss->media_dev);
>> err_cleanup_notifier:
>> v4l2_async_nf_cleanup(&stfcamss->notifier);
>> return ret;
>> @@ -268,6 +337,7 @@ static int stfcamss_remove(struct platform_device *pdev)
>> {
>> struct stfcamss *stfcamss = platform_get_drvdata(pdev);
>>
>> + stfcamss_unregister_subdevices(stfcamss);
>> v4l2_device_unregister(&stfcamss->v4l2_dev);
>> media_device_cleanup(&stfcamss->media_dev);
>> pm_runtime_disable(&pdev->dev);
>> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
>> index 15c4f34b9054..9482081288fa 100644
>> --- a/drivers/media/platform/starfive/camss/stf_camss.h
>> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
>> @@ -18,6 +18,8 @@
>> #include <media/v4l2-async.h>
>> #include <media/v4l2-device.h>
>>
>> +#include "stf_isp.h"
>> +
>> #define STF_DVP_NAME "stf_dvp"
>> #define STF_CSI_NAME "cdns_csi2rx"
>> #define STF_ISP_NAME "stf_isp"
>> @@ -65,6 +67,7 @@ struct stfcamss {
>> struct media_device media_dev;
>> struct media_pipeline pipe;
>> struct device *dev;
>> + struct stf_isp_dev isp_dev;
>> struct v4l2_async_notifier notifier;
>> void __iomem *syscon_base;
>> void __iomem *isp_base;
>> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
>> new file mode 100644
>> index 000000000000..933a583b398c
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
>> @@ -0,0 +1,519 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_isp.c
>> + *
>> + * StarFive Camera Subsystem - ISP Module
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +#include <linux/firmware.h>
>
> This doesn't seem needed.
>
>> +#include <media/v4l2-event.h>
>> +
>> +#include "stf_camss.h"
>> +
>> +#define SINK_FORMATS_INDEX 0
>> +#define UO_FORMATS_INDEX 1
>
> What does "UO" stand for ?
>
"UO" is Usual Out, just represents output. :-)
>> +
>> +static int isp_set_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel);
>> +
>> +static const struct isp_format isp_formats_sink[] = {
>> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
>> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
>> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
>> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
>> +};
>> +
>> +static const struct isp_format isp_formats_uo[] = {
>> + { MEDIA_BUS_FMT_Y12_1X12, 8 },
>
> Y12 is a greyscale format, I don't think that's what you need here.
>
>> +};
>> +
>> +static const struct isp_format_table isp_formats_st7110[] = {
>> + { isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
>> + { isp_formats_uo, ARRAY_SIZE(isp_formats_uo) },
>> +};
>> +
>> +int stf_isp_subdev_init(struct stfcamss *stfcamss)
>
> This function doesn't initialize the subdev, I'd call it stf_isp_init().
> I would also create a stf_isp_cleanup() function to be consistent, and
> move the mutex_destroy() call there.
>
>> +{
>> + struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>> +
>> + isp_dev->stfcamss = stfcamss;
>> + isp_dev->formats = isp_formats_st7110;
>> + isp_dev->nformats = ARRAY_SIZE(isp_formats_st7110);
>> +
>> + mutex_init(&isp_dev->stream_lock);
>> + return 0;
>> +}
>> +
>> +static struct v4l2_mbus_framefmt *
>> +__isp_get_format(struct stf_isp_dev *isp_dev,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + if (which == V4L2_SUBDEV_FORMAT_TRY)
>> + return v4l2_subdev_get_try_format(&isp_dev->subdev, state, pad);
>> +
>> + return &isp_dev->fmt[pad];
>
> Use the subdev active state API to store the active format on subdev
> pads, it will simplify the driver. See commit a2514b9a634a ("media: i2c:
> imx290: Use V4L2 subdev active state") for an example of a subdev driver
> being converted to that API, it should help understanding how to use it.
> You will be able to drop the stf_isp_dev fmt and rect fields. For the
> bpp value stored in the rect structure, replace it with a const struct
> isp_format pointer in stf_isp_dev that you set in .set_fmt() (as well as
> initialization time) to the active format information.
>
>> +}
>> +
>> +static int isp_set_stream(struct v4l2_subdev *sd, int enable)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + int ret = 0;
>> + struct v4l2_mbus_framefmt *fmt;
>> + struct v4l2_event src_ch = { 0 };
>> +
>> + fmt = __isp_get_format(isp_dev, NULL, STF_ISP_PAD_SINK,
>> + V4L2_SUBDEV_FORMAT_ACTIVE);
>> + mutex_lock(&isp_dev->stream_lock);
>> + if (enable) {
>> + if (isp_dev->stream_count == 0) {
>> + stf_isp_clk_enable(isp_dev);
>> + stf_isp_reset(isp_dev);
>> + stf_isp_init_cfg(isp_dev);
>> + stf_isp_settings(isp_dev, isp_dev->rect, fmt->code);
>> + stf_isp_stream_set(isp_dev);
>> + }
>> + isp_dev->stream_count++;
>
> The subdev .s_stream() operation isn't supposed to be called multiple
> times. If you need to count how many video nodes have started streaming,
> that should be handled in the vb2 queue .start_streaming() and
> .stop_streaming() operations.
>
>> + } else {
>> + if (isp_dev->stream_count == 0)
>> + goto exit;
>> +
>> + if (isp_dev->stream_count == 1)
>> + stf_isp_clk_disable(isp_dev);
>> +
>> + isp_dev->stream_count--;
>> + }
>> + src_ch.type = V4L2_EVENT_SOURCE_CHANGE,
>
> ; instead of , at the end of the line. Same for the next line.
>
>> + src_ch.u.src_change.changes = isp_dev->stream_count,
>> +
>> + v4l2_subdev_notify_event(sd, &src_ch);
>
> This doesn't seem right. V4L2_EVENT_SOURCE_CHANGE is meant to report
> changes to the source format. I would drop support for events
> completely from this patch, it doesn't seem to be needed.
>
>> +exit:
>> + mutex_unlock(&isp_dev->stream_lock);
>> +
>> + return ret;
>
> ret is never set to a value other than 9, return 0 and drop the
> variable.
>
>> +}
>> +
>> +static void isp_try_format(struct stf_isp_dev *isp_dev,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + struct v4l2_mbus_framefmt *fmt,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + const struct isp_format_table *formats;
>> + struct stf_isp_crop *rect;
>> + unsigned int i;
>> +
>> + if (pad == STF_ISP_PAD_SINK) {
>> + /* Set format on sink pad */
>> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
>> + rect = &isp_dev->rect[SINK_FORMATS_INDEX];
>> + } else if (pad == STF_ISP_PAD_SRC) {
>> + formats = &isp_dev->formats[UO_FORMATS_INDEX];
>> + rect = &isp_dev->rect[UO_FORMATS_INDEX];
>> + }
>> +
>> + fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
>> + STFCAMSS_FRAME_MAX_WIDTH);
>> + fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
>> + STFCAMSS_FRAME_MAX_HEIGHT);
>> + fmt->height &= ~0x1;
>> + fmt->field = V4L2_FIELD_NONE;
>> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> + fmt->flags = 0;
>> +
>> + for (i = 0; i < formats->nfmts; i++) {
>> + if (fmt->code == formats->fmts[i].code)
>> + break;
>> + }
>> +
>> + if (i >= formats->nfmts) {
>> + fmt->code = formats->fmts[0].code;
>> + rect->bpp = formats->fmts[0].bpp;
>> + } else {
>> + rect->bpp = formats->fmts[i].bpp;
>> + }
>> +}
>> +
>> +static int isp_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + const struct isp_format_table *formats;
>> +
>> + if (code->index >= isp_dev->nformats)
>
> That doesn't seem right, isp_dev->nformats isn't the number of formats
> supported on the ISP, but the number of format types (sink and uo).
>
>> + return -EINVAL;
>
> Add a blank line here, it makes code easier to read.
>
>> + if (code->pad == STF_ISP_PAD_SINK) {
>> + formats = &isp_dev->formats[SINK_FORMATS_INDEX];
>> + code->code = formats->fmts[code->index].code;
>> + } else {
>> + struct v4l2_mbus_framefmt *sink_fmt;
>> +
>> + sink_fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK,
>> + code->which);
>> +
>> + code->code = sink_fmt->code;
>
> This doesn't seem right, you need to enumerate the ISP output media bus
> codes here, and the function returns the sink media bus code instead.
>
>> + if (!code->code)
>> + return -EINVAL;
>> + }
>> + code->flags = 0;
>> +
>> + return 0;
>> +}
>> +
>> +static int isp_enum_frame_size(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt format;
>> +
>> + if (fse->index != 0)
>> + return -EINVAL;
>> +
>> + format.code = fse->code;
>> + format.width = 1;
>> + format.height = 1;
>> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
>> + fse->min_width = format.width;
>> + fse->min_height = format.height;
>> +
>> + if (format.code != fse->code)
>> + return -EINVAL;
>> +
>> + format.code = fse->code;
>> + format.width = -1;
>> + format.height = -1;
>> + isp_try_format(isp_dev, state, fse->pad, &format, fse->which);
>> + fse->max_width = format.width;
>> + fse->max_height = format.height;
>
> This seems unnecessarily complicated, you can just set the min and max
> width and height to STFCAMSS_FRAME_MIN_WIDTH, STFCAMSS_FRAME_MAX_WIDTH,
> STFCAMSS_FRAME_MIN_HEIGHT and STFCAMSS_FRAME_MAX_HEIGHT here.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int isp_get_format(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt *format;
>> +
>> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
>> + if (!format)
>> + return -EINVAL;
>> +
>> + fmt->format = *format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isp_set_format(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + struct v4l2_mbus_framefmt *format;
>> +
>> + format = __isp_get_format(isp_dev, state, fmt->pad, fmt->which);
>> + if (!format)
>> + return -EINVAL;
>> +
>> + mutex_lock(&isp_dev->stream_lock);
>> +
>> + isp_try_format(isp_dev, state, fmt->pad, &fmt->format, fmt->which);
>> + *format = fmt->format;
>> +
>> + mutex_unlock(&isp_dev->stream_lock);
>> +
>> + /* Propagate to in crop */
>> + if (fmt->pad == STF_ISP_PAD_SINK) {
>> + struct v4l2_subdev_selection sel = { 0 };
>> + int ret;
>> +
>> + /* Reset sink pad compose selection */
>> + sel.which = fmt->which;
>> + sel.pad = STF_ISP_PAD_SINK;
>> + sel.target = V4L2_SEL_TGT_CROP;
>> + sel.r.width = fmt->format.width;
>> + sel.r.height = fmt->format.height;
>> + ret = isp_set_selection(sd, state, &sel);
>> + if (ret < 0)
>> + return ret;
>
> You're setting the which, pad and target values manually, so they're
> guaranteed to be correct. isp_set_selection() will thus never return an
> error. You can drop the error check.
>
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static struct v4l2_rect *
>> +__isp_get_crop(struct stf_isp_dev *isp_dev,
>> + struct v4l2_subdev_state *state,
>> + unsigned int pad,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + if (which == V4L2_SUBDEV_FORMAT_TRY)
>> + return v4l2_subdev_get_try_crop(&isp_dev->subdev, state,
>> + STF_ISP_PAD_SINK);
>> +
>> + return &isp_dev->rect[pad].rect;
>> +}
>> +
>> +static void isp_try_crop(struct stf_isp_dev *isp_dev,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_rect *rect,
>> + enum v4l2_subdev_format_whence which)
>> +{
>> + struct v4l2_mbus_framefmt *fmt;
>> +
>> + fmt = __isp_get_format(isp_dev, state, STF_ISP_PAD_SINK, which);
>> +
>> + if (rect->width > fmt->width)
>> + rect->width = fmt->width;
>> +
>> + if (rect->width + rect->left > fmt->width)
>> + rect->left = fmt->width - rect->width;
>> +
>> + if (rect->height > fmt->height)
>> + rect->height = fmt->height;
>> +
>> + if (rect->height + rect->top > fmt->height)
>> + rect->top = fmt->height - rect->height;
>> +
>> + if (rect->width < STFCAMSS_FRAME_MIN_WIDTH) {
>> + rect->left = 0;
>> + rect->width = STFCAMSS_FRAME_MAX_WIDTH;
>> + }
>> +
>> + if (rect->height < STFCAMSS_FRAME_MIN_HEIGHT) {
>> + rect->top = 0;
>> + rect->height = STFCAMSS_FRAME_MAX_HEIGHT;
>> + }
>> + rect->height &= ~0x1;
>
> The v4l2_rect_set_min_size() and v4l2_rect_map_inside() functions can
> simplify this code.
>
>> +}
>> +
>> +static int isp_get_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + struct v4l2_subdev_format fmt = { 0 };
>> + struct v4l2_rect *rect;
>> + int ret;
>> +
>> + switch (sel->target) {
>> + case V4L2_SEL_TGT_CROP_BOUNDS:
>> + if (sel->pad == STF_ISP_PAD_SINK) {
>> + fmt.pad = sel->pad;
>> + fmt.which = sel->which;
>> + ret = isp_get_format(sd, state, &fmt);
>> + if (ret < 0)
>> + return ret;
>> +
>> + sel->r.left = 0;
>> + sel->r.top = 0;
>> + sel->r.width = fmt.format.width;
>> + sel->r.height = fmt.format.height;
>> + } else if (sel->pad == STF_ISP_PAD_SRC) {
>> + rect = __isp_get_crop(isp_dev, state,
>> + sel->pad, sel->which);
>> + sel->r = *rect;
>> + }
>> + break;
>> +
>> + case V4L2_SEL_TGT_CROP:
>> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
>> + if (!rect)
>> + return -EINVAL;
>> +
>> + sel->r = *rect;
>> + break;
>> +
>> + default:
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int isp_set_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>> + struct v4l2_rect *rect;
>> + int ret = 0;
>> +
>
> if (sel->target != V4L2_SEL_TGT_CROP)
> return -EINVAL;
>
>> + if (sel->target == V4L2_SEL_TGT_CROP &&
>> + sel->pad == STF_ISP_PAD_SINK) {
>> + struct v4l2_subdev_selection crop = { 0 };
>> +
>> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
>> + if (!rect)
>> + return -EINVAL;
>> +
>> + mutex_lock(&isp_dev->stream_lock);
>> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
>> + *rect = sel->r;
>> + mutex_unlock(&isp_dev->stream_lock);
>> +
>> + /* Reset source crop selection */
>> + crop.which = sel->which;
>> + crop.pad = STF_ISP_PAD_SRC;
>> + crop.target = V4L2_SEL_TGT_CROP;
>> + crop.r = *rect;
>> + ret = isp_set_selection(sd, state, &crop);
>> + } else if (sel->target == V4L2_SEL_TGT_CROP &&
>> + sel->pad == STF_ISP_PAD_SRC) {
>> + struct v4l2_subdev_format fmt = { 0 };
>> +
>> + rect = __isp_get_crop(isp_dev, state, sel->pad, sel->which);
>> + if (!rect)
>> + return -EINVAL;
>> +
>> + mutex_lock(&isp_dev->stream_lock);
>> + isp_try_crop(isp_dev, state, &sel->r, sel->which);
>> + *rect = sel->r;
>> + mutex_unlock(&isp_dev->stream_lock);
>> +
>> + /* Reset source pad format width and height */
>> + fmt.which = sel->which;
>> + fmt.pad = STF_ISP_PAD_SRC;
>> + fmt.format.width = rect->width;
>> + fmt.format.height = rect->height;
>> + ret = isp_set_format(sd, state, &fmt);
>> + if (ret < 0)
>> + return ret;
>
> You can drop the error check here too.
>
>> + }
>> +
>> + dev_dbg(isp_dev->stfcamss->dev, "pad: %d sel(%d,%d)/%dx%d\n",
>> + sel->pad, sel->r.left, sel->r.top, sel->r.width, sel->r.height);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp_init_formats(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_fh *fh)
>> +{
>> + struct v4l2_subdev_format format = {
>> + .pad = STF_ISP_PAD_SINK,
>> + .which =
>> + fh ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
>> + .format = {
>> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
>
> Pick a format that the driver supports.
>
>> + .width = 1920,
>> + .height = 1080
>> + }
>> + };
>> +
>> + return isp_set_format(sd, fh ? fh->state : NULL, &format);
>> +}
>> +
>> +static int isp_link_setup(struct media_entity *entity,
>> + const struct media_pad *local,
>> + const struct media_pad *remote, u32 flags)
>> +{
>> + if (flags & MEDIA_LNK_FL_ENABLED)
>> + if (media_pad_remote_pad_first(local))
>> + return -EBUSY;
>> + return 0;
>
> Is this check really needed ?
>
>> +}
>> +
>> +static int stf_isp_subscribe_event(struct v4l2_subdev *sd,
>> + struct v4l2_fh *fh,
>> + struct v4l2_event_subscription *sub)
>> +{
>> + switch (sub->type) {
>> + case V4L2_EVENT_SOURCE_CHANGE:
>> + return v4l2_src_change_event_subdev_subscribe(sd, fh, sub);
>> + default:
>> + return -EINVAL;
>> + }
>> +}
>> +
>> +static const struct v4l2_subdev_core_ops isp_core_ops = {
>> + .subscribe_event = stf_isp_subscribe_event,
>> + .unsubscribe_event = v4l2_event_subdev_unsubscribe,
>> +};
>
> Drop events support.
>
>> +
>> +static const struct v4l2_subdev_video_ops isp_video_ops = {
>> + .s_stream = isp_set_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_pad_ops isp_pad_ops = {
>> + .enum_mbus_code = isp_enum_mbus_code,
>> + .enum_frame_size = isp_enum_frame_size,
>> + .get_fmt = isp_get_format,
>> + .set_fmt = isp_set_format,
>> + .get_selection = isp_get_selection,
>> + .set_selection = isp_set_selection,
>> +};
>> +
>> +static const struct v4l2_subdev_ops isp_v4l2_ops = {
>> + .core = &isp_core_ops,
>> + .video = &isp_video_ops,
>> + .pad = &isp_pad_ops,
>> +};
>> +
>> +static const struct v4l2_subdev_internal_ops isp_v4l2_internal_ops = {
>> + .open = isp_init_formats,
>> +};
>
> Replace this with the .init_cfg() pad operation.
>
>> +
>> +static const struct media_entity_operations isp_media_ops = {
>> + .link_setup = isp_link_setup,
>> + .link_validate = v4l2_subdev_link_validate,
>> +};
>> +
>> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
>> +{
>> + struct v4l2_subdev *sd = &isp_dev->subdev;
>> + struct media_pad *pads = isp_dev->pads;
>> + int ret;
>> +
>> + v4l2_subdev_init(sd, &isp_v4l2_ops);
>> + sd->internal_ops = &isp_v4l2_internal_ops;
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
>
> Drop events here too.
>
>> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d", STF_ISP_NAME, 0);
>> + v4l2_set_subdevdata(sd, isp_dev);
>> +
>> + ret = isp_init_formats(sd, NULL);
>> + if (ret < 0) {
>> + dev_err(isp_dev->stfcamss->dev, "Failed to init format: %d\n",
>> + ret);
>> + return ret;
>> + }
>> +
>> + pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> + pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>
> MEDIA_ENT_F_PROC_VIDEO_ISP seems a better match.
>
>> + sd->entity.ops = &isp_media_ops;
>> + ret = media_entity_pads_init(&sd->entity, STF_ISP_PAD_MAX, pads);
>> + if (ret < 0) {
>> + dev_err(isp_dev->stfcamss->dev,
>> + "Failed to init media entity: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
>> + if (ret < 0) {
>> + dev_err(isp_dev->stfcamss->dev,
>> + "Failed to register subdev: %d\n", ret);
>> + goto err_sreg;
>> + }
>> +
>> + return 0;
>> +
>> +err_sreg:
>> + media_entity_cleanup(&sd->entity);
>> + return ret;
>> +}
>> +
>> +int stf_isp_unregister(struct stf_isp_dev *isp_dev)
>> +{
>> + v4l2_device_unregister_subdev(&isp_dev->subdev);
>> + media_entity_cleanup(&isp_dev->subdev.entity);
>> + mutex_destroy(&isp_dev->stream_lock);
>> + return 0;
>> +}
>> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
>> new file mode 100644
>> index 000000000000..1e5c98482350
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
>> @@ -0,0 +1,479 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * stf_isp.h
>> + *
>> + * StarFive Camera Subsystem - ISP Module
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + */
>> +
>> +#ifndef STF_ISP_H
>> +#define STF_ISP_H
>> +
>> +#include <media/v4l2-subdev.h>
>> +
>> +#include "stf_video.h"
>> +
>> +#define ISP_RAW_DATA_BITS 12
>> +#define SCALER_RATIO_MAX 1
>> +#define STF_ISP_REG_OFFSET_MAX 0x0fff
>> +#define STF_ISP_REG_DELAY_MAX 100
>> +
>> +/* isp registers */
>> +#define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
>> +#define CSI_SCD_ERR BIT(6)
>> +#define CSI_ITU656_ERR BIT(4)
>> +#define CSI_ITU656_F BIT(3)
>> +#define CSI_SCD_DONE BIT(2)
>> +#define CSI_BUSY_S BIT(1)
>> +#define CSI_EN_S BIT(0)
>
> Could you align the macros values consistently in the whole file ? It
> would be easier to read. For the lines above, that would be
>
> #define ISP_RAW_DATA_BITS 12
> #define SCALER_RATIO_MAX 1
> #define STF_ISP_REG_OFFSET_MAX 0x0fff
> #define STF_ISP_REG_DELAY_MAX 100
>
> /* isp registers */
> #define ISP_REG_CSI_INPUT_EN_AND_STATUS 0x000
> #define CSI_SCD_ERR BIT(6)
> #define CSI_ITU656_ERR BIT(4)
> #define CSI_ITU656_F BIT(3)
> #define CSI_SCD_DONE BIT(2)
> #define CSI_BUSY_S BIT(1)
> #define CSI_EN_S BIT(0)
>
> (possibly with additional indentation if some of the lines below require
> that)
>
> You could also move the register definitions to a stf_isp_regs.h header
> if desired, as they're only needed by stf_isp_hw_ops.c, not by the other
> files that include stf_isp.h. up to you.
>
>> +
>> +#define ISP_REG_CSIINTS 0x008
>> +#define CSI_INTS(n) ((n) << 16)
>> +#define CSI_SHA_M(n) ((n) << 0)
>> +#define CSI_INTS_MASK GENMASK(17, 16)
>> +
>> +#define ISP_REG_CSI_MODULE_CFG 0x010
>> +#define CSI_DUMP_EN BIT(19)
>> +#define CSI_VS_EN BIT(18)
>> +#define CSI_SC_EN BIT(17)
>> +#define CSI_OBA_EN BIT(16)
>> +#define CSI_AWB_EN BIT(7)
>> +#define CSI_LCCF_EN BIT(6)
>> +#define CSI_OECFHM_EN BIT(5)
>> +#define CSI_OECF_EN BIT(4)
>> +#define CSI_LCBQ_EN BIT(3)
>> +#define CSI_OBC_EN BIT(2)
>> +#define CSI_DEC_EN BIT(1)
>> +#define CSI_DC_EN BIT(0)
>> +
>> +#define ISP_REG_SENSOR 0x014
>> +#define DVP_SYNC_POL(n) ((n) << 2)
>> +#define ITU656_EN(n) ((n) << 1)
>> +#define IMAGER_SEL(n) ((n) << 0)
>> +
>> +#define ISP_REG_RAW_FORMAT_CFG 0x018
>> +#define SMY13(n) ((n) << 14)
>> +#define SMY12(n) ((n) << 12)
>> +#define SMY11(n) ((n) << 10)
>> +#define SMY10(n) ((n) << 8)
>> +#define SMY3(n) ((n) << 6)
>> +#define SMY2(n) ((n) << 4)
>> +#define SMY1(n) ((n) << 2)
>> +#define SMY0(n) ((n) << 0)
>> +
>> +#define ISP_REG_PIC_CAPTURE_START_CFG 0x01c
>> +#define VSTART_CAP(n) ((n) << 16)
>> +#define HSTART_CAP(n) ((n) << 0)
>> +
>> +#define ISP_REG_PIC_CAPTURE_END_CFG 0x020
>> +#define VEND_CAP(n) ((n) << 16)
>> +#define HEND_CAP(n) ((n) << 0)
>> +
>> +#define ISP_REG_DUMP_CFG_0 0x024
>> +#define ISP_REG_DUMP_CFG_1 0x028
>> +#define DUMP_ID(n) ((n) << 24)
>> +#define DUMP_SHT(n) ((n) << 20)
>> +#define DUMP_BURST_LEN(n) ((n) << 16)
>> +#define DUMP_SD(n) ((n) << 0)
>> +#define DUMP_BURST_LEN_MASK GENMASK(17, 16)
>> +#define DUMP_SD_MASK GENMASK(15, 0)
>> +
>> +#define ISP_REG_DEC_CFG 0x030
>> +#define DEC_V_KEEP(n) ((n) << 24)
>> +#define DEC_V_PERIOD(n) ((n) << 16)
>> +#define DEC_H_KEEP(n) ((n) << 8)
>> +#define DEC_H_PERIOD(n) ((n) << 0)
>> +
>> +#define ISP_REG_OBC_CFG 0x034
>> +#define OBC_W_H(y) ((y) << 4)
>> +#define OBC_W_W(x) ((x) << 0)
>> +
>> +#define ISP_REG_DC_CFG_1 0x044
>> +#define DC_AXI_ID(n) ((n) << 0)
>> +
>> +#define ISP_REG_LCCF_CFG_0 0x050
>> +#define Y_DISTANCE(y) ((y) << 16)
>> +#define X_DISTANCE(x) ((x) << 0)
>> +
>> +#define ISP_REG_LCCF_CFG_1 0x058
>> +#define LCCF_MAX_DIS(n) ((n) << 0)
>> +
>> +#define ISP_REG_LCBQ_CFG_0 0x074
>> +#define H_LCBQ(y) ((y) << 12)
>> +#define W_LCBQ(x) ((x) << 8)
>> +
>> +#define ISP_REG_LCBQ_CFG_1 0x07c
>> +#define Y_COOR(y) ((y) << 16)
>> +#define X_COOR(x) ((x) << 0)
>> +
>> +#define ISP_REG_LCCF_CFG_2 0x0e0
>> +#define ISP_REG_LCCF_CFG_3 0x0e4
>> +#define ISP_REG_LCCF_CFG_4 0x0e8
>> +#define ISP_REG_LCCF_CFG_5 0x0ec
>> +#define LCCF_F2_PAR(n) ((n) << 16)
>> +#define LCCF_F1_PAR(n) ((n) << 0)
>> +
>> +#define ISP_REG_OECF_X0_CFG0 0x100
>> +#define ISP_REG_OECF_X0_CFG1 0x104
>> +#define ISP_REG_OECF_X0_CFG2 0x108
>> +#define ISP_REG_OECF_X0_CFG3 0x10c
>> +#define ISP_REG_OECF_X0_CFG4 0x110
>> +#define ISP_REG_OECF_X0_CFG5 0x114
>> +#define ISP_REG_OECF_X0_CFG6 0x118
>> +#define ISP_REG_OECF_X0_CFG7 0x11c
>> +
>> +#define ISP_REG_OECF_Y3_CFG0 0x1e0
>> +#define ISP_REG_OECF_Y3_CFG1 0x1e4
>> +#define ISP_REG_OECF_Y3_CFG2 0x1e8
>> +#define ISP_REG_OECF_Y3_CFG3 0x1ec
>> +#define ISP_REG_OECF_Y3_CFG4 0x1f0
>> +#define ISP_REG_OECF_Y3_CFG5 0x1f4
>> +#define ISP_REG_OECF_Y3_CFG6 0x1f8
>> +#define ISP_REG_OECF_Y3_CFG7 0x1fc
>> +
>> +#define ISP_REG_OECF_S0_CFG0 0x200
>> +#define ISP_REG_OECF_S3_CFG7 0x27c
>> +#define OCEF_PAR_H(n) ((n) << 16)
>> +#define OCEF_PAR_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_AWB_X0_CFG_0 0x280
>> +#define ISP_REG_AWB_X0_CFG_1 0x284
>> +#define ISP_REG_AWB_X1_CFG_0 0x288
>> +#define ISP_REG_AWB_X1_CFG_1 0x28c
>> +#define ISP_REG_AWB_X2_CFG_0 0x290
>> +#define ISP_REG_AWB_X2_CFG_1 0x294
>> +#define ISP_REG_AWB_X3_CFG_0 0x298
>> +#define ISP_REG_AWB_X3_CFG_1 0x29c
>> +#define AWB_X_SYMBOL_H(n) ((n) << 16)
>> +#define AWB_X_SYMBOL_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_AWB_Y0_CFG_0 0x2a0
>> +#define ISP_REG_AWB_Y0_CFG_1 0x2a4
>> +#define ISP_REG_AWB_Y1_CFG_0 0x2a8
>> +#define ISP_REG_AWB_Y1_CFG_1 0x2ac
>> +#define ISP_REG_AWB_Y2_CFG_0 0x2b0
>> +#define ISP_REG_AWB_Y2_CFG_1 0x2b4
>> +#define ISP_REG_AWB_Y3_CFG_0 0x2b8
>> +#define ISP_REG_AWB_Y3_CFG_1 0x2bc
>> +#define AWB_Y_SYMBOL_H(n) ((n) << 16)
>> +#define AWB_Y_SYMBOL_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_AWB_S0_CFG_0 0x2c0
>> +#define ISP_REG_AWB_S0_CFG_1 0x2c4
>> +#define ISP_REG_AWB_S1_CFG_0 0x2c8
>> +#define ISP_REG_AWB_S1_CFG_1 0x2cc
>> +#define ISP_REG_AWB_S2_CFG_0 0x2d0
>> +#define ISP_REG_AWB_S2_CFG_1 0x2d4
>> +#define ISP_REG_AWB_S3_CFG_0 0x2d8
>> +#define ISP_REG_AWB_S3_CFG_1 0x2dc
>> +#define AWB_S_SYMBOL_H(n) ((n) << 16)
>> +#define AWB_S_SYMBOL_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_OBCG_CFG_0 0x2e0
>> +#define ISP_REG_OBCG_CFG_1 0x2e4
>> +#define ISP_REG_OBCG_CFG_2 0x2e8
>> +#define ISP_REG_OBCG_CFG_3 0x2ec
>> +#define ISP_REG_OBCO_CFG_0 0x2f0
>> +#define ISP_REG_OBCO_CFG_1 0x2f4
>> +#define ISP_REG_OBCO_CFG_2 0x2f8
>> +#define ISP_REG_OBCO_CFG_3 0x2fc
>> +#define GAIN_D_POINT(x) ((x) << 24)
>> +#define GAIN_C_POINT(x) ((x) << 16)
>> +#define GAIN_B_POINT(x) ((x) << 8)
>> +#define GAIN_A_POINT(x) ((x) << 0)
>> +#define OFFSET_D_POINT(x) ((x) << 24)
>> +#define OFFSET_C_POINT(x) ((x) << 16)
>> +#define OFFSET_B_POINT(x) ((x) << 8)
>> +#define OFFSET_A_POINT(x) ((x) << 0)
>> +
>> +#define ISP_REG_ISP_CTRL_0 0xa00
>> +#define ISPC_SCFEINT BIT(27)
>> +#define ISPC_VSFWINT BIT(26)
>> +#define ISPC_VSINT BIT(25)
>> +#define ISPC_INTS BIT(24)
>> +#define ISPC_ENUO BIT(20)
>> +#define ISPC_ENLS BIT(17)
>> +#define ISPC_ENSS1 BIT(12)
>> +#define ISPC_ENSS0 BIT(11)
>> +#define ISPC_RST BIT(1)
>> +#define ISPC_EN BIT(0)
>> +#define ISPC_RST_MASK BIT(1)
>> +
>> +#define ISP_REG_ISP_CTRL_1 0xa08
>> +#define CTRL_SAT(n) ((n) << 28)
>> +#define CTRL_DBC BIT(22)
>> +#define CTRL_CTC BIT(21)
>> +#define CTRL_YHIST BIT(20)
>> +#define CTRL_YCURVE BIT(19)
>> +#define CTRL_CTM BIT(18)
>> +#define CTRL_BIYUV BIT(17)
>> +#define CTRL_SCE BIT(8)
>> +#define CTRL_EE BIT(7)
>> +#define CTRL_CCE BIT(5)
>> +#define CTRL_RGE BIT(4)
>> +#define CTRL_CME BIT(3)
>> +#define CTRL_AE BIT(2)
>> +#define CTRL_CE BIT(1)
>> +#define CTRL_SAT_MASK GENMASK(31, 28)
>> +
>> +#define ISP_REG_PIPELINE_XY_SIZE 0xa0c
>> +#define H_ACT_CAP(n) ((n) << 16)
>> +#define W_ACT_CAP(n) ((n) << 0)
>> +
>> +#define ISP_REG_ICTC 0xa10
>> +#define GF_MODE(n) ((n) << 30)
>> +#define MAXGT(n) ((n) << 16)
>> +#define MINGT(n) ((n) << 0)
>> +
>> +#define ISP_REG_IDBC 0xa14
>> +#define BADGT(n) ((n) << 16)
>> +#define BADXT(n) ((n) << 0)
>> +
>> +#define ISP_REG_ICFAM 0xa1c
>> +#define CROSS_COV(n) ((n) << 4)
>> +#define HV_W(n) ((n) << 0)
>> +
>> +#define ISP_REG_CS_GAIN 0xa30
>> +#define CMAD(n) ((n) << 16)
>> +#define CMAB(n) ((n) << 0)
>> +
>> +#define ISP_REG_CS_THRESHOLD 0xa34
>> +#define CMD(n) ((n) << 16)
>> +#define CMB(n) ((n) << 0)
>> +
>> +#define ISP_REG_CS_OFFSET 0xa38
>> +#define VOFF(n) ((n) << 16)
>> +#define UOFF(n) ((n) << 0)
>> +
>> +#define ISP_REG_CS_HUE_F 0xa3c
>> +#define SIN(n) ((n) << 16)
>> +#define COS(n) ((n) << 0)
>> +
>> +#define ISP_REG_CS_SCALE 0xa40
>> +#define CMSF(n) ((n) << 0)
>> +
>> +#define ISP_REG_IESHD 0xa50
>> +#define SHAD_UP_M BIT(1)
>> +#define SHAD_UP_EN BIT(0)
>> +
>> +#define ISP_REG_YADJ0 0xa54
>> +#define YOIR(n) ((n) << 16)
>> +#define YIMIN(n) ((n) << 0)
>> +
>> +#define ISP_REG_YADJ1 0xa58
>> +#define YOMAX(n) ((n) << 16)
>> +#define YOMIN(n) ((n) << 0)
>> +
>> +#define ISP_REG_Y_PLANE_START_ADDR 0xa80
>> +#define ISP_REG_UV_PLANE_START_ADDR 0xa84
>> +
>> +#define ISP_REG_STRIDE 0xa88
>> +#define IMG_STR(n) ((n) << 0)
>
> For registers that contain a single field at offset 0, you don't need to
> define a macro, you can pass the value directly to the register write
> function (IMG_STR is actually not used :-)). Same above and/or below
> where applicable.
>
>> +
>> +#define ISP_REG_ITIIWSR 0xb20
>> +#define ITI_HSIZE(n) ((n) << 16)
>> +#define ITI_WSIZE(n) ((n) << 0)
>> +
>> +#define ISP_REG_ITIDWLSR 0xb24
>> +#define ITI_WSTRIDE(n) ((n) << 0)
>> +
>> +#define ISP_REG_ITIPDFR 0xb38
>> +#define ITI_PACKAGE_FMT(n) ((n) << 0)
>> +
>> +#define ISP_REG_ITIDRLSR 0xb3C
>> +#define ITI_STRIDE_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_DNYUV_YSWR0 0xc00
>> +#define ISP_REG_DNYUV_YSWR1 0xc04
>> +#define ISP_REG_DNYUV_CSWR0 0xc08
>> +#define ISP_REG_DNYUV_CSWR1 0xc0c
>> +#define YUVSW5(n) ((n) << 20)
>> +#define YUVSW4(n) ((n) << 16)
>> +#define YUVSW3(n) ((n) << 12)
>> +#define YUVSW2(n) ((n) << 8)
>> +#define YUVSW1(n) ((n) << 4)
>> +#define YUVSW0(n) ((n) << 0)
>> +
>> +#define ISP_REG_DNYUV_YDR0 0xc10
>> +#define ISP_REG_DNYUV_YDR1 0xc14
>> +#define ISP_REG_DNYUV_YDR2 0xc18
>> +#define ISP_REG_DNYUV_CDR0 0xc1c
>> +#define ISP_REG_DNYUV_CDR1 0xc20
>> +#define ISP_REG_DNYUV_CDR2 0xc24
>> +#define CURVE_D_H(n) ((n) << 16)
>> +#define CURVE_D_L(n) ((n) << 0)
>> +
>> +#define ISP_REG_ICAMD_0 0xc40
>> +#define DNRM_F(n) ((n) << 16)
>> +#define ISP_REG_ICAMD_12 0xc70
>> +#define ISP_REG_ICAMD_20 0xc90
>> +#define ISP_REG_ICAMD_24 0xca0
>> +#define ISP_REG_ICAMD_25 0xca4
>> +#define CCM_M_DAT(n) ((n) << 0)
>> +
>> +#define ISP_REG_GAMMA_VAL0 0xe00
>> +#define ISP_REG_GAMMA_VAL1 0xe04
>> +#define ISP_REG_GAMMA_VAL2 0xe08
>> +#define ISP_REG_GAMMA_VAL3 0xe0c
>> +#define ISP_REG_GAMMA_VAL4 0xe10
>> +#define ISP_REG_GAMMA_VAL5 0xe14
>> +#define ISP_REG_GAMMA_VAL6 0xe18
>> +#define ISP_REG_GAMMA_VAL7 0xe1c
>> +#define ISP_REG_GAMMA_VAL8 0xe20
>> +#define ISP_REG_GAMMA_VAL9 0xe24
>> +#define ISP_REG_GAMMA_VAL10 0xe28
>> +#define ISP_REG_GAMMA_VAL11 0xe2c
>> +#define ISP_REG_GAMMA_VAL12 0xe30
>> +#define ISP_REG_GAMMA_VAL13 0xe34
>> +#define ISP_REG_GAMMA_VAL14 0xe38
>> +#define GAMMA_S_VAL(n) ((n) << 16)
>> +#define GAMMA_VAL(n) ((n) << 0)
>> +
>> +#define ISP_REG_R2Y_0 0xe40
>> +#define ISP_REG_R2Y_1 0xe44
>> +#define ISP_REG_R2Y_2 0xe48
>> +#define ISP_REG_R2Y_3 0xe4c
>> +#define ISP_REG_R2Y_4 0xe50
>> +#define ISP_REG_R2Y_5 0xe54
>> +#define ISP_REG_R2Y_6 0xe58
>> +#define ISP_REG_R2Y_7 0xe5c
>> +#define ISP_REG_R2Y_8 0xe60
>> +#define CSC_M(n) ((n) << 0)
>> +
>> +#define ISP_REG_SHARPEN0 0xe80
>> +#define ISP_REG_SHARPEN1 0xe84
>> +#define ISP_REG_SHARPEN2 0xe88
>> +#define ISP_REG_SHARPEN3 0xe8c
>> +#define ISP_REG_SHARPEN4 0xe90
>> +#define ISP_REG_SHARPEN5 0xe94
>> +#define ISP_REG_SHARPEN6 0xe98
>> +#define ISP_REG_SHARPEN7 0xe9c
>> +#define ISP_REG_SHARPEN8 0xea0
>> +#define ISP_REG_SHARPEN9 0xea4
>> +#define ISP_REG_SHARPEN10 0xea8
>> +#define ISP_REG_SHARPEN11 0xeac
>> +#define ISP_REG_SHARPEN12 0xeb0
>> +#define ISP_REG_SHARPEN13 0xeb4
>> +#define ISP_REG_SHARPEN14 0xeb8
>> +#define S_DELTA(n) ((n) << 16)
>> +#define S_WEIGHT(n) ((n) << 8)
>> +
>> +#define ISP_REG_SHARPEN_FS0 0xebc
>> +#define ISP_REG_SHARPEN_FS1 0xec0
>> +#define ISP_REG_SHARPEN_FS2 0xec4
>> +#define ISP_REG_SHARPEN_FS3 0xec8
>> +#define ISP_REG_SHARPEN_FS4 0xecc
>> +#define ISP_REG_SHARPEN_FS5 0xed0
>> +#define S_FACTOR(n) ((n) << 24)
>> +#define S_SLOPE(n) ((n) << 0)
>> +
>> +#define ISP_REG_SHARPEN_WN 0xed4
>> +#define PDIRF(n) ((n) << 28)
>> +#define NDIRF(n) ((n) << 24)
>> +#define WSUM(n) ((n) << 0)
>> +
>> +#define ISP_REG_IUVS1 0xed8
>> +#define UVDIFF2(n) ((n) << 16)
>> +#define UVDIFF1(n) ((n) << 0)
>> +
>> +#define ISP_REG_IUVS2 0xedc
>> +#define UVF(n) ((n) << 24)
>> +#define UVSLOPE(n) ((n) << 0)
>> +
>> +#define ISP_REG_IUVCKS1 0xee0
>> +#define UVCKDIFF2(n) ((n) << 16)
>> +#define UVCKDIFF1(n) ((n) << 0)
>> +
>> +#define ISP_REG_IUVCKS2 0xee4
>> +#define UVCKSLOPE(n) ((n) << 0)
>> +
>> +#define ISP_REG_ISHRPET 0xee8
>> +#define TH(n) ((n) << 8)
>> +#define EN(n) ((n) << 0)
>> +
>> +#define ISP_REG_YCURVE_0 0xf00
>> +#define ISP_REG_YCURVE_63 0xffc
>> +#define L_PARAM(n) ((n) << 0)
>> +
>> +#define IMAGE_MAX_WIDTH 1920
>> +#define IMAGE_MAX_HEIGH 1080
>> +
>> +/* The output line of ISP */
>
> What is an ISP "line" ?
>
A pipeline contains ISP.
>> +enum isp_line_id {
>> + STF_ISP_LINE_INVALID = -1,
>> + STF_ISP_LINE_SRC = 1,
>> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
>> +};
>> +
>> +/* pad id for media framework */
>> +enum isp_pad_id {
>> + STF_ISP_PAD_SINK = 0,
>> + STF_ISP_PAD_SRC,
>> + STF_ISP_PAD_MAX
>> +};
>> +
>> +enum {
>> + EN_INT_NONE = 0,
>> + EN_INT_ISP_DONE = (0x1 << 24),
>> + EN_INT_CSI_DONE = (0x1 << 25),
>> + EN_INT_SC_DONE = (0x1 << 26),
>> + EN_INT_LINE_INT = (0x1 << 27),
>> + EN_INT_ALL = (0xF << 24),
>
> Please use lowercase for hex constants.
>
> But I'm wondering if this is really needed, it seems to duplicate the
> ISPC_* bits defined above. I would use those directly.
>
>> +};
>> +
>> +enum {
>> + INTERFACE_DVP = 0,
>> + INTERFACE_CSI,
>> +};
>
> This isn't used.
>
>> +
>> +struct isp_format {
>> + u32 code;
>> + u8 bpp;
>> +};
>> +
>> +struct isp_format_table {
>> + const struct isp_format *fmts;
>> + int nfmts;
>> +};
>> +
>> +struct regval_t {
>> + u32 addr;
>> + u32 val;
>> + u32 delay_ms;
>> +};
>> +
>> +struct reg_table {
>> + const struct regval_t *regval;
>> + int regval_num;
>> +};
>
> Those two structures are not used.
>
>> +
>> +struct stf_isp_crop {
>> + struct v4l2_rect rect;
>> + u32 bpp;
>> +};
>> +
>> +struct stf_isp_dev {
>> + struct stfcamss *stfcamss;
>> + struct v4l2_subdev subdev;
>> + struct media_pad pads[STF_ISP_PAD_MAX];
>> + struct v4l2_mbus_framefmt fmt[STF_ISP_PAD_MAX];
>> + struct stf_isp_crop rect[STF_ISP_PAD_MAX];
>> + const struct isp_format_table *formats;
>> + unsigned int nformats;
>> + struct mutex stream_lock; /* serialize stream control */
>
> You should explicitly list the fields protected by the mutex.
>
>> + int stream_count;
>> +};
>> +
>> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev);
>> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev);
>> +int stf_isp_reset(struct stf_isp_dev *isp_dev);
>> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev);
>> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
>> + struct stf_isp_crop *crop_array, u32 mcode);
>> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev);
>> +int stf_isp_subdev_init(struct stfcamss *stfcamss);
>> +int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev);
>> +int stf_isp_unregister(struct stf_isp_dev *isp_dev);
>> +
>> +#endif /* STF_ISP_H */
>> diff --git a/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>> new file mode 100644
>> index 000000000000..2088b6bd0d61
>> --- /dev/null
>> +++ b/drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>> @@ -0,0 +1,468 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * stf_isp_hw_ops.c
>> + *
>> + * Register interface file for StarFive ISP driver
>> + *
>> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> + *
>> + */
>> +
>> +#include "stf_camss.h"
>> +
>> +static void stf_isp_config_obc(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_val, reg_add;
>> +
>> + stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG, OBC_W_H(11) | OBC_W_W(11));
>> +
>> + reg_val = GAIN_D_POINT(0x40) | GAIN_C_POINT(0x40) |
>> + GAIN_B_POINT(0x40) | GAIN_A_POINT(0x40);
>> + for (reg_add = ISP_REG_OBCG_CFG_0; reg_add <= ISP_REG_OBCG_CFG_3;) {
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + }
>> +
>> + reg_val = OFFSET_D_POINT(0) | OFFSET_C_POINT(0) |
>> + OFFSET_B_POINT(0) | OFFSET_A_POINT(0);
>> + for (reg_add = ISP_REG_OBCO_CFG_0; reg_add <= ISP_REG_OBCO_CFG_3;) {
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + }
>> +}
>> +
>> +static void stf_isp_config_oecf(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_add, par_val;
>> + u16 par_h, par_l;
>> +
>> + par_h = 0x10; par_l = 0;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG0; reg_add <= ISP_REG_OECF_Y3_CFG0;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x40; par_l = 0x20;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG1; reg_add <= ISP_REG_OECF_Y3_CFG1;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x80; par_l = 0x60;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG2; reg_add <= ISP_REG_OECF_Y3_CFG2;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0xc0; par_l = 0xa0;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG3; reg_add <= ISP_REG_OECF_Y3_CFG3;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x100; par_l = 0xe0;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG4; reg_add <= ISP_REG_OECF_Y3_CFG4;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x200; par_l = 0x180;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG5; reg_add <= ISP_REG_OECF_Y3_CFG5;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x300; par_l = 0x280;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG6; reg_add <= ISP_REG_OECF_Y3_CFG6;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x3fe; par_l = 0x380;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_X0_CFG7; reg_add <= ISP_REG_OECF_Y3_CFG7;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 0x20;
>> + }
>> +
>> + par_h = 0x80; par_l = 0x80;
>> + par_val = OCEF_PAR_H(par_h) | OCEF_PAR_L(par_l);
>> + for (reg_add = ISP_REG_OECF_S0_CFG0; reg_add <= ISP_REG_OECF_S3_CFG7;) {
>> + stf_isp_reg_write(stfcamss, reg_add, par_val);
>> + reg_add += 4;
>> + }
>> +}
>> +
>> +static void stf_isp_config_lccf(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_add;
>> +
>> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
>> + Y_DISTANCE(0x21C) | X_DISTANCE(0x3C0));
>> + stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1, LCCF_MAX_DIS(0xb));
>> +
>> + for (reg_add = ISP_REG_LCCF_CFG_2; reg_add <= ISP_REG_LCCF_CFG_5;) {
>> + stf_isp_reg_write(stfcamss, reg_add,
>> + LCCF_F2_PAR(0x0) | LCCF_F1_PAR(0x0));
>> + reg_add += 4;
>> + }
>> +}
>> +
>> +static void stf_isp_config_awb(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_val, reg_add;
>> + u16 symbol_h, symbol_l;
>> +
>> + symbol_h = 0x0; symbol_l = 0x0;
>> + reg_val = AWB_X_SYMBOL_H(symbol_h) | AWB_X_SYMBOL_L(symbol_l);
>> +
>> + for (reg_add = ISP_REG_AWB_X0_CFG_0; reg_add <= ISP_REG_AWB_X3_CFG_1;) {
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + }
>> +
>> + symbol_h = 0x0, symbol_l = 0x0;
>> + reg_val = AWB_Y_SYMBOL_H(symbol_h) | AWB_Y_SYMBOL_L(symbol_l);
>> +
>> + for (reg_add = ISP_REG_AWB_Y0_CFG_0; reg_add <= ISP_REG_AWB_Y3_CFG_1;) {
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + }
>> +
>> + symbol_h = 0x80, symbol_l = 0x80;
>> + reg_val = AWB_S_SYMBOL_H(symbol_h) | AWB_S_SYMBOL_L(symbol_l);
>> +
>> + for (reg_add = ISP_REG_AWB_S0_CFG_0; reg_add <= ISP_REG_AWB_S3_CFG_1;) {
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + }
>> +}
>> +
>> +static void stf_isp_config_grgb(struct stfcamss *stfcamss)
>> +{
>> + stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
>> + GF_MODE(1) | MAXGT(0x140) | MINGT(0x40));
>> + stf_isp_reg_write(stfcamss, ISP_REG_IDBC, BADGT(0x200) | BADXT(0x200));
>> +}
>> +
>> +static void stf_isp_config_cfa(struct stfcamss *stfcamss)
>> +{
>> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG,
>> + SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) | SMY3(2) |
>> + SMY2(3) | SMY1(2) | SMY0(3));
>> + stf_isp_reg_write(stfcamss, ISP_REG_ICFAM, CROSS_COV(3) | HV_W(2));
>> +}
>> +
>> +static void stf_isp_config_ccm(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_add;
>> +
>> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6) | CCM_M_DAT(0));
>> +
>> + for (reg_add = ISP_REG_ICAMD_12; reg_add <= ISP_REG_ICAMD_20;) {
>> + stf_isp_reg_write(stfcamss, reg_add, CCM_M_DAT(0x80));
>> + reg_add += 0x10;
>> + }
>> +
>> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_24, CCM_M_DAT(0x700));
>> + stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_25, CCM_M_DAT(0x200));
>> +}
>> +
>> +static void stf_isp_config_gamma(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_val, reg_add;
>> + u16 gamma_slope_v, gamma_v;
>> +
>> + gamma_slope_v = 0x2400; gamma_v = 0x0;
>> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
>> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL0, reg_val);
>> +
>> + gamma_slope_v = 0x800; gamma_v = 0x20;
>> + for (reg_add = ISP_REG_GAMMA_VAL1; reg_add <= ISP_REG_GAMMA_VAL7;) {
>> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + gamma_v += 0x20;
>> + }
>> +
>> + gamma_v = 0x100;
>> + for (reg_add = ISP_REG_GAMMA_VAL8; reg_add <= ISP_REG_GAMMA_VAL13;) {
>> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
>> + stf_isp_reg_write(stfcamss, reg_add, reg_val);
>> + reg_add += 4;
>> + gamma_v += 0x80;
>> + }
>> +
>> + gamma_v = 0x3fe;
>> + reg_val = GAMMA_S_VAL(gamma_slope_v) | GAMMA_VAL(gamma_v);
>> + stf_isp_reg_write(stfcamss, ISP_REG_GAMMA_VAL14, reg_val);
>> +}
>> +
>> +static void stf_isp_config_r2y(struct stfcamss *stfcamss)
>> +{
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_0, CSC_M(0x4C));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_1, CSC_M(0x97));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_2, CSC_M(0x1d));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_3, CSC_M(0x1d5));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_4, CSC_M(0x1ac));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_5, CSC_M(0x80));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_6, CSC_M(0x80));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_7, CSC_M(0x194));
>> + stf_isp_reg_write(stfcamss, ISP_REG_R2Y_8, CSC_M(0x1ec));
>> +}
>> +
>> +static void stf_isp_config_y_curve(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_add;
>> + u16 y_curve;
>> +
>> + y_curve = 0x0;
>> + for (reg_add = ISP_REG_YCURVE_0; reg_add <= ISP_REG_YCURVE_63;) {
>> + stf_isp_reg_write(stfcamss, reg_add, y_curve);
>> + reg_add += 4;
>> + y_curve += 0x10;
>> + }
>> +}
>> +
>> +static void stf_isp_config_sharpen(struct stfcamss *sc)
>> +{
>> + u32 reg_add;
>> +
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN0, S_DELTA(0x7) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN1, S_DELTA(0x18) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN2, S_DELTA(0x80) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN3, S_DELTA(0x100) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN4, S_DELTA(0x10) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN5, S_DELTA(0x60) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN6, S_DELTA(0x100) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN7, S_DELTA(0x190) | S_WEIGHT(0xf));
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN8, S_DELTA(0x0) | S_WEIGHT(0xf));
>> +
>> + for (reg_add = ISP_REG_SHARPEN9; reg_add <= ISP_REG_SHARPEN14;) {
>> + stf_isp_reg_write(sc, reg_add, S_WEIGHT(0xf));
>> + reg_add += 4;
>> + }
>> +
>> + for (reg_add = ISP_REG_SHARPEN_FS0; reg_add <= ISP_REG_SHARPEN_FS5;) {
>> + stf_isp_reg_write(sc, reg_add, S_FACTOR(0x10) | S_SLOPE(0x0));
>> + reg_add += 4;
>> + }
>> +
>> + stf_isp_reg_write(sc, ISP_REG_SHARPEN_WN,
>> + PDIRF(0x8) | NDIRF(0x8) | WSUM(0xd7c));
>> + stf_isp_reg_write(sc, ISP_REG_IUVS1, UVDIFF2(0xC0) | UVDIFF1(0x40));
>> + stf_isp_reg_write(sc, ISP_REG_IUVS2, UVF(0xff) | UVSLOPE(0x0));
>> + stf_isp_reg_write(sc, ISP_REG_IUVCKS1,
>> + UVCKDIFF2(0xa0) | UVCKDIFF1(0x40));
>> +}
>> +
>> +static void stf_isp_config_dnyuv(struct stfcamss *stfcamss)
>> +{
>> + u32 reg_val;
>> +
>> + reg_val = YUVSW5(7) | YUVSW4(7) | YUVSW3(7) | YUVSW2(7) |
>> + YUVSW1(7) | YUVSW0(7);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0, reg_val);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0, reg_val);
>> +
>> + reg_val = YUVSW3(7) | YUVSW2(7) | YUVSW1(7) | YUVSW0(7);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1, reg_val);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1, reg_val);
>> +
>> + reg_val = CURVE_D_H(0x60) | CURVE_D_L(0x40);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0, reg_val);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0, reg_val);
>> +
>> + reg_val = CURVE_D_H(0xd8) | CURVE_D_L(0x90);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1, reg_val);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1, reg_val);
>> +
>> + reg_val = CURVE_D_H(0x1e6) | CURVE_D_L(0x144);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2, reg_val);
>> + stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2, reg_val);
>> +}
>> +
>> +static void stf_isp_config_sat(struct stfcamss *stfcamss)
>> +{
>> + stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN, CMAD(0x0) | CMAB(0x100));
>> + stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD, CMD(0x1f) | CMB(0x1));
>> + stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET, VOFF(0x0) | UOFF(0x0));
>> + stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F, SIN(0x0) | COS(0x100));
>> + stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, CMSF(0x8));
>> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ0, YOIR(0x401) | YIMIN(0x1));
>> + stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
>> +}
>> +
>> +int stf_isp_clk_enable(struct stf_isp_dev *isp_dev)
>> +{
>> + struct stfcamss *stfcamss = isp_dev->stfcamss;
>> +
>> + clk_prepare_enable(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
>> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
>> + reset_control_deassert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
>
> Missing error handling.
>
> Is there a specific hardware requirement to split the clock enabling and
> reset deassertion in two, with some of them handled in the runtime PM
> resume handler and the rest here ? If not, moving all clock and reset
> control to the runtime PM handlers would simplify the code.
>
>> +
>> + return 0;
>> +}
>> +
>> +int stf_isp_clk_disable(struct stf_isp_dev *isp_dev)
>> +{
>> + struct stfcamss *stfcamss = isp_dev->stfcamss;
>> +
>> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_C].rstc);
>> + reset_control_assert(stfcamss->sys_rst[STF_RST_WRAPPER_P].rstc);
>> + clk_disable_unprepare(stfcamss->sys_clk[STF_CLK_WRAPPER_CLK_C].clk);
>> +
>> + return 0;
>> +}
>> +
>> +int stf_isp_reset(struct stf_isp_dev *isp_dev)
>> +{
>> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
>> + ISPC_RST_MASK, ISPC_RST);
>> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
>> + ISPC_RST_MASK, 0);
>> +
>> + return 0;
>> +}
>> +
>> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
>> +{
>> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
>> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
>> + DEC_V_KEEP(0x0) |
>> + DEC_V_PERIOD(0x0) |
>> + DEC_H_KEEP(0x0) |
>> + DEC_H_PERIOD(0x0));
>> +
>> + stf_isp_config_obc(isp_dev->stfcamss);
>> + stf_isp_config_oecf(isp_dev->stfcamss);
>> + stf_isp_config_lccf(isp_dev->stfcamss);
>> + stf_isp_config_awb(isp_dev->stfcamss);
>> + stf_isp_config_grgb(isp_dev->stfcamss);
>> + stf_isp_config_cfa(isp_dev->stfcamss);
>> + stf_isp_config_ccm(isp_dev->stfcamss);
>> + stf_isp_config_gamma(isp_dev->stfcamss);
>> + stf_isp_config_r2y(isp_dev->stfcamss);
>> + stf_isp_config_y_curve(isp_dev->stfcamss);
>> + stf_isp_config_sharpen(isp_dev->stfcamss);
>> + stf_isp_config_dnyuv(isp_dev->stfcamss);
>> + stf_isp_config_sat(isp_dev->stfcamss);
>
> All these parameters are hardcoded, why are they not exposed to
> userspace ?
>
Here is a basic startup configuration for the ISP registers. The
function name is confusing, as if it is configuring a specific
function. In fact, it is just a basic init configuration.
>> +
>> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
>> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
>> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
>> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
>> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
>> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
>> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
>> +}
>> +
>> +static void stf_isp_config_crop(struct stfcamss *stfcamss,
>> + struct stf_isp_crop *crop)
>> +{
>> + struct v4l2_rect *rect = &crop[STF_ISP_PAD_SRC].rect;
>> + u32 bpp = crop[STF_ISP_PAD_SRC].bpp;
>> + u32 val;
>> +
>> + val = VSTART_CAP(rect->top) | HSTART_CAP(rect->left);
>> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_START_CFG, val);
>> +
>> + val = VEND_CAP(rect->height + rect->top - 1) |
>> + HEND_CAP(rect->width + rect->left - 1);
>> + stf_isp_reg_write(stfcamss, ISP_REG_PIC_CAPTURE_END_CFG, val);
>> +
>> + val = H_ACT_CAP(rect->height) | W_ACT_CAP(rect->width);
>> + stf_isp_reg_write(stfcamss, ISP_REG_PIPELINE_XY_SIZE, val);
>> +
>> + val = ALIGN(rect->width * bpp / 8, STFCAMSS_FRAME_WIDTH_ALIGN_8);
>> + stf_isp_reg_write(stfcamss, ISP_REG_STRIDE, val);
>> +}
>> +
>> +static void stf_isp_config_raw_fmt(struct stfcamss *stfcamss, u32 mcode)
>> +{
>> + u32 val, val1;
>> +
>> + switch (mcode) {
>> + case MEDIA_BUS_FMT_SRGGB10_1X10:
>> + case MEDIA_BUS_FMT_SRGGB8_1X8:
>> + /* 3 2 3 2 1 0 1 0 B Gb B Gb Gr R Gr R */
>> + val = SMY13(3) | SMY12(2) | SMY11(3) | SMY10(2) |
>> + SMY3(1) | SMY2(0) | SMY1(1) | SMY0(0);
>> + val1 = CTRL_SAT(0x0);
>> + break;
>> + case MEDIA_BUS_FMT_SGRBG10_1X10:
>> + case MEDIA_BUS_FMT_SGRBG8_1X8:
>> + /* 2 3 2 3 0 1 0 1, Gb B Gb B R Gr R Gr */
>> + val = SMY13(2) | SMY12(3) | SMY11(2) | SMY10(3) |
>> + SMY3(0) | SMY2(1) | SMY1(0) | SMY0(1);
>> + val1 = CTRL_SAT(0x2);
>> + break;
>> + case MEDIA_BUS_FMT_SGBRG10_1X10:
>> + case MEDIA_BUS_FMT_SGBRG8_1X8:
>> + /* 1 0 1 0 3 2 3 2, Gr R Gr R B Gb B Gb */
>> + val = SMY13(1) | SMY12(0) | SMY11(1) | SMY10(0) |
>> + SMY3(3) | SMY2(2) | SMY1(3) | SMY0(2);
>> + val1 = CTRL_SAT(0x3);
>> + break;
>> + case MEDIA_BUS_FMT_SBGGR10_1X10:
>> + case MEDIA_BUS_FMT_SBGGR8_1X8:
>> + /* 0 1 0 1 2 3 2 3 R Gr R Gr Gb B Gb B */
>> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
>> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
>> + val1 = CTRL_SAT(0x1);
>> + break;
>> + default:
>> + val = SMY13(0) | SMY12(1) | SMY11(0) | SMY10(1) |
>> + SMY3(2) | SMY2(3) | SMY1(2) | SMY0(3);
>> + val1 = CTRL_SAT(0x1);
>> + break;
>> + }
>> + stf_isp_reg_write(stfcamss, ISP_REG_RAW_FORMAT_CFG, val);
>> + stf_isp_reg_set_bit(stfcamss, ISP_REG_ISP_CTRL_1, CTRL_SAT_MASK, val1);
>> +}
>> +
>> +void stf_isp_settings(struct stf_isp_dev *isp_dev,
>> + struct stf_isp_crop *crop, u32 mcode)
>> +{
>> + struct stfcamss *stfcamss = isp_dev->stfcamss;
>> +
>> + stf_isp_config_crop(stfcamss, crop);
>> + stf_isp_config_raw_fmt(stfcamss, mcode);
>> +
>> + stf_isp_reg_set_bit(stfcamss, ISP_REG_DUMP_CFG_1,
>> + DUMP_BURST_LEN_MASK | DUMP_SD_MASK,
>> + DUMP_BURST_LEN(3));
>> +
>> + stf_isp_reg_write(stfcamss, ISP_REG_ITIIWSR,
>> + ITI_HSIZE(IMAGE_MAX_HEIGH) |
>> + ITI_WSIZE(IMAGE_MAX_WIDTH));
>> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDWLSR, ITI_WSTRIDE(0x960));
>> + stf_isp_reg_write(stfcamss, ISP_REG_ITIDRLSR, ITI_STRIDE_L(0x960));
>> + stf_isp_reg_write(stfcamss, ISP_REG_SENSOR, 0x1);
>
> Should 0x1 be IMAGER_SEL(1) or are those unrelated ?
>
>> +}
>> +
>> +void stf_isp_stream_set(struct stf_isp_dev *isp_dev)
>> +{
>> + struct stfcamss *stfcamss = isp_dev->stfcamss;
>> +
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
>> + ISPC_ENUO | ISPC_ENLS | ISPC_RST, 10);
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
>> + ISPC_ENUO | ISPC_ENLS, 10);
>> + stf_isp_reg_write(stfcamss, ISP_REG_IESHD, SHAD_UP_M);
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_ISP_CTRL_0,
>> + ISPC_ENUO | ISPC_ENLS | ISPC_EN, 10);
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
>> + CSI_INTS(1) | CSI_SHA_M(4), 10);
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSIINTS,
>> + CSI_INTS(2) | CSI_SHA_M(4), 10);
>> + stf_isp_reg_write_delay(stfcamss, ISP_REG_CSI_INPUT_EN_AND_STATUS,
>> + CSI_EN_S, 10);
>> +}
>
Regards,
Jack Zhu
Hi Jack,
On Wed, Aug 02, 2023 at 05:57:57PM +0800, Jack Zhu wrote:
> On 2023/7/28 4:41, Laurent Pinchart wrote:
> > On Mon, Jun 19, 2023 at 07:28:37PM +0800, Jack Zhu wrote:
> >> Add ISP driver for StarFive Camera Subsystem.
> >>
> >> Signed-off-by: Jack Zhu <[email protected]>
> >> ---
> >> .../media/platform/starfive/camss/Makefile | 2 +
> >> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
> >> .../media/platform/starfive/camss/stf_camss.h | 3 +
> >> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
> >> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
> >> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
> >> 6 files changed, 1544 insertions(+), 3 deletions(-)
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
[snip]
> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
> >> new file mode 100644
> >> index 000000000000..933a583b398c
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
> >> @@ -0,0 +1,519 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * stf_isp.c
> >> + *
> >> + * StarFive Camera Subsystem - ISP Module
> >> + *
> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> + */
> >> +#include <linux/firmware.h>
> >
> > This doesn't seem needed.
> >
> >> +#include <media/v4l2-event.h>
> >> +
> >> +#include "stf_camss.h"
> >> +
> >> +#define SINK_FORMATS_INDEX 0
> >> +#define UO_FORMATS_INDEX 1
> >
> > What does "UO" stand for ?
>
> "UO" is Usual Out, just represents output. :-)
Maybe "out", "output" or "source" would make the code easier to read
then ?
> >> +
> >> +static int isp_set_selection(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_selection *sel);
> >> +
> >> +static const struct isp_format isp_formats_sink[] = {
> >> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
> >> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
> >> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
> >> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
> >> +};
[snip]
> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
> >> new file mode 100644
> >> index 000000000000..1e5c98482350
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
> >> @@ -0,0 +1,479 @@
[snip]
> >> +/* The output line of ISP */
> >
> > What is an ISP "line" ?
>
> A pipeline contains ISP.
Patch 6/6 uses STF_ISP_LINE_MAX to iterate over the ISP lines. This
makes the code somehow generic, but you only support a single line at
the moment. Does this or other SoCs in your product line integrate the
same ISP with multiple lines ? If so, would it be possible to share a
block diagram, to better understand the other hardware architectures
that this driver will need to support in the future ?
> >> +enum isp_line_id {
> >> + STF_ISP_LINE_INVALID = -1,
> >> + STF_ISP_LINE_SRC = 1,
> >> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
> >> +};
[snip]
> >> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
> >> +{
> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
> >> + DEC_V_KEEP(0x0) |
> >> + DEC_V_PERIOD(0x0) |
> >> + DEC_H_KEEP(0x0) |
> >> + DEC_H_PERIOD(0x0));
> >> +
> >> + stf_isp_config_obc(isp_dev->stfcamss);
> >> + stf_isp_config_oecf(isp_dev->stfcamss);
> >> + stf_isp_config_lccf(isp_dev->stfcamss);
> >> + stf_isp_config_awb(isp_dev->stfcamss);
> >> + stf_isp_config_grgb(isp_dev->stfcamss);
> >> + stf_isp_config_cfa(isp_dev->stfcamss);
> >> + stf_isp_config_ccm(isp_dev->stfcamss);
> >> + stf_isp_config_gamma(isp_dev->stfcamss);
> >> + stf_isp_config_r2y(isp_dev->stfcamss);
> >> + stf_isp_config_y_curve(isp_dev->stfcamss);
> >> + stf_isp_config_sharpen(isp_dev->stfcamss);
> >> + stf_isp_config_dnyuv(isp_dev->stfcamss);
> >> + stf_isp_config_sat(isp_dev->stfcamss);
> >
> > All these parameters are hardcoded, why are they not exposed to
> > userspace ?
>
> Here is a basic startup configuration for the ISP registers. The
> function name is confusing, as if it is configuring a specific
> function. In fact, it is just a basic init configuration.
Did I miss a place in the patch series where all these parameters can be
configured by userspace, or is that not possible at the moment ? If it
isn't possible, do you plan to implement that ?
> >> +
> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
> >> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
> >> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
> >> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
> >> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
> >> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
> >> +}
[snip]
--
Regards,
Laurent Pinchart
Hi Jack,
On Wed, Aug 02, 2023 at 05:58:26PM +0800, Jack Zhu wrote:
> On 2023/7/28 4:49, Laurent Pinchart wrote:
> > On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
> >> Add Video In Controller driver for StarFive Camera Subsystem.
> >
> > I haven't reviewed this patch in details, as I have a high-level
> > question: why do you need VIN subdevs ? They don't seem to represent any
> > particular piece of hardware, their input and output formats are always
> > identical, and they're not used to configure registers. The contents of
> > this patch seems to belong to the video device, I think you can drop the
> > VIN subdevs.
>
> The VIN module corresponds to a hardware module, which is mainly responsible
> for data routing and partial interrupt management (when the image data does
> not pass through the isp but directly goes to the ddr), the relevant registers
> need to be configured.
That's fine, but I don't think you need a subdev for it. As far as I
understand, the VIn modules are (more or less) DMA engines. You can just
model them as video devices, connected directly to the CSI-2 RX and ISP
source pads.
Does the "vin0_wr" have the ability to capture raw data from the DVP
interface as well, or only from the CSI-2 RX ?
> >> Signed-off-by: Jack Zhu <[email protected]>
> >> ---
> >> .../media/platform/starfive/camss/Makefile | 4 +-
> >> .../media/platform/starfive/camss/stf_camss.c | 42 +-
> >> .../media/platform/starfive/camss/stf_camss.h | 2 +
> >> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
> >> .../media/platform/starfive/camss/stf_vin.h | 173 +++
> >> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
> >> 6 files changed, 1478 insertions(+), 4 deletions(-)
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> >>
> >> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> >> index cdf57e8c9546..ef574e01ca47 100644
> >> --- a/drivers/media/platform/starfive/camss/Makefile
> >> +++ b/drivers/media/platform/starfive/camss/Makefile
> >> @@ -7,6 +7,8 @@ starfive-camss-objs += \
> >> stf_camss.o \
> >> stf_isp.o \
> >> stf_isp_hw_ops.o \
> >> - stf_video.o
> >> + stf_video.o \
> >> + stf_vin.o \
> >> + stf_vin_hw_ops.o
> >>
> >> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> >> index 6f56b45f57db..834ea63eb833 100644
> >> --- a/drivers/media/platform/starfive/camss/stf_camss.c
> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> >> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
> >> return ret;
> >> }
> >>
> >> + ret = stf_vin_subdev_init(stfcamss);
> >> + if (ret < 0) {
> >> + dev_err(stfcamss->dev, "Failed to init vin subdev: %d\n", ret);
> >> + return ret;
> >> + }
> >> return ret;
> >> }
> >>
> >> static int stfcamss_register_subdevices(struct stfcamss *stfcamss)
> >> {
> >> int ret;
> >> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> >> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> >>
> >> ret = stf_isp_register(isp_dev, &stfcamss->v4l2_dev);
> >> if (ret < 0) {
> >> dev_err(stfcamss->dev,
> >> "Failed to register stf isp%d entity: %d\n", 0, ret);
> >> - return ret;
> >> + goto err_reg_isp;
> >> + }
> >> +
> >> + ret = stf_vin_register(vin_dev, &stfcamss->v4l2_dev);
> >> + if (ret < 0) {
> >> + dev_err(stfcamss->dev,
> >> + "Failed to register vin entity: %d\n", ret);
> >> + goto err_reg_vin;
> >> }
> >>
> >> + ret = media_create_pad_link(&isp_dev->subdev.entity,
> >> + STF_ISP_PAD_SRC,
> >> + &vin_dev->line[VIN_LINE_ISP].subdev.entity,
> >> + STF_VIN_PAD_SINK,
> >> + 0);
> >> + if (ret < 0) {
> >> + dev_err(stfcamss->dev,
> >> + "Failed to link %s->%s entities: %d\n",
> >> + isp_dev->subdev.entity.name,
> >> + vin_dev->line[VIN_LINE_ISP].subdev.entity.name, ret);
> >> + goto err_link;
> >> + }
> >> +
> >> + return ret;
> >> +
> >> +err_link:
> >> + stf_vin_unregister(&stfcamss->vin_dev);
> >> +err_reg_vin:
> >> + stf_isp_unregister(&stfcamss->isp_dev);
> >> +err_reg_isp:
> >> return ret;
> >> }
> >>
> >> static void stfcamss_unregister_subdevices(struct stfcamss *stfcamss)
> >> {
> >> stf_isp_unregister(&stfcamss->isp_dev);
> >> + stf_vin_unregister(&stfcamss->vin_dev);
> >> }
> >>
> >> static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> >> @@ -164,12 +198,14 @@ static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> >> container_of(asd, struct stfcamss_async_subdev, asd);
> >> enum stf_port_num port = csd->port;
> >> struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> >> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> >> struct media_pad *pad[STF_PADS_NUM];
> >> unsigned int i, pad_num;
> >>
> >> if (port == STF_PORT_CSI2RX) {
> >> - pad[0] = &isp_dev->pads[STF_PAD_SINK];
> >> - pad_num = 1;
> >> + pad[0] = &vin_dev->line[VIN_LINE_WR].pads[STF_PAD_SINK];
> >> + pad[1] = &isp_dev->pads[STF_PAD_SINK];
> >> + pad_num = 2;
> >> } else if (port == STF_PORT_DVP) {
> >> dev_err(stfcamss->dev, "Not support DVP sensor\n");
> >> return -EPERM;
> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.h b/drivers/media/platform/starfive/camss/stf_camss.h
> >> index 9482081288fa..a14f22bc0742 100644
> >> --- a/drivers/media/platform/starfive/camss/stf_camss.h
> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.h
> >> @@ -19,6 +19,7 @@
> >> #include <media/v4l2-device.h>
> >>
> >> #include "stf_isp.h"
> >> +#include "stf_vin.h"
> >>
> >> #define STF_DVP_NAME "stf_dvp"
> >> #define STF_CSI_NAME "cdns_csi2rx"
> >> @@ -67,6 +68,7 @@ struct stfcamss {
> >> struct media_device media_dev;
> >> struct media_pipeline pipe;
> >> struct device *dev;
> >> + struct stf_vin_dev vin_dev;
> >> struct stf_isp_dev isp_dev;
> >> struct v4l2_async_notifier notifier;
> >> void __iomem *syscon_base;
> >> diff --git a/drivers/media/platform/starfive/camss/stf_vin.c b/drivers/media/platform/starfive/camss/stf_vin.c
> >> new file mode 100644
> >> index 000000000000..0efa4bbb079c
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_vin.c
> >> @@ -0,0 +1,1069 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * stf_vin.c
> >> + *
> >> + * StarFive Camera Subsystem - VIN Module
> >> + *
> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> + */
> >> +
> >> +#include <linux/dma-mapping.h>
> >> +#include <linux/pm_runtime.h>
> >> +
> >> +#include "stf_camss.h"
> >> +
> >> +#define vin_line_array(ptr_line) \
> >> + ((const struct vin_line (*)[]) &(ptr_line)[-((ptr_line)->id)])
> >> +
> >> +#define line_to_vin_dev(ptr_line) \
> >> + container_of(vin_line_array(ptr_line), struct stf_vin_dev, line)
> >> +
> >> +#define VIN_FRAME_DROP_MIN_VAL 4
> >> +
> >> +/* ISP ctrl need 1 sec to let frames become stable. */
> >> +#define VIN_FRAME_DROP_SEC_FOR_ISP_CTRL 1
> >> +
> >> +static const struct vin_format vin_formats_wr[] = {
> >> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10},
> >> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10},
> >> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10},
> >> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10},
> >> +};
> >> +
> >> +static const struct vin_format vin_formats_uo[] = {
> >> + { MEDIA_BUS_FMT_Y12_1X12, 8},
> >> +};
> >> +
> >> +static const struct vin_format_table vin_formats_table[] = {
> >> + /* VIN_LINE_WR */
> >> + { vin_formats_wr, ARRAY_SIZE(vin_formats_wr) },
> >> + /* VIN_LINE_ISP */
> >> + { vin_formats_uo, ARRAY_SIZE(vin_formats_uo) },
> >> +};
> >> +
> >> +static void vin_buffer_done(struct vin_line *line);
> >> +static void vin_change_buffer(struct vin_line *line);
> >> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output);
> >> +static void vin_output_init_addrs(struct vin_line *line);
> >> +static struct v4l2_mbus_framefmt *
> >> +__vin_get_format(struct vin_line *line,
> >> + struct v4l2_subdev_state *state,
> >> + unsigned int pad,
> >> + enum v4l2_subdev_format_whence which);
> >> +
> >> +static char *vin_get_line_subdevname(int line_id)
> >> +{
> >> + char *name = NULL;
> >> +
> >> + switch (line_id) {
> >> + case VIN_LINE_WR:
> >> + name = "wr";
> >> + break;
> >> + case VIN_LINE_ISP:
> >> + name = "isp0";
> >> + break;
> >> + default:
> >> + name = "unknown";
> >> + break;
> >> + }
> >> + return name;
> >> +}
> >> +
> >> +static enum isp_line_id vin_map_isp_line(enum vin_line_id line)
> >> +{
> >> + enum isp_line_id line_id;
> >> +
> >> + if (line > VIN_LINE_WR && line < VIN_LINE_MAX)
> >> + line_id = STF_ISP_LINE_SRC;
> >> + else
> >> + line_id = STF_ISP_LINE_INVALID;
> >> +
> >> + return line_id;
> >> +}
> >> +
> >> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def)
> >> +{
> >> + enum isp_pad_id pad_id;
> >> +
> >> + if (line == VIN_LINE_WR)
> >> + pad_id = STF_ISP_PAD_SINK;
> >> + else if ((line > VIN_LINE_WR) && (line < VIN_LINE_MAX))
> >> + pad_id = (enum isp_pad_id)vin_map_isp_line(line);
> >> + else
> >> + pad_id = def;
> >> +
> >> + return pad_id;
> >> +}
> >> +
> >> +int stf_vin_subdev_init(struct stfcamss *stfcamss)
> >> +{
> >> + struct device *dev = stfcamss->dev;
> >> + struct stf_vin_dev *vin_dev = &stfcamss->vin_dev;
> >> + int i, ret = 0;
> >> +
> >> + vin_dev->stfcamss = stfcamss;
> >> +
> >> + vin_dev->isr_ops = devm_kzalloc(dev, sizeof(*vin_dev->isr_ops),
> >> + GFP_KERNEL);
> >> + if (!vin_dev->isr_ops)
> >> + return -ENOMEM;
> >> + vin_dev->isr_ops->isr_buffer_done = vin_buffer_done;
> >> + vin_dev->isr_ops->isr_change_buffer = vin_change_buffer;
> >> +
> >> + atomic_set(&vin_dev->ref_count, 0);
> >> +
> >> + ret = devm_request_irq(dev,
> >> + stfcamss->irq[STF_IRQ_VINWR],
> >> + stf_vin_wr_irq_handler,
> >> + 0, "vin_axiwr_irq", vin_dev);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to request irq\n");
> >> + goto out;
> >> + }
> >> +
> >> + ret = devm_request_irq(dev,
> >> + stfcamss->irq[STF_IRQ_ISP],
> >> + stf_vin_isp_irq_handler,
> >> + 0, "vin_isp_irq", vin_dev);
> >> + if (ret) {
> >> + dev_err(dev, "Failed to request isp irq\n");
> >> + goto out;
> >> + }
> >> +
> >> + ret = devm_request_irq(dev,
> >> + stfcamss->irq[STF_IRQ_ISPCSIL],
> >> + stf_vin_isp_irq_csiline_handler,
> >> + 0, "vin_isp_irq_csiline", vin_dev);
> >> + if (ret) {
> >> + dev_err(dev, "failed to request isp irq csiline\n");
> >> + goto out;
> >> + }
> >> +
> >> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++) {
> >> + struct dummy_buffer *dummy_buffer = &vin_dev->dummy_buffer[i];
> >> +
> >> + mutex_init(&dummy_buffer->stream_lock);
> >> + dummy_buffer->nums =
> >> + i == 0 ? VIN_DUMMY_BUFFER_NUMS : ISP_DUMMY_BUFFER_NUMS;
> >> + dummy_buffer->stream_count = 0;
> >> + dummy_buffer->buffer =
> >> + devm_kzalloc(dev,
> >> + dummy_buffer->nums * sizeof(struct vin_dummy_buffer),
> >> + GFP_KERNEL);
> >> + atomic_set(&dummy_buffer->frame_skip, 0);
> >> + }
> >> +
> >> + for (i = VIN_LINE_WR; i < STF_ISP_LINE_MAX + 1; i++) {
> >> + struct vin_line *l = &vin_dev->line[i];
> >> +
> >> + l->video_out.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> >> + l->video_out.stfcamss = stfcamss;
> >> + l->id = i;
> >> + l->formats = vin_formats_table[i].fmts;
> >> + l->nformats = vin_formats_table[i].nfmts;
> >> + spin_lock_init(&l->output_lock);
> >> +
> >> + mutex_init(&l->stream_lock);
> >> + l->stream_count = 0;
> >> + }
> >> +
> >> + return 0;
> >> +out:
> >> + return ret;
> >> +}
> >> +
> >> +static enum link vin_get_link(struct media_entity *entity)
> >> +{
> >> + struct v4l2_subdev *subdev;
> >> + struct media_pad *pad;
> >> + bool isp = false;
> >> +
> >> + while (1) {
> >> + pad = &entity->pads[0];
> >> + if (!(pad->flags & MEDIA_PAD_FL_SINK))
> >> + return LINK_ERROR;
> >> +
> >> + pad = media_pad_remote_pad_first(pad);
> >> + if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
> >> + return LINK_ERROR;
> >> +
> >> + entity = pad->entity;
> >> + subdev = media_entity_to_v4l2_subdev(entity);
> >> +
> >> + if (!strncmp(subdev->name, STF_CSI_NAME,
> >> + strlen(STF_CSI_NAME))) {
> >> + if (isp)
> >> + return LINK_CSI_TO_ISP;
> >> + else
> >> + return LINK_CSI_TO_WR;
> >> + } else if (!strncmp(subdev->name, STF_DVP_NAME,
> >> + strlen(STF_DVP_NAME))) {
> >> + if (isp)
> >> + return LINK_DVP_TO_ISP;
> >> + else
> >> + return LINK_DVP_TO_WR;
> >> + } else if (!strncmp(subdev->name, STF_ISP_NAME,
> >> + strlen(STF_ISP_NAME))) {
> >> + isp = true;
> >> + } else {
> >> + return LINK_ERROR;
> >> + }
> >> + }
> >> +}
> >> +
> >> +static int vin_enable_output(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> + unsigned long flags;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + output->state = VIN_OUTPUT_IDLE;
> >> +
> >> + output->buf[0] = vin_buf_get_pending(output);
> >> +
> >> + if (!output->buf[0] && output->buf[1]) {
> >> + output->buf[0] = output->buf[1];
> >> + output->buf[1] = NULL;
> >> + }
> >> +
> >> + if (output->buf[0])
> >> + output->state = VIN_OUTPUT_SINGLE;
> >> +
> >> + output->sequence = 0;
> >> +
> >> + vin_output_init_addrs(line);
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_disable_output(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> + unsigned long flags;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + output->state = VIN_OUTPUT_OFF;
> >> +
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> + return 0;
> >> +}
> >> +
> >> +static u32 vin_line_to_dummy_module(struct vin_line *line)
> >> +{
> >> + u32 dummy_module = 0;
> >> +
> >> + switch (line->id) {
> >> + case VIN_LINE_WR:
> >> + dummy_module = STF_DUMMY_VIN;
> >> + break;
> >> + case VIN_LINE_ISP:
> >> + dummy_module = STF_DUMMY_ISP;
> >> + break;
> >> + default:
> >> + dummy_module = STF_DUMMY_VIN;
> >> + break;
> >> + }
> >> +
> >> + return dummy_module;
> >> +}
> >> +
> >> +static int vin_alloc_dummy_buffer(struct stf_vin_dev *vin_dev,
> >> + struct v4l2_mbus_framefmt *fmt,
> >> + int dummy_module)
> >> +{
> >> + struct device *dev = vin_dev->stfcamss->dev;
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[dummy_module];
> >> + struct vin_dummy_buffer *buffer = NULL;
> >> + int ret = 0, i;
> >> + u32 aligns;
> >> +
> >> + for (i = 0; i < dummy_buffer->nums; i++) {
> >> + buffer = &vin_dev->dummy_buffer[dummy_module].buffer[i];
> >> + buffer->width = fmt->width;
> >> + buffer->height = fmt->height;
> >> + buffer->mcode = fmt->code;
> >> + if (i == STF_VIN_PAD_SINK) {
> >> + aligns = ALIGN(fmt->width * 4,
> >> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
> >> + buffer->buffer_size = PAGE_ALIGN(aligns * fmt->height);
> >> + } else if (i == STF_ISP_PAD_SRC) {
> >> + aligns = ALIGN(fmt->width,
> >> + STFCAMSS_FRAME_WIDTH_ALIGN_8);
> >> + buffer->buffer_size =
> >> + PAGE_ALIGN(aligns * fmt->height * 3 / 2);
> >> + } else {
> >> + continue;
> >> + }
> >> +
> >> + buffer->vaddr = dma_alloc_coherent(dev,
> >> + buffer->buffer_size,
> >> + &buffer->paddr[0],
> >> + GFP_KERNEL | __GFP_NOWARN);
> >> +
> >> + if (buffer->vaddr) {
> >> + if (i == STF_ISP_PAD_SRC)
> >> + buffer->paddr[1] =
> >> + (dma_addr_t)(buffer->paddr[0] + aligns * fmt->height);
> >> + else
> >> + dev_dbg(dev, "signal plane\n");
> >> + }
> >> + }
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static void vin_free_dummy_buffer(struct stf_vin_dev *vin_dev, int dummy_module)
> >> +{
> >> + struct device *dev = vin_dev->stfcamss->dev;
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[dummy_module];
> >> + struct vin_dummy_buffer *buffer = NULL;
> >> + int i;
> >> +
> >> + for (i = 0; i < dummy_buffer->nums; i++) {
> >> + buffer = &dummy_buffer->buffer[i];
> >> + if (buffer->vaddr)
> >> + dma_free_coherent(dev, buffer->buffer_size,
> >> + buffer->vaddr, buffer->paddr[0]);
> >> + memset(buffer, 0, sizeof(struct vin_dummy_buffer));
> >> + }
> >> +}
> >> +
> >> +static void vin_set_dummy_buffer(struct vin_line *line, u32 pad)
> >> +{
> >> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> >> + int dummy_module = vin_line_to_dummy_module(line);
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[dummy_module];
> >> + struct vin_dummy_buffer *buffer = NULL;
> >> +
> >> + switch (pad) {
> >> + case STF_VIN_PAD_SINK:
> >> + if (line->id == VIN_LINE_WR) {
> >> + buffer = &dummy_buffer->buffer[STF_VIN_PAD_SINK];
> >> + stf_vin_wr_set_ping_addr(vin_dev, buffer->paddr[0]);
> >> + stf_vin_wr_set_pong_addr(vin_dev, buffer->paddr[0]);
> >> + } else {
> >> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
> >> + stf_vin_isp_set_yuv_addr(vin_dev,
> >> + buffer->paddr[0],
> >> + buffer->paddr[1]);
> >> + }
> >> + break;
> >> + case STF_ISP_PAD_SRC:
> >> + buffer = &dummy_buffer->buffer[STF_ISP_PAD_SRC];
> >> + stf_vin_isp_set_yuv_addr(vin_dev,
> >> + buffer->paddr[0],
> >> + buffer->paddr[1]);
> >> + break;
> >> + default:
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static int vin_set_stream(struct v4l2_subdev *sd, int enable)
> >> +{
> >> + struct vin_line *line = v4l2_get_subdevdata(sd);
> >> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> >> + int dummy_module = vin_line_to_dummy_module(line);
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[dummy_module];
> >> + struct v4l2_mbus_framefmt *fmt;
> >> + enum link link;
> >> +
> >> + fmt = __vin_get_format(line, NULL,
> >> + STF_VIN_PAD_SINK, V4L2_SUBDEV_FORMAT_ACTIVE);
> >> + mutex_lock(&dummy_buffer->stream_lock);
> >> + if (enable) {
> >> + if (dummy_buffer->stream_count == 0) {
> >> + vin_alloc_dummy_buffer(vin_dev, fmt, dummy_module);
> >> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
> >> + atomic_set(&dummy_buffer->frame_skip,
> >> + VIN_FRAME_DROP_MIN_VAL);
> >> + }
> >> + dummy_buffer->stream_count++;
> >> + } else {
> >> + if (dummy_buffer->stream_count == 1) {
> >> + vin_free_dummy_buffer(vin_dev, dummy_module);
> >> + /* set buffer addr to zero */
> >> + vin_set_dummy_buffer(line, STF_VIN_PAD_SINK);
> >> + } else {
> >> + vin_set_dummy_buffer(line,
> >> + stf_vin_map_isp_pad(line->id, STF_ISP_PAD_SINK));
> >> + }
> >> +
> >> + dummy_buffer->stream_count--;
> >> + }
> >> + mutex_unlock(&dummy_buffer->stream_lock);
> >> +
> >> + mutex_lock(&line->stream_lock);
> >> + link = vin_get_link(&sd->entity);
> >> + if (link == LINK_ERROR)
> >> + goto exit;
> >> +
> >> + if (enable) {
> >> + if (line->stream_count == 0) {
> >> + stf_vin_stream_set(vin_dev, link);
> >> + if (line->id == VIN_LINE_WR) {
> >> + stf_vin_wr_irq_enable(vin_dev, 1);
> >> + stf_vin_wr_stream_set(vin_dev);
> >> + }
> >> + }
> >> + line->stream_count++;
> >> + } else {
> >> + if (line->stream_count == 1 && line->id == VIN_LINE_WR)
> >> + stf_vin_wr_irq_enable(vin_dev, 0);
> >> + line->stream_count--;
> >> + }
> >> +exit:
> >> + mutex_unlock(&line->stream_lock);
> >> +
> >> + if (enable)
> >> + vin_enable_output(line);
> >> + else
> >> + vin_disable_output(line);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static struct v4l2_mbus_framefmt *
> >> +__vin_get_format(struct vin_line *line,
> >> + struct v4l2_subdev_state *state,
> >> + unsigned int pad,
> >> + enum v4l2_subdev_format_whence which)
> >> +{
> >> + if (which == V4L2_SUBDEV_FORMAT_TRY)
> >> + return v4l2_subdev_get_try_format(&line->subdev, state, pad);
> >> + return &line->fmt[pad];
> >> +}
> >> +
> >> +static void vin_try_format(struct vin_line *line,
> >> + struct v4l2_subdev_state *state,
> >> + unsigned int pad,
> >> + struct v4l2_mbus_framefmt *fmt,
> >> + enum v4l2_subdev_format_whence which)
> >> +{
> >> + unsigned int i;
> >> +
> >> + switch (pad) {
> >> + case STF_VIN_PAD_SINK:
> >> + /* Set format on sink pad */
> >> + for (i = 0; i < line->nformats; i++)
> >> + if (fmt->code == line->formats[i].code)
> >> + break;
> >> +
> >> + /* If not found, use UYVY as default */
> >> + if (i >= line->nformats)
> >> + fmt->code = line->formats[0].code;
> >> +
> >> + fmt->width = clamp_t(u32, fmt->width,
> >> + STFCAMSS_FRAME_MIN_WIDTH,
> >> + STFCAMSS_FRAME_MAX_WIDTH);
> >> + fmt->height = clamp_t(u32, fmt->height,
> >> + STFCAMSS_FRAME_MIN_HEIGHT,
> >> + STFCAMSS_FRAME_MAX_HEIGHT);
> >> +
> >> + fmt->field = V4L2_FIELD_NONE;
> >> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> >> + fmt->flags = 0;
> >> + break;
> >> +
> >> + case STF_VIN_PAD_SRC:
> >> + /* Set and return a format same as sink pad */
> >> + *fmt = *__vin_get_format(line, state, STF_VIN_PAD_SINK, which);
> >> + break;
> >> + }
> >> +
> >> + fmt->colorspace = V4L2_COLORSPACE_SRGB;
> >> +}
> >> +
> >> +static int vin_enum_mbus_code(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> + struct vin_line *line = v4l2_get_subdevdata(sd);
> >> +
> >> + if (code->index >= line->nformats)
> >> + return -EINVAL;
> >> + if (code->pad == STF_VIN_PAD_SINK) {
> >> + code->code = line->formats[code->index].code;
> >> + } else {
> >> + struct v4l2_mbus_framefmt *sink_fmt;
> >> +
> >> + sink_fmt = __vin_get_format(line, state, STF_VIN_PAD_SINK,
> >> + code->which);
> >> +
> >> + code->code = sink_fmt->code;
> >> + if (!code->code)
> >> + return -EINVAL;
> >> + }
> >> + code->flags = 0;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_enum_frame_size(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_frame_size_enum *fse)
> >> +{
> >> + struct vin_line *line = v4l2_get_subdevdata(sd);
> >> + struct v4l2_mbus_framefmt format;
> >> +
> >> + if (fse->index != 0)
> >> + return -EINVAL;
> >> +
> >> + format.code = fse->code;
> >> + format.width = 1;
> >> + format.height = 1;
> >> + vin_try_format(line, state, fse->pad, &format, fse->which);
> >> + fse->min_width = format.width;
> >> + fse->min_height = format.height;
> >> +
> >> + if (format.code != fse->code)
> >> + return -EINVAL;
> >> +
> >> + format.code = fse->code;
> >> + format.width = -1;
> >> + format.height = -1;
> >> + vin_try_format(line, state, fse->pad, &format, fse->which);
> >> + fse->max_width = format.width;
> >> + fse->max_height = format.height;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_get_format(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_format *fmt)
> >> +{
> >> + struct vin_line *line = v4l2_get_subdevdata(sd);
> >> + struct v4l2_mbus_framefmt *format;
> >> +
> >> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
> >> + if (!format)
> >> + return -EINVAL;
> >> +
> >> + fmt->format = *format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_set_format(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_format *fmt)
> >> +{
> >> + struct vin_line *line = v4l2_get_subdevdata(sd);
> >> + struct v4l2_mbus_framefmt *format;
> >> +
> >> + format = __vin_get_format(line, state, fmt->pad, fmt->which);
> >> + if (!format)
> >> + return -EINVAL;
> >> +
> >> + mutex_lock(&line->stream_lock);
> >> + if (line->stream_count) {
> >> + fmt->format = *format;
> >> + mutex_unlock(&line->stream_lock);
> >> + goto out;
> >> + } else {
> >> + vin_try_format(line, state, fmt->pad, &fmt->format, fmt->which);
> >> + *format = fmt->format;
> >> + }
> >> + mutex_unlock(&line->stream_lock);
> >> +
> >> + if (fmt->pad == STF_VIN_PAD_SINK) {
> >> + /* Propagate the format from sink to source */
> >> + format = __vin_get_format(line, state, STF_VIN_PAD_SRC,
> >> + fmt->which);
> >> +
> >> + *format = fmt->format;
> >> + vin_try_format(line, state, STF_VIN_PAD_SRC, format,
> >> + fmt->which);
> >> + }
> >> +
> >> +out:
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_init_formats(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_fh *fh)
> >> +{
> >> + struct v4l2_subdev_format format = {
> >> + .pad = STF_VIN_PAD_SINK,
> >> + .which = fh ?
> >> + V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE,
> >> + .format = {
> >> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> >> + .width = 1920,
> >> + .height = 1080
> >> + }
> >> + };
> >> +
> >> + return vin_set_format(sd, fh ? fh->state : NULL, &format);
> >> +}
> >> +
> >> +static void vin_output_init_addrs(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> >> + dma_addr_t ping_addr;
> >> + dma_addr_t y_addr, uv_addr;
> >> +
> >> + output->active_buf = 0;
> >> +
> >> + if (output->buf[0]) {
> >> + ping_addr = output->buf[0]->addr[0];
> >> + y_addr = output->buf[0]->addr[0];
> >> + uv_addr = output->buf[0]->addr[1];
> >> + } else {
> >> + return;
> >> + }
> >> +
> >> + switch (vin_map_isp_line(line->id)) {
> >> + case STF_ISP_LINE_SRC:
> >> + stf_vin_isp_set_yuv_addr(vin_dev, y_addr, uv_addr);
> >> + break;
> >> + default:
> >> + if (line->id == VIN_LINE_WR) {
> >> + stf_vin_wr_set_ping_addr(vin_dev, ping_addr);
> >> + stf_vin_wr_set_pong_addr(vin_dev, ping_addr);
> >> + }
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static void vin_init_outputs(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> +
> >> + output->state = VIN_OUTPUT_OFF;
> >> + output->buf[0] = NULL;
> >> + output->buf[1] = NULL;
> >> + output->active_buf = 0;
> >> + INIT_LIST_HEAD(&output->pending_bufs);
> >> + INIT_LIST_HEAD(&output->ready_bufs);
> >> +}
> >> +
> >> +static void vin_buf_add_ready(struct vin_output *output,
> >> + struct stfcamss_buffer *buffer)
> >> +{
> >> + INIT_LIST_HEAD(&buffer->queue);
> >> + list_add_tail(&buffer->queue, &output->ready_bufs);
> >> +}
> >> +
> >> +static struct stfcamss_buffer *vin_buf_get_ready(struct vin_output *output)
> >> +{
> >> + struct stfcamss_buffer *buffer = NULL;
> >> +
> >> + if (!list_empty(&output->ready_bufs)) {
> >> + buffer = list_first_entry(&output->ready_bufs,
> >> + struct stfcamss_buffer,
> >> + queue);
> >> + list_del(&buffer->queue);
> >> + }
> >> +
> >> + return buffer;
> >> +}
> >> +
> >> +static void vin_buf_add_pending(struct vin_output *output,
> >> + struct stfcamss_buffer *buffer)
> >> +{
> >> + INIT_LIST_HEAD(&buffer->queue);
> >> + list_add_tail(&buffer->queue, &output->pending_bufs);
> >> +}
> >> +
> >> +static struct stfcamss_buffer *vin_buf_get_pending(struct vin_output *output)
> >> +{
> >> + struct stfcamss_buffer *buffer = NULL;
> >> +
> >> + if (!list_empty(&output->pending_bufs)) {
> >> + buffer = list_first_entry(&output->pending_bufs,
> >> + struct stfcamss_buffer,
> >> + queue);
> >> + list_del(&buffer->queue);
> >> + }
> >> +
> >> + return buffer;
> >> +}
> >> +
> >> +static void vin_buf_update_on_last(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> +
> >> + switch (output->state) {
> >> + case VIN_OUTPUT_CONTINUOUS:
> >> + output->state = VIN_OUTPUT_SINGLE;
> >> + output->active_buf = !output->active_buf;
> >> + break;
> >> + case VIN_OUTPUT_SINGLE:
> >> + output->state = VIN_OUTPUT_STOPPING;
> >> + break;
> >> + default:
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static void vin_buf_update_on_next(struct vin_line *line)
> >> +{
> >> + struct vin_output *output = &line->output;
> >> +
> >> + switch (output->state) {
> >> + case VIN_OUTPUT_CONTINUOUS:
> >> + output->active_buf = !output->active_buf;
> >> + break;
> >> + case VIN_OUTPUT_SINGLE:
> >> + default:
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static void vin_buf_update_on_new(struct vin_line *line,
> >> + struct vin_output *output,
> >> + struct stfcamss_buffer *new_buf)
> >> +{
> >> + switch (output->state) {
> >> + case VIN_OUTPUT_SINGLE:
> >> + vin_buf_add_pending(output, new_buf);
> >> + break;
> >> + case VIN_OUTPUT_IDLE:
> >> + if (!output->buf[0]) {
> >> + output->buf[0] = new_buf;
> >> + vin_output_init_addrs(line);
> >> + output->state = VIN_OUTPUT_SINGLE;
> >> + } else {
> >> + vin_buf_add_pending(output, new_buf);
> >> + }
> >> + break;
> >> + case VIN_OUTPUT_STOPPING:
> >> + if (output->last_buffer) {
> >> + output->buf[output->active_buf] = output->last_buffer;
> >> + output->last_buffer = NULL;
> >> + }
> >> +
> >> + output->state = VIN_OUTPUT_SINGLE;
> >> + vin_buf_add_pending(output, new_buf);
> >> + break;
> >> + case VIN_OUTPUT_CONTINUOUS:
> >> + default:
> >> + vin_buf_add_pending(output, new_buf);
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static void vin_buf_flush(struct vin_output *output,
> >> + enum vb2_buffer_state state)
> >> +{
> >> + struct stfcamss_buffer *buf;
> >> + struct stfcamss_buffer *t;
> >> +
> >> + list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
> >> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> >> + list_del(&buf->queue);
> >> + }
> >> + list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
> >> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> >> + list_del(&buf->queue);
> >> + }
> >> +}
> >> +
> >> +static void vin_buffer_done(struct vin_line *line)
> >> +{
> >> + struct stfcamss_buffer *ready_buf;
> >> + struct vin_output *output = &line->output;
> >> + unsigned long flags;
> >> + u64 ts = ktime_get_ns();
> >> +
> >> + if (output->state == VIN_OUTPUT_OFF ||
> >> + output->state == VIN_OUTPUT_RESERVED)
> >> + return;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + while ((ready_buf = vin_buf_get_ready(output))) {
> >> + ready_buf->vb.vb2_buf.timestamp = ts;
> >> + ready_buf->vb.sequence = output->sequence++;
> >> +
> >> + vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> >> + }
> >> +
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> +}
> >> +
> >> +static void vin_change_buffer(struct vin_line *line)
> >> +{
> >> + struct stfcamss_buffer *ready_buf;
> >> + struct vin_output *output = &line->output;
> >> + struct stf_vin_dev *vin_dev = line_to_vin_dev(line);
> >> + dma_addr_t *new_addr;
> >> + unsigned long flags;
> >> + u32 active_index;
> >> +
> >> + if (output->state == VIN_OUTPUT_OFF ||
> >> + output->state == VIN_OUTPUT_STOPPING ||
> >> + output->state == VIN_OUTPUT_RESERVED ||
> >> + output->state == VIN_OUTPUT_IDLE)
> >> + return;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + active_index = output->active_buf;
> >> +
> >> + ready_buf = output->buf[active_index];
> >> + if (!ready_buf) {
> >> + dev_warn(vin_dev->stfcamss->dev, "Missing ready buf %d %d!\n",
> >> + active_index, output->state);
> >> + active_index = !active_index;
> >> + ready_buf = output->buf[active_index];
> >> + if (!ready_buf) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Missing ready buf2 %d %d!\n",
> >> + active_index, output->state);
> >> + goto out_unlock;
> >> + }
> >> + }
> >> +
> >> + /* Get next buffer */
> >> + output->buf[active_index] = vin_buf_get_pending(output);
> >> + if (!output->buf[active_index]) {
> >> + /* No next buffer - set same address */
> >> + new_addr = ready_buf->addr;
> >> + vin_buf_update_on_last(line);
> >> + } else {
> >> + new_addr = output->buf[active_index]->addr;
> >> + vin_buf_update_on_next(line);
> >> + }
> >> +
> >> + if (output->state == VIN_OUTPUT_STOPPING) {
> >> + output->last_buffer = ready_buf;
> >> + } else {
> >> + switch (vin_map_isp_line(line->id)) {
> >> + case STF_ISP_LINE_SRC:
> >> + stf_vin_isp_set_yuv_addr(vin_dev,
> >> + new_addr[0],
> >> + new_addr[1]);
> >> + break;
> >> + default:
> >> + if (line->id == VIN_LINE_WR) {
> >> + stf_vin_wr_set_ping_addr(vin_dev, new_addr[0]);
> >> + stf_vin_wr_set_pong_addr(vin_dev, new_addr[0]);
> >> + }
> >> + break;
> >> + }
> >> +
> >> + vin_buf_add_ready(output, ready_buf);
> >> + }
> >> +
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> + return;
> >> +
> >> +out_unlock:
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> +}
> >> +
> >> +static int vin_queue_buffer(struct stfcamss_video *vid,
> >> + struct stfcamss_buffer *buf)
> >> +{
> >> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
> >> + struct vin_output *output;
> >> + unsigned long flags;
> >> +
> >> + output = &line->output;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + vin_buf_update_on_new(line, output, buf);
> >> +
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_flush_buffers(struct stfcamss_video *vid,
> >> + enum vb2_buffer_state state)
> >> +{
> >> + struct vin_line *line = container_of(vid, struct vin_line, video_out);
> >> + struct vin_output *output = &line->output;
> >> + unsigned long flags;
> >> +
> >> + spin_lock_irqsave(&line->output_lock, flags);
> >> +
> >> + vin_buf_flush(output, state);
> >> + if (output->buf[0])
> >> + vb2_buffer_done(&output->buf[0]->vb.vb2_buf, state);
> >> +
> >> + if (output->buf[1])
> >> + vb2_buffer_done(&output->buf[1]->vb.vb2_buf, state);
> >> +
> >> + if (output->last_buffer) {
> >> + vb2_buffer_done(&output->last_buffer->vb.vb2_buf, state);
> >> + output->last_buffer = NULL;
> >> + }
> >> + output->buf[0] = NULL;
> >> + output->buf[1] = NULL;
> >> +
> >> + spin_unlock_irqrestore(&line->output_lock, flags);
> >> + return 0;
> >> +}
> >> +
> >> +static int vin_link_setup(struct media_entity *entity,
> >> + const struct media_pad *local,
> >> + const struct media_pad *remote, u32 flags)
> >> +{
> >> + if (flags & MEDIA_LNK_FL_ENABLED)
> >> + if (media_pad_remote_pad_first(local))
> >> + return -EBUSY;
> >> + return 0;
> >> +}
> >> +
> >> +static const struct v4l2_subdev_video_ops vin_video_ops = {
> >> + .s_stream = vin_set_stream,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_pad_ops vin_pad_ops = {
> >> + .enum_mbus_code = vin_enum_mbus_code,
> >> + .enum_frame_size = vin_enum_frame_size,
> >> + .get_fmt = vin_get_format,
> >> + .set_fmt = vin_set_format,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_ops vin_v4l2_ops = {
> >> + .video = &vin_video_ops,
> >> + .pad = &vin_pad_ops,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_internal_ops vin_v4l2_internal_ops = {
> >> + .open = vin_init_formats,
> >> +};
> >> +
> >> +static const struct stfcamss_video_ops stfcamss_vin_video_ops = {
> >> + .queue_buffer = vin_queue_buffer,
> >> + .flush_buffers = vin_flush_buffers,
> >> +};
> >> +
> >> +static const struct media_entity_operations vin_media_ops = {
> >> + .link_setup = vin_link_setup,
> >> + .link_validate = v4l2_subdev_link_validate,
> >> +};
> >> +
> >> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev)
> >> +{
> >> + struct v4l2_subdev *sd;
> >> + struct stfcamss_video *video_out;
> >> + struct media_pad *pads;
> >> + int ret;
> >> + int i;
> >> +
> >> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
> >> + char name[32];
> >> + char *sub_name = vin_get_line_subdevname(i);
> >> +
> >> + sd = &vin_dev->line[i].subdev;
> >> + pads = vin_dev->line[i].pads;
> >> + video_out = &vin_dev->line[i].video_out;
> >> + video_out->id = i;
> >> +
> >> + vin_init_outputs(&vin_dev->line[i]);
> >> +
> >> + v4l2_subdev_init(sd, &vin_v4l2_ops);
> >> + sd->internal_ops = &vin_v4l2_internal_ops;
> >> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >> + snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d_%s",
> >> + STF_VIN_NAME, 0, sub_name);
> >> + v4l2_set_subdevdata(sd, &vin_dev->line[i]);
> >> +
> >> + ret = vin_init_formats(sd, NULL);
> >> + if (ret < 0) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Failed to init format: %d\n", ret);
> >> + goto err_init;
> >> + }
> >> +
> >> + pads[STF_VIN_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> + pads[STF_VIN_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> >> +
> >> + sd->entity.function =
> >> + MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> >> + sd->entity.ops = &vin_media_ops;
> >> + ret = media_entity_pads_init(&sd->entity,
> >> + STF_VIN_PADS_NUM, pads);
> >> + if (ret < 0) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Failed to init media entity: %d\n",
> >> + ret);
> >> + goto err_init;
> >> + }
> >> +
> >> + ret = v4l2_device_register_subdev(v4l2_dev, sd);
> >> + if (ret < 0) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Failed to register subdev: %d\n", ret);
> >> + goto err_reg_subdev;
> >> + }
> >> +
> >> + video_out->ops = &stfcamss_vin_video_ops;
> >> + video_out->bpl_alignment = 16 * 8;
> >> +
> >> + snprintf(name, ARRAY_SIZE(name), "%s_%s%d",
> >> + sd->name, "video", i);
> >> + ret = stf_video_register(video_out, v4l2_dev, name);
> >> + if (ret < 0) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Failed to register video node: %d\n", ret);
> >> + goto err_vid_reg;
> >> + }
> >> +
> >> + ret = media_create_pad_link(
> >> + &sd->entity, STF_VIN_PAD_SRC,
> >> + &video_out->vdev.entity, 0,
> >> + MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> >> + if (ret < 0) {
> >> + dev_err(vin_dev->stfcamss->dev,
> >> + "Failed to link %s->%s entities: %d\n",
> >> + sd->entity.name, video_out->vdev.entity.name,
> >> + ret);
> >> + goto err_create_link;
> >> + }
> >> + }
> >> +
> >> + return 0;
> >> +
> >> +err_create_link:
> >> + stf_video_unregister(video_out);
> >> +err_vid_reg:
> >> + v4l2_device_unregister_subdev(sd);
> >> +err_reg_subdev:
> >> + media_entity_cleanup(&sd->entity);
> >> +err_init:
> >> + for (i--; i >= 0; i--) {
> >> + sd = &vin_dev->line[i].subdev;
> >> + video_out = &vin_dev->line[i].video_out;
> >> +
> >> + stf_video_unregister(video_out);
> >> + v4l2_device_unregister_subdev(sd);
> >> + media_entity_cleanup(&sd->entity);
> >> + }
> >> + return ret;
> >> +}
> >> +
> >> +int stf_vin_unregister(struct stf_vin_dev *vin_dev)
> >> +{
> >> + struct v4l2_subdev *sd;
> >> + struct stfcamss_video *video_out;
> >> + int i;
> >> +
> >> + for (i = 0; i < STF_DUMMY_MODULE_NUMS; i++)
> >> + mutex_destroy(&vin_dev->dummy_buffer[i].stream_lock);
> >> +
> >> + for (i = 0; i < STF_ISP_LINE_MAX + 1; i++) {
> >> + sd = &vin_dev->line[i].subdev;
> >> + video_out = &vin_dev->line[i].video_out;
> >> +
> >> + stf_video_unregister(video_out);
> >> + v4l2_device_unregister_subdev(sd);
> >> + media_entity_cleanup(&sd->entity);
> >> + mutex_destroy(&vin_dev->line[i].stream_lock);
> >> + }
> >> + return 0;
> >> +}
> >> diff --git a/drivers/media/platform/starfive/camss/stf_vin.h b/drivers/media/platform/starfive/camss/stf_vin.h
> >> new file mode 100644
> >> index 000000000000..28572eb6abe4
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_vin.h
> >> @@ -0,0 +1,173 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> +/*
> >> + * stf_vin.h
> >> + *
> >> + * StarFive Camera Subsystem - VIN Module
> >> + *
> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> + */
> >> +
> >> +#ifndef STF_VIN_H
> >> +#define STF_VIN_H
> >> +
> >> +#include <linux/interrupt.h>
> >> +#include <linux/spinlock_types.h>
> >> +#include <media/v4l2-subdev.h>
> >> +
> >> +#include "stf_video.h"
> >> +
> >> +#define SYSCONSAIF_SYSCFG(x) (x)
> >> +
> >> +/* syscon offset 0 */
> >> +#define U0_VIN_CNFG_AXI_DVP_EN BIT(2)
> >> +
> >> +/* syscon offset 20 */
> >> +#define U0_VIN_CHANNEL_SEL_MASK GENMASK(3, 0)
> >> +#define U0_VIN_AXIWR0_EN BIT(4)
> >> +#define CHANNEL(x) ((x) << 0)
> >> +
> >> +/* syscon offset 32 */
> >> +#define U0_VIN_INTR_CLEAN BIT(0)
> >> +#define U0_VIN_INTR_M BIT(1)
> >> +#define U0_VIN_PIX_CNT_END_MASK GENMASK(12, 2)
> >> +#define U0_VIN_PIX_CT_MASK GENMASK(14, 13)
> >> +#define U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS GENMASK(16, 15)
> >> +
> >> +#define PIX_CNT_END(x) ((x) << 2)
> >> +#define PIX_CT(x) ((x) << 13)
> >> +#define PIXEL_HEIGH_BIT_SEL(x) ((x) << 15)
> >> +
> >> +/* syscon offset 36 */
> >> +#define U0_VIN_CNFG_DVP_HS_POS BIT(1)
> >> +#define U0_VIN_CNFG_DVP_SWAP_EN BIT(2)
> >> +#define U0_VIN_CNFG_DVP_VS_POS BIT(3)
> >> +#define U0_VIN_CNFG_GEN_EN_AXIRD BIT(4)
> >> +#define U0_VIN_CNFG_ISP_DVP_EN0 BIT(5)
> >> +#define U0_VIN_MIPI_BYTE_EN_ISP0(n) ((n) << 6)
> >> +#define U0_VIN_MIPI_CHANNEL_SEL0(n) ((n) << 8)
> >> +#define U0_VIN_P_I_MIPI_HAEDER_EN0(n) ((n) << 12)
> >> +#define U0_VIN_PIX_NUM(n) ((n) << 13)
> >> +#define U0_VIN_MIPI_BYTE_EN_ISP0_MASK GENMASK(7, 6)
> >> +#define U0_VIN_MIPI_CHANNEL_SEL0_MASK GENMASK(11, 8)
> >> +#define U0_VIN_P_I_MIPI_HAEDER_EN0_MASK BIT(12)
> >> +#define U0_VIN_PIX_NUM_MASK GENMASK(16, 13)
> >> +
> >> +#define STF_VIN_PAD_SINK 0
> >> +#define STF_VIN_PAD_SRC 1
> >> +#define STF_VIN_PADS_NUM 2
> >> +
> >> +#define ISP_DUMMY_BUFFER_NUMS STF_ISP_PAD_MAX
> >> +#define VIN_DUMMY_BUFFER_NUMS 1
> >> +
> >> +enum {
> >> + STF_DUMMY_VIN,
> >> + STF_DUMMY_ISP,
> >> + STF_DUMMY_MODULE_NUMS,
> >> +};
> >> +
> >> +enum link {
> >> + LINK_ERROR = -1,
> >> + LINK_DVP_TO_WR,
> >> + LINK_DVP_TO_ISP,
> >> + LINK_CSI_TO_WR,
> >> + LINK_CSI_TO_ISP,
> >> +};
> >> +
> >> +struct vin_format {
> >> + u32 code;
> >> + u8 bpp;
> >> +};
> >> +
> >> +struct vin_format_table {
> >> + const struct vin_format *fmts;
> >> + int nfmts;
> >> +};
> >> +
> >> +enum vin_output_state {
> >> + VIN_OUTPUT_OFF,
> >> + VIN_OUTPUT_RESERVED,
> >> + VIN_OUTPUT_SINGLE,
> >> + VIN_OUTPUT_CONTINUOUS,
> >> + VIN_OUTPUT_IDLE,
> >> + VIN_OUTPUT_STOPPING
> >> +};
> >> +
> >> +struct vin_output {
> >> + int active_buf;
> >> + struct stfcamss_buffer *buf[2];
> >> + struct stfcamss_buffer *last_buffer;
> >> + struct list_head pending_bufs;
> >> + struct list_head ready_bufs;
> >> + enum vin_output_state state;
> >> + unsigned int sequence;
> >> + unsigned int frame_skip;
> >> +};
> >> +
> >> +/* The vin output lines */
> >> +enum vin_line_id {
> >> + VIN_LINE_NONE = -1,
> >> + VIN_LINE_WR = 0,
> >> + VIN_LINE_ISP,
> >> + VIN_LINE_MAX,
> >> +};
> >> +
> >> +struct vin_line {
> >> + enum vin_line_id id;
> >> + struct v4l2_subdev subdev;
> >> + struct media_pad pads[STF_VIN_PADS_NUM];
> >> + struct v4l2_mbus_framefmt fmt[STF_VIN_PADS_NUM];
> >> + struct stfcamss_video video_out;
> >> + struct mutex stream_lock; /* serialize stream control */
> >> + int stream_count;
> >> + struct vin_output output; /* pipeline and stream states */
> >> + spinlock_t output_lock;
> >> + const struct vin_format *formats;
> >> + unsigned int nformats;
> >> +};
> >> +
> >> +struct vin_dummy_buffer {
> >> + dma_addr_t paddr[3];
> >> + void *vaddr;
> >> + u32 buffer_size;
> >> + u32 width;
> >> + u32 height;
> >> + u32 mcode;
> >> +};
> >> +
> >> +struct dummy_buffer {
> >> + struct vin_dummy_buffer *buffer;
> >> + u32 nums;
> >> + struct mutex stream_lock; /* protects buffer data */
> >> + int stream_count;
> >> + atomic_t frame_skip;
> >> +};
> >> +
> >> +struct vin_isr_ops {
> >> + void (*isr_buffer_done)(struct vin_line *line);
> >> + void (*isr_change_buffer)(struct vin_line *line);
> >> +};
> >> +
> >> +struct stf_vin_dev {
> >> + struct stfcamss *stfcamss;
> >> + struct vin_line line[VIN_LINE_MAX];
> >> + struct dummy_buffer dummy_buffer[STF_DUMMY_MODULE_NUMS];
> >> + struct vin_isr_ops *isr_ops;
> >> + atomic_t ref_count;
> >> +};
> >> +
> >> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev);
> >> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link);
> >> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable);
> >> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
> >> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr);
> >> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
> >> + dma_addr_t y_addr, dma_addr_t uv_addr);
> >> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv);
> >> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv);
> >> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv);
> >> +int stf_vin_subdev_init(struct stfcamss *stfcamss);
> >> +int stf_vin_register(struct stf_vin_dev *vin_dev, struct v4l2_device *v4l2_dev);
> >> +int stf_vin_unregister(struct stf_vin_dev *vin_dev);
> >> +enum isp_pad_id stf_vin_map_isp_pad(enum vin_line_id line, enum isp_pad_id def);
> >> +
> >> +#endif /* STF_VIN_H */
> >> diff --git a/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> >> new file mode 100644
> >> index 000000000000..7bd3265128d0
> >> --- /dev/null
> >> +++ b/drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> >> @@ -0,0 +1,192 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * stf_vin_hw_ops.c
> >> + *
> >> + * Register interface file for StarFive VIN module driver
> >> + *
> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> + */
> >> +#include "stf_camss.h"
> >> +
> >> +static void vin_intr_clear(struct stfcamss *stfcamss)
> >> +{
> >> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_CLEAN);
> >> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_CLEAN);
> >> +}
> >> +
> >> +irqreturn_t stf_vin_wr_irq_handler(int irq, void *priv)
> >> +{
> >> + struct stf_vin_dev *vin_dev = priv;
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[STF_DUMMY_VIN];
> >> +
> >> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
> >> + vin_dev->isr_ops->isr_change_buffer(&vin_dev->line[VIN_LINE_WR]);
> >> + vin_dev->isr_ops->isr_buffer_done(&vin_dev->line[VIN_LINE_WR]);
> >> + }
> >> +
> >> + vin_intr_clear(stfcamss);
> >> +
> >> + return IRQ_HANDLED;
> >> +}
> >> +
> >> +irqreturn_t stf_vin_isp_irq_handler(int irq, void *priv)
> >> +{
> >> + struct stf_vin_dev *vin_dev = priv;
> >> + u32 int_status;
> >> +
> >> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
> >> +
> >> + if (int_status & ISPC_INTS) {
> >> + if ((int_status & ISPC_ENUO))
> >> + vin_dev->isr_ops->isr_buffer_done(
> >> + &vin_dev->line[VIN_LINE_ISP]);
> >> +
> >> + /* clear interrupt */
> >> + stf_isp_reg_write(vin_dev->stfcamss,
> >> + ISP_REG_ISP_CTRL_0,
> >> + (int_status & ~EN_INT_ALL) |
> >> + EN_INT_ISP_DONE |
> >> + EN_INT_CSI_DONE |
> >> + EN_INT_SC_DONE);
> >> + }
> >> +
> >> + return IRQ_HANDLED;
> >> +}
> >> +
> >> +irqreturn_t stf_vin_isp_irq_csiline_handler(int irq, void *priv)
> >> +{
> >> + struct stf_vin_dev *vin_dev = priv;
> >> + struct stf_isp_dev *isp_dev;
> >> + u32 int_status;
> >> +
> >> + isp_dev = &vin_dev->stfcamss->isp_dev;
> >> +
> >> + int_status = stf_isp_reg_read(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0);
> >> + if (int_status & ISPC_SCFEINT) {
> >> + struct dummy_buffer *dummy_buffer =
> >> + &vin_dev->dummy_buffer[STF_DUMMY_ISP];
> >> +
> >> + if (atomic_dec_if_positive(&dummy_buffer->frame_skip) < 0) {
> >> + if ((int_status & ISPC_ENUO))
> >> + vin_dev->isr_ops->isr_change_buffer(
> >> + &vin_dev->line[VIN_LINE_ISP]);
> >> + }
> >> +
> >> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_CSIINTS,
> >> + CSI_INTS_MASK, CSI_INTS(0x3));
> >> + stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_IESHD,
> >> + SHAD_UP_M | SHAD_UP_EN, 0x3);
> >> +
> >> + /* clear interrupt */
> >> + stf_isp_reg_write(vin_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> >> + (int_status & ~EN_INT_ALL) | EN_INT_LINE_INT);
> >> + }
> >> +
> >> + return IRQ_HANDLED;
> >> +}
> >> +
> >> +int stf_vin_wr_stream_set(struct stf_vin_dev *vin_dev)
> >> +{
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> +
> >> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(20),
> >> + U0_VIN_AXIWR0_EN);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int stf_vin_stream_set(struct stf_vin_dev *vin_dev, enum link link)
> >> +{
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> + u32 val;
> >> +
> >> + switch (link) {
> >> + case LINK_CSI_TO_WR:
> >> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(20));
> >> + val &= ~U0_VIN_CHANNEL_SEL_MASK;
> >> + val |= CHANNEL(0);
> >> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(20), val);
> >> +
> >> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(28));
> >> + val &= ~U0_VIN_PIX_CT_MASK;
> >> + val |= PIX_CT(1);
> >> +
> >> + val &= ~U0_VIN_PIXEL_HEIGH_BIT_SEL_MAKS;
> >> + val |= PIXEL_HEIGH_BIT_SEL(0);
> >> +
> >> + val &= ~U0_VIN_PIX_CNT_END_MASK;
> >> + val |= PIX_CNT_END(IMAGE_MAX_WIDTH / 4 - 1);
> >> +
> >> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(28), val);
> >> + break;
> >> + case LINK_CSI_TO_ISP:
> >> + val = stf_syscon_reg_read(stfcamss, SYSCONSAIF_SYSCFG(36));
> >> + val &= ~U0_VIN_MIPI_BYTE_EN_ISP0_MASK;
> >> + val |= U0_VIN_MIPI_BYTE_EN_ISP0(0);
> >> +
> >> + val &= ~U0_VIN_MIPI_CHANNEL_SEL0_MASK;
> >> + val |= U0_VIN_MIPI_CHANNEL_SEL0(0);
> >> +
> >> + val &= ~U0_VIN_PIX_NUM_MASK;
> >> + val |= U0_VIN_PIX_NUM(0);
> >> +
> >> + val &= ~U0_VIN_P_I_MIPI_HAEDER_EN0_MASK;
> >> + val |= U0_VIN_P_I_MIPI_HAEDER_EN0(1);
> >> +
> >> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(36), val);
> >> + break;
> >> + case LINK_DVP_TO_WR:
> >> + case LINK_DVP_TO_ISP:
> >> + default:
> >> + return -EINVAL;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +void stf_vin_wr_irq_enable(struct stf_vin_dev *vin_dev, int enable)
> >> +{
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> +
> >> + if (enable) {
> >> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_M);
> >> + } else {
> >> + /* clear vin interrupt */
> >> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_CLEAN);
> >> + stf_syscon_reg_clear_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_CLEAN);
> >> + stf_syscon_reg_set_bit(stfcamss, SYSCONSAIF_SYSCFG(28),
> >> + U0_VIN_INTR_M);
> >> + }
> >> +}
> >> +
> >> +void stf_vin_wr_set_ping_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
> >> +{
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> +
> >> + /* set the start address */
> >> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(32), (long)addr);
> >> +}
> >> +
> >> +void stf_vin_wr_set_pong_addr(struct stf_vin_dev *vin_dev, dma_addr_t addr)
> >> +{
> >> + struct stfcamss *stfcamss = vin_dev->stfcamss;
> >> +
> >> + /* set the start address */
> >> + stf_syscon_reg_write(stfcamss, SYSCONSAIF_SYSCFG(24), (long)addr);
> >> +}
> >> +
> >> +void stf_vin_isp_set_yuv_addr(struct stf_vin_dev *vin_dev,
> >> + dma_addr_t y_addr, dma_addr_t uv_addr)
> >> +{
> >> + stf_isp_reg_write(vin_dev->stfcamss,
> >> + ISP_REG_Y_PLANE_START_ADDR, y_addr);
> >> + stf_isp_reg_write(vin_dev->stfcamss,
> >> + ISP_REG_UV_PLANE_START_ADDR, uv_addr);
> >> +}
--
Regards,
Laurent Pinchart
Hi Laurent,
On 2023/8/2 18:48, Laurent Pinchart wrote:
> Hi Jack,
>
> On Wed, Aug 02, 2023 at 05:57:57PM +0800, Jack Zhu wrote:
>> On 2023/7/28 4:41, Laurent Pinchart wrote:
>> > On Mon, Jun 19, 2023 at 07:28:37PM +0800, Jack Zhu wrote:
>> >> Add ISP driver for StarFive Camera Subsystem.
>> >>
>> >> Signed-off-by: Jack Zhu <[email protected]>
>> >> ---
>> >> .../media/platform/starfive/camss/Makefile | 2 +
>> >> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
>> >> .../media/platform/starfive/camss/stf_camss.h | 3 +
>> >> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
>> >> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
>> >> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
>> >> 6 files changed, 1544 insertions(+), 3 deletions(-)
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
>
> [snip]
>
>> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
>> >> new file mode 100644
>> >> index 000000000000..933a583b398c
>> >> --- /dev/null
>> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
>> >> @@ -0,0 +1,519 @@
>> >> +// SPDX-License-Identifier: GPL-2.0
>> >> +/*
>> >> + * stf_isp.c
>> >> + *
>> >> + * StarFive Camera Subsystem - ISP Module
>> >> + *
>> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
>> >> + */
>> >> +#include <linux/firmware.h>
>> >
>> > This doesn't seem needed.
>> >
>> >> +#include <media/v4l2-event.h>
>> >> +
>> >> +#include "stf_camss.h"
>> >> +
>> >> +#define SINK_FORMATS_INDEX 0
>> >> +#define UO_FORMATS_INDEX 1
>> >
>> > What does "UO" stand for ?
>>
>> "UO" is Usual Out, just represents output. :-)
>
> Maybe "out", "output" or "source" would make the code easier to read
> then ?
>
>> >> +
>> >> +static int isp_set_selection(struct v4l2_subdev *sd,
>> >> + struct v4l2_subdev_state *state,
>> >> + struct v4l2_subdev_selection *sel);
>> >> +
>> >> +static const struct isp_format isp_formats_sink[] = {
>> >> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
>> >> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
>> >> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
>> >> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
>> >> +};
>
> [snip]
>
>> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
>> >> new file mode 100644
>> >> index 000000000000..1e5c98482350
>> >> --- /dev/null
>> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
>> >> @@ -0,0 +1,479 @@
>
> [snip]
>
>> >> +/* The output line of ISP */
>> >
>> > What is an ISP "line" ?
>>
>> A pipeline contains ISP.
>
> Patch 6/6 uses STF_ISP_LINE_MAX to iterate over the ISP lines. This
> makes the code somehow generic, but you only support a single line at
> the moment. Does this or other SoCs in your product line integrate the
> same ISP with multiple lines ? If so, would it be possible to share a
> block diagram, to better understand the other hardware architectures
> that this driver will need to support in the future ?
>
Yes, OK, I will add a block diagram and a more detailed description in
the starfive_camss.rst file in the next version.
>> >> +enum isp_line_id {
>> >> + STF_ISP_LINE_INVALID = -1,
>> >> + STF_ISP_LINE_SRC = 1,
>> >> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
>> >> +};
>
> [snip]
>
>> >> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
>> >> +{
>> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
>> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
>> >> + DEC_V_KEEP(0x0) |
>> >> + DEC_V_PERIOD(0x0) |
>> >> + DEC_H_KEEP(0x0) |
>> >> + DEC_H_PERIOD(0x0));
>> >> +
>> >> + stf_isp_config_obc(isp_dev->stfcamss);
>> >> + stf_isp_config_oecf(isp_dev->stfcamss);
>> >> + stf_isp_config_lccf(isp_dev->stfcamss);
>> >> + stf_isp_config_awb(isp_dev->stfcamss);
>> >> + stf_isp_config_grgb(isp_dev->stfcamss);
>> >> + stf_isp_config_cfa(isp_dev->stfcamss);
>> >> + stf_isp_config_ccm(isp_dev->stfcamss);
>> >> + stf_isp_config_gamma(isp_dev->stfcamss);
>> >> + stf_isp_config_r2y(isp_dev->stfcamss);
>> >> + stf_isp_config_y_curve(isp_dev->stfcamss);
>> >> + stf_isp_config_sharpen(isp_dev->stfcamss);
>> >> + stf_isp_config_dnyuv(isp_dev->stfcamss);
>> >> + stf_isp_config_sat(isp_dev->stfcamss);
>> >
>> > All these parameters are hardcoded, why are they not exposed to
>> > userspace ?
>>
>> Here is a basic startup configuration for the ISP registers. The
>> function name is confusing, as if it is configuring a specific
>> function. In fact, it is just a basic init configuration.
>
> Did I miss a place in the patch series where all these parameters can be
> configured by userspace, or is that not possible at the moment ? If it
> isn't possible, do you plan to implement that ?
>
Yes, we are doing related development internally.
>> >> +
>> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
>> >> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
>> >> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
>> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
>> >> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
>> >> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
>> >> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
>> >> +}
>
> [snip]
>
Regards,
Jack Zhu
Hi Laurent,
On 2023/8/2 18:38, Laurent Pinchart wrote:
> Hi Jack,
>
> On Wed, Aug 02, 2023 at 05:58:26PM +0800, Jack Zhu wrote:
>> On 2023/7/28 4:49, Laurent Pinchart wrote:
>> > On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
>> >> Add Video In Controller driver for StarFive Camera Subsystem.
>> >
>> > I haven't reviewed this patch in details, as I have a high-level
>> > question: why do you need VIN subdevs ? They don't seem to represent any
>> > particular piece of hardware, their input and output formats are always
>> > identical, and they're not used to configure registers. The contents of
>> > this patch seems to belong to the video device, I think you can drop the
>> > VIN subdevs.
>>
>> The VIN module corresponds to a hardware module, which is mainly responsible
>> for data routing and partial interrupt management (when the image data does
>> not pass through the isp but directly goes to the ddr), the relevant registers
>> need to be configured.
>
> That's fine, but I don't think you need a subdev for it. As far as I
> understand, the VIn modules are (more or less) DMA engines. You can just
> model them as video devices, connected directly to the CSI-2 RX and ISP
> source pads.
>
The VIN hardware can also route input data, it can decide whether DVP sensor
or MIPI sensor is used as input data.
> Does the "vin0_wr" have the ability to capture raw data from the DVP
> interface as well, or only from the CSI-2 RX ?
>
Yes, the "vin0_wr" has the ability to capture raw data from the DVP
interface.
>> >> Signed-off-by: Jack Zhu <[email protected]>
>> >> ---
>> >> .../media/platform/starfive/camss/Makefile | 4 +-
>> >> .../media/platform/starfive/camss/stf_camss.c | 42 +-
>> >> .../media/platform/starfive/camss/stf_camss.h | 2 +
>> >> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
>> >> .../media/platform/starfive/camss/stf_vin.h | 173 +++
>> >> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
>> >> 6 files changed, 1478 insertions(+), 4 deletions(-)
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
>> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>> >>
>> >> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> >> index cdf57e8c9546..ef574e01ca47 100644
>> >> --- a/drivers/media/platform/starfive/camss/Makefile
>> >> +++ b/drivers/media/platform/starfive/camss/Makefile
>> >> @@ -7,6 +7,8 @@ starfive-camss-objs += \
>> >> stf_camss.o \
>> >> stf_isp.o \
>> >> stf_isp_hw_ops.o \
>> >> - stf_video.o
>> >> + stf_video.o \
>> >> + stf_vin.o \
>> >> + stf_vin_hw_ops.o
>> >>
>> >> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> index 6f56b45f57db..834ea63eb833 100644
>> >> --- a/drivers/media/platform/starfive/camss/stf_camss.c
>> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
>> >> return ret;
>> >> }
>> >>
Regards,
Jack Zhu
Hi Jack,
On Thu, Aug 03, 2023 at 10:41:08AM +0800, Jack Zhu wrote:
> On 2023/8/2 18:48, Laurent Pinchart wrote:
> > On Wed, Aug 02, 2023 at 05:57:57PM +0800, Jack Zhu wrote:
> >> On 2023/7/28 4:41, Laurent Pinchart wrote:
> >> > On Mon, Jun 19, 2023 at 07:28:37PM +0800, Jack Zhu wrote:
> >> >> Add ISP driver for StarFive Camera Subsystem.
> >> >>
> >> >> Signed-off-by: Jack Zhu <[email protected]>
> >> >> ---
> >> >> .../media/platform/starfive/camss/Makefile | 2 +
> >> >> .../media/platform/starfive/camss/stf_camss.c | 76 ++-
> >> >> .../media/platform/starfive/camss/stf_camss.h | 3 +
> >> >> .../media/platform/starfive/camss/stf_isp.c | 519 ++++++++++++++++++
> >> >> .../media/platform/starfive/camss/stf_isp.h | 479 ++++++++++++++++
> >> >> .../platform/starfive/camss/stf_isp_hw_ops.c | 468 ++++++++++++++++
> >> >> 6 files changed, 1544 insertions(+), 3 deletions(-)
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.c
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp.h
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_isp_hw_ops.c
> >
> > [snip]
> >
> >> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.c b/drivers/media/platform/starfive/camss/stf_isp.c
> >> >> new file mode 100644
> >> >> index 000000000000..933a583b398c
> >> >> --- /dev/null
> >> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.c
> >> >> @@ -0,0 +1,519 @@
> >> >> +// SPDX-License-Identifier: GPL-2.0
> >> >> +/*
> >> >> + * stf_isp.c
> >> >> + *
> >> >> + * StarFive Camera Subsystem - ISP Module
> >> >> + *
> >> >> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> >> >> + */
> >> >> +#include <linux/firmware.h>
> >> >
> >> > This doesn't seem needed.
> >> >
> >> >> +#include <media/v4l2-event.h>
> >> >> +
> >> >> +#include "stf_camss.h"
> >> >> +
> >> >> +#define SINK_FORMATS_INDEX 0
> >> >> +#define UO_FORMATS_INDEX 1
> >> >
> >> > What does "UO" stand for ?
> >>
> >> "UO" is Usual Out, just represents output. :-)
> >
> > Maybe "out", "output" or "source" would make the code easier to read
> > then ?
> >
> >> >> +
> >> >> +static int isp_set_selection(struct v4l2_subdev *sd,
> >> >> + struct v4l2_subdev_state *state,
> >> >> + struct v4l2_subdev_selection *sel);
> >> >> +
> >> >> +static const struct isp_format isp_formats_sink[] = {
> >> >> + { MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
> >> >> + { MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
> >> >> + { MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
> >> >> + { MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
> >> >> +};
> >
> > [snip]
> >
> >> >> diff --git a/drivers/media/platform/starfive/camss/stf_isp.h b/drivers/media/platform/starfive/camss/stf_isp.h
> >> >> new file mode 100644
> >> >> index 000000000000..1e5c98482350
> >> >> --- /dev/null
> >> >> +++ b/drivers/media/platform/starfive/camss/stf_isp.h
> >> >> @@ -0,0 +1,479 @@
> >
> > [snip]
> >
> >> >> +/* The output line of ISP */
> >> >
> >> > What is an ISP "line" ?
> >>
> >> A pipeline contains ISP.
> >
> > Patch 6/6 uses STF_ISP_LINE_MAX to iterate over the ISP lines. This
> > makes the code somehow generic, but you only support a single line at
> > the moment. Does this or other SoCs in your product line integrate the
> > same ISP with multiple lines ? If so, would it be possible to share a
> > block diagram, to better understand the other hardware architectures
> > that this driver will need to support in the future ?
>
> Yes, OK, I will add a block diagram and a more detailed description in
> the starfive_camss.rst file in the next version.
>
> >> >> +enum isp_line_id {
> >> >> + STF_ISP_LINE_INVALID = -1,
> >> >> + STF_ISP_LINE_SRC = 1,
> >> >> + STF_ISP_LINE_MAX = STF_ISP_LINE_SRC
> >> >> +};
> >
> > [snip]
> >
> >> >> +void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
> >> >> +{
> >> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DC_CFG_1, DC_AXI_ID(0x0));
> >> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DEC_CFG,
> >> >> + DEC_V_KEEP(0x0) |
> >> >> + DEC_V_PERIOD(0x0) |
> >> >> + DEC_H_KEEP(0x0) |
> >> >> + DEC_H_PERIOD(0x0));
> >> >> +
> >> >> + stf_isp_config_obc(isp_dev->stfcamss);
> >> >> + stf_isp_config_oecf(isp_dev->stfcamss);
> >> >> + stf_isp_config_lccf(isp_dev->stfcamss);
> >> >> + stf_isp_config_awb(isp_dev->stfcamss);
> >> >> + stf_isp_config_grgb(isp_dev->stfcamss);
> >> >> + stf_isp_config_cfa(isp_dev->stfcamss);
> >> >> + stf_isp_config_ccm(isp_dev->stfcamss);
> >> >> + stf_isp_config_gamma(isp_dev->stfcamss);
> >> >> + stf_isp_config_r2y(isp_dev->stfcamss);
> >> >> + stf_isp_config_y_curve(isp_dev->stfcamss);
> >> >> + stf_isp_config_sharpen(isp_dev->stfcamss);
> >> >> + stf_isp_config_dnyuv(isp_dev->stfcamss);
> >> >> + stf_isp_config_sat(isp_dev->stfcamss);
> >> >
> >> > All these parameters are hardcoded, why are they not exposed to
> >> > userspace ?
> >>
> >> Here is a basic startup configuration for the ISP registers. The
> >> function name is confusing, as if it is configuring a specific
> >> function. In fact, it is just a basic init configuration.
> >
> > Did I miss a place in the patch series where all these parameters can be
> > configured by userspace, or is that not possible at the moment ? If it
> > isn't possible, do you plan to implement that ?
>
> Yes, we are doing related development internally.
That's nice to hear :-) Without the ability to control the ISP from
userspace, the driver will have very limited usefulness. Still, I
understand that incremental development will be easier to handle, so I'm
not against merging this patch series with hardcoded parameters first,
and adding support for ISP control on top. It may however make sense to
merge the driver in drivers/staging/media/ first, and move it to
drivers/media/ once support for ISP control will be available. That
would give you more room to change the userspace API exposed by the
driver when implementing support for ISP control without having to keep
backward compatibility. Sakari, Hans, do you have any opinion on this ?
Do you have an estimated time frame for when ISP control will be ready ?
> >> >> +
> >> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
> >> >> + CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
> >> >> + CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
> >> >> + stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_ISP_CTRL_1,
> >> >> + CTRL_SAT(1) | CTRL_DBC | CTRL_CTC | CTRL_YHIST |
> >> >> + CTRL_YCURVE | CTRL_BIYUV | CTRL_SCE | CTRL_EE |
> >> >> + CTRL_CCE | CTRL_RGE | CTRL_CME | CTRL_AE | CTRL_CE);
> >> >> +}
> >
> > [snip]
--
Regards,
Laurent Pinchart
Hi Jack,
On Thu, Aug 03, 2023 at 10:44:50AM +0800, Jack Zhu wrote:
> On 2023/8/2 18:38, Laurent Pinchart wrote:
> > On Wed, Aug 02, 2023 at 05:58:26PM +0800, Jack Zhu wrote:
> >> On 2023/7/28 4:49, Laurent Pinchart wrote:
> >> > On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
> >> >> Add Video In Controller driver for StarFive Camera Subsystem.
> >> >
> >> > I haven't reviewed this patch in details, as I have a high-level
> >> > question: why do you need VIN subdevs ? They don't seem to represent any
> >> > particular piece of hardware, their input and output formats are always
> >> > identical, and they're not used to configure registers. The contents of
> >> > this patch seems to belong to the video device, I think you can drop the
> >> > VIN subdevs.
> >>
> >> The VIN module corresponds to a hardware module, which is mainly responsible
> >> for data routing and partial interrupt management (when the image data does
> >> not pass through the isp but directly goes to the ddr), the relevant registers
> >> need to be configured.
> >
> > That's fine, but I don't think you need a subdev for it. As far as I
> > understand, the VIn modules are (more or less) DMA engines. You can just
> > model them as video devices, connected directly to the CSI-2 RX and ISP
> > source pads.
>
> The VIN hardware can also route input data, it can decide whether DVP sensor
> or MIPI sensor is used as input data.
>
> > Does the "vin0_wr" have the ability to capture raw data from the DVP
> > interface as well, or only from the CSI-2 RX ?
>
> Yes, the "vin0_wr" has the ability to capture raw data from the DVP
> interface.
Then I would recommend something similar to the following media graph:
digraph board {
rankdir=TB
imx219 [label="{{} | imx219 6-0010\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
imx219:port0 -> csi2rx:port0 [style=bold]
csi2rx [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
csi2rx:port1 -> vin:port0 [style=bold]
ov5640 [label="{{} | ov5640 6-0020\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
ov5640:port0 -> vin:port1 [style=bold]
vin [label="{{<port0> 0 | <port1> 1} | stf_vin0\n/dev/v4l-subdev2 | {<port2> 2}}", shape=Mrecord, style=filled, fillcolor=green]
vin:port2 -> raw_capture [style=dashed]
vin:port2 -> isp:port0 [style=dashed]
raw_capture [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
isp [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
isp:port1 -> yuv_capture [style=bold]
yuv_capture [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
}
Here, the stf_vin0 subdev is used to route either the CSI-2 input or the
DVP input to the raw capture video device and the ISP.
Does this match the hardware architecture ?
What are ports 2, 3 and 4 for in the CSI-2 RX ?
> >> >> Signed-off-by: Jack Zhu <[email protected]>
> >> >> ---
> >> >> .../media/platform/starfive/camss/Makefile | 4 +-
> >> >> .../media/platform/starfive/camss/stf_camss.c | 42 +-
> >> >> .../media/platform/starfive/camss/stf_camss.h | 2 +
> >> >> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
> >> >> .../media/platform/starfive/camss/stf_vin.h | 173 +++
> >> >> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
> >> >> 6 files changed, 1478 insertions(+), 4 deletions(-)
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
> >> >>
> >> >> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
> >> >> index cdf57e8c9546..ef574e01ca47 100644
> >> >> --- a/drivers/media/platform/starfive/camss/Makefile
> >> >> +++ b/drivers/media/platform/starfive/camss/Makefile
> >> >> @@ -7,6 +7,8 @@ starfive-camss-objs += \
> >> >> stf_camss.o \
> >> >> stf_isp.o \
> >> >> stf_isp_hw_ops.o \
> >> >> - stf_video.o
> >> >> + stf_video.o \
> >> >> + stf_vin.o \
> >> >> + stf_vin_hw_ops.o
> >> >>
> >> >> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> >> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
> >> >> index 6f56b45f57db..834ea63eb833 100644
> >> >> --- a/drivers/media/platform/starfive/camss/stf_camss.c
> >> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
> >> >> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
> >> >> return ret;
> >> >> }
> >> >>
--
Regards,
Laurent Pinchart
Hi Laurent,
Thank you for your comment.
On 2023/8/4 6:18, Laurent Pinchart wrote:
> Hi Jack,
>
> On Thu, Aug 03, 2023 at 10:44:50AM +0800, Jack Zhu wrote:
>> On 2023/8/2 18:38, Laurent Pinchart wrote:
>> > On Wed, Aug 02, 2023 at 05:58:26PM +0800, Jack Zhu wrote:
>> >> On 2023/7/28 4:49, Laurent Pinchart wrote:
>> >> > On Mon, Jun 19, 2023 at 07:28:38PM +0800, Jack Zhu wrote:
>> >> >> Add Video In Controller driver for StarFive Camera Subsystem.
>> >> >
>> >> > I haven't reviewed this patch in details, as I have a high-level
>> >> > question: why do you need VIN subdevs ? They don't seem to represent any
>> >> > particular piece of hardware, their input and output formats are always
>> >> > identical, and they're not used to configure registers. The contents of
>> >> > this patch seems to belong to the video device, I think you can drop the
>> >> > VIN subdevs.
>> >>
>> >> The VIN module corresponds to a hardware module, which is mainly responsible
>> >> for data routing and partial interrupt management (when the image data does
>> >> not pass through the isp but directly goes to the ddr), the relevant registers
>> >> need to be configured.
>> >
>> > That's fine, but I don't think you need a subdev for it. As far as I
>> > understand, the VIn modules are (more or less) DMA engines. You can just
>> > model them as video devices, connected directly to the CSI-2 RX and ISP
>> > source pads.
>>
>> The VIN hardware can also route input data, it can decide whether DVP sensor
>> or MIPI sensor is used as input data.
>>
>> > Does the "vin0_wr" have the ability to capture raw data from the DVP
>> > interface as well, or only from the CSI-2 RX ?
>>
>> Yes, the "vin0_wr" has the ability to capture raw data from the DVP
>> interface.
>
> Then I would recommend something similar to the following media graph:
>
> digraph board {
> rankdir=TB
> imx219 [label="{{} | imx219 6-0010\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> imx219:port0 -> csi2rx:port0 [style=bold]
> csi2rx [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> csi2rx:port1 -> vin:port0 [style=bold]
> ov5640 [label="{{} | ov5640 6-0020\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> ov5640:port0 -> vin:port1 [style=bold]
> vin [label="{{<port0> 0 | <port1> 1} | stf_vin0\n/dev/v4l-subdev2 | {<port2> 2}}", shape=Mrecord, style=filled, fillcolor=green]
> vin:port2 -> raw_capture [style=dashed]
> vin:port2 -> isp:port0 [style=dashed]
> raw_capture [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> isp [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> isp:port1 -> yuv_capture [style=bold]
> yuv_capture [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> }
>
> Here, the stf_vin0 subdev is used to route either the CSI-2 input or the
> DVP input to the raw capture video device and the ISP.
>
> Does this match the hardware architecture ?
>
Yes, but I have a question to ask, is there any way for stf_vin0 subdev
to distinguish whether the current scene is 'raw_capture' or 'yuv_capture'?
> What are ports 2, 3 and 4 for in the CSI-2 RX ?
>
port2-4 are useless, they should be created by 'cdns-csi2rx.c' by default.
>> >> >> Signed-off-by: Jack Zhu <[email protected]>
>> >> >> ---
>> >> >> .../media/platform/starfive/camss/Makefile | 4 +-
>> >> >> .../media/platform/starfive/camss/stf_camss.c | 42 +-
>> >> >> .../media/platform/starfive/camss/stf_camss.h | 2 +
>> >> >> .../media/platform/starfive/camss/stf_vin.c | 1069 +++++++++++++++++
>> >> >> .../media/platform/starfive/camss/stf_vin.h | 173 +++
>> >> >> .../platform/starfive/camss/stf_vin_hw_ops.c | 192 +++
>> >> >> 6 files changed, 1478 insertions(+), 4 deletions(-)
>> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.c
>> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin.h
>> >> >> create mode 100644 drivers/media/platform/starfive/camss/stf_vin_hw_ops.c
>> >> >>
>> >> >> diff --git a/drivers/media/platform/starfive/camss/Makefile b/drivers/media/platform/starfive/camss/Makefile
>> >> >> index cdf57e8c9546..ef574e01ca47 100644
>> >> >> --- a/drivers/media/platform/starfive/camss/Makefile
>> >> >> +++ b/drivers/media/platform/starfive/camss/Makefile
>> >> >> @@ -7,6 +7,8 @@ starfive-camss-objs += \
>> >> >> stf_camss.o \
>> >> >> stf_isp.o \
>> >> >> stf_isp_hw_ops.o \
>> >> >> - stf_video.o
>> >> >> + stf_video.o \
>> >> >> + stf_vin.o \
>> >> >> + stf_vin_hw_ops.o
>> >> >>
>> >> >> obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
>> >> >> diff --git a/drivers/media/platform/starfive/camss/stf_camss.c b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> >> index 6f56b45f57db..834ea63eb833 100644
>> >> >> --- a/drivers/media/platform/starfive/camss/stf_camss.c
>> >> >> +++ b/drivers/media/platform/starfive/camss/stf_camss.c
>> >> >> @@ -131,27 +131,61 @@ static int stfcamss_init_subdevices(struct stfcamss *stfcamss)
>> >> >> return ret;
>> >> >> }
>> >> >>
>
Regards,
Jack Zhu