This series introduces support for the Allwinner A31 and A83T MIPI CSI-2
controllers as well as the Allwinner A31 Image Signal Processor (ISP).
It follows v5 of the "Allwinner MIPI CSI-2 support for A31/V3s/A83T"
series, with the addition of ISP support for the V3. Since both aspect
are quite dependent due to changes to the sun6i-csi driver, they are
merged into this new series.
Aside from the ISP driver itself, the most outstanding change is a
significant rework of the CSI driver, to bring it to a state that
makes it possible to integrate with the ISP workflow.
A number of other improvments are also added along the way.
The commit message introducing the ISP driver should also contain useful
details regarding the implementation and outstanding specifics of the hardware.
This was tested on the V3s and A83T, using the IMX219, OV5648, OV8856
and OV8865 sensors.
v4l2-compliance seems pretty happy about the video nodes, see the detailed
reports below.
Thanks!
-- Changelog
Changes since v1:
- Split changes to the sun6i-csi driver into specific patches;
- Added myself as maintainer of the reworked driver;
- Defined and used mbus interconnect as requested, although the approach
appears to be broken currently (see final commit);
- Removed the patches dealing with PLL clock parenting issues,
since they are a bit out of scope;
- Fixed various minor issues caught in code review;
- Fixed an undefined return issue;
- Reset capture sequence to 0 at stream start;
- Kept runtime pm only in the bridge/proc paths;
- Followed v4l2 API changes (notifier -> nf);
- A few misc cosmetic changes;
-- Previous MIPI CSI-2 series changelog
Changes since v5:
- D-PHY direction is no longer represented with a submode since this is
not a runtime decision: no switching between the two submodes is
possible and each instance of a controller will be dedicated to one
direction only. Instead, a device-tree property is used.
A separate compatible was considered, but it feels unfit since the
direction does not describe the particular type of hardware
implementation, but rather how it is used;
- Updated comments about channels based on latest information;
- Various cosmetic changes (and splitting) to the code;
-- Allwinner MIPI CSI-2 support for A31/V3s/A83T changelog
Changes since v4:
- Added patch to stop using v4l2_async_notifier_parse_fwnode_endpoints;
- Fixed checkpatch strict issues (parenthesis alignment);
- Fixed runtime PM call order and disable;
- Fixed fwnode_handle_put order;
- Brought back phy-names for A31 since it's mandatory according to the generic
PHY binding and needed by the code;
- Added collected tags.
Changes since v3:
- Fixed single-item phys description in sun6i mipi csi-2 binding;
- Fixed variables names in macros using container_of;
- Fixed style issue with operators at the end of lines;
- Reworked source endpoint/subdev assignment in sun6i-csi to handle
link_validate error case;
- Removed unrelated dt change in sun8i-a83t mipi csi-2 driver;
- Added collected tags.
Changes since v2:
- added Kconfig depend on PM since it's not optional;
- removed phy-names for A31 MIPI CSI-2 controller;
- removed v3s compatible in the A31 MIPI CSI-2 controller driver;
- removed A31 CSI controller single-port binding deprecation;
- removed empty dt port definitions;
- fixed minor checkpatch warnings;
- added collected tags;
- added media-ctl output in cover letter.
Changes since v1:
- reworked fwnode and media graph on the CSI controller end to have one port
per interface, which solves the bus type representation issue;
- removed unused IRQ handlers in the MIPI CSI-2 bridges;
- avoided the use of devm_regmap_init_mmio_clk;
- deasserted reset before enabling clocks;
- fixed reported return code issues (ret |=, missing checks);
- applied requested cosmetic changes (backward goto, etc);
- switched over to runtime PM for the mipi csi-2 bridge drivers;
- selected PHY_SUN6I_MIPI_DPHY in Kconfig for sun6i-mipi-csi2;
- registered nodes with mipi csi-2 bridge subdevs;
- used V4L2 format info instead of switch/case for sun6i-csi bpp;
- fixed device-tree bindings as requested (useless properties, license);
- fixed mipi bridge dt instances names;
- added PHY API documentation about mode/power on order requirement;
- fixed clock error return code in d-phy code;
- fixed D-PHY mode check in d-phy code;
- added MAINTAINERS entries for the new drivers;
- added V4L2 compliance results;
- added various comments and rework commit mesages as requested.
-- V3 media topology
Media controller API version 5.13.0
Media device information
------------------------
driver sun6i-isp
model Allwinner A31 ISP Device
serial
bus info platform:1cb8000.isp
hw revision 0x0
driver version 5.13.0
Device topology
- entity 1: sun6i-isp-proc (3 pads, 3 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
<- "sun6i-csi-bridge":1 [ENABLED]
pad1: Sink
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
<- "sun6i-isp-params":0 [ENABLED,IMMUTABLE]
pad2: Source
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
-> "sun6i-isp-capture":0 [ENABLED,IMMUTABLE]
- entity 5: sun6i-csi-bridge (2 pads, 3 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
<- "sun6i-mipi-csi2":1 [ENABLED]
pad1: Source
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
-> "sun6i-isp-proc":0 [ENABLED]
-> "sun6i-csi-capture":0 []
- entity 10: sun6i-mipi-csi2 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
<- "imx219 1-0010":0 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:raw]
-> "sun6i-csi-bridge":0 [ENABLED]
- entity 13: sun6i-csi-capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "sun6i-csi-bridge":1 []
- entity 21: sun6i-isp-capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "sun6i-isp-proc":2 [ENABLED,IMMUTABLE]
- entity 27: sun6i-isp-params (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Source
-> "sun6i-isp-proc":1 [ENABLED,IMMUTABLE]
- entity 33: imx219 1-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev3
pad0: Source
[fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
crop.bounds:(8,8)/3280x2464
crop:(688,700)/1920x1080]
-> "sun6i-mipi-csi2":0 [ENABLED,IMMUTABLE]
-- sun6i-csi-capture v4l2-compliance run
v4l2-compliance SHA: not available, 32 bits
Compliance test for sun6i-csi device /dev/video0:
Driver Info:
Driver name : sun6i-csi
Card type : sun6i-csi-capture
Bus info : platform:1cb0000.camera
Driver version : 5.13.0
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : sun6i-isp
Model : Allwinner A31 ISP Device
Serial :
Bus info : platform:1cb8000.isp
Media version : 5.13.0
Hardware revision: 0x00000000 (0)
Driver version : 5.13.0
Interface Info:
ID : 0x0300000f
Type : V4L Video
Entity Info:
ID : 0x0000000d (13)
Name : sun6i-csi-capture
Function : V4L2 I/O
Pad 0x0100000e : 0: Sink
Link 0x02000011: from remote pad 0x1000007 of entity 'sun6i-csi-bridge': Data
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
warn: v4l2-compliance.cpp(633): media bus_info 'platform:1cb8000.isp' differs from V4L2 bus_info 'platform:1cb0000.camera'
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video0 open: OK
warn: v4l2-compliance.cpp(633): media bus_info 'platform:1cb8000.isp' differs from V4L2 bus_info 'platform:1cb0000.camera'
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 17 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for sun6i-csi device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 2
-- sun6i-isp-capture v4l2-compliance run
v4l2-compliance SHA: not available, 32 bits
Compliance test for sun6i-isp device /dev/video1:
Driver Info:
Driver name : sun6i-isp
Card type : sun6i-isp-capture
Bus info : platform:1cb8000.isp
Driver version : 5.13.0
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : sun6i-isp
Model : Allwinner A31 ISP Device
Serial :
Bus info : platform:1cb8000.isp
Media version : 5.13.0
Hardware revision: 0x00000000 (0)
Driver version : 5.13.0
Interface Info:
ID : 0x03000017
Type : V4L Video
Entity Info:
ID : 0x00000015 (21)
Name : sun6i-isp-capture
Function : V4L2 I/O
Pad 0x01000016 : 0: Sink, Must Connect
Link 0x02000019: from remote pad 0x1000004 of entity 'sun6i-isp-proc': Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 17 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for sun6i-isp device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0
-- sun6i-isp-params v4l2-compliance run
v4l2-compliance SHA: not available, 32 bits
Compliance test for sun6i-isp device /dev/video2:
Driver Info:
Driver name : sun6i-isp
Card type : sun6i-isp-params
Bus info : platform:1cb8000.isp
Driver version : 5.13.0
Capabilities : 0x8c200000
Metadata Output
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x0c200000
Metadata Output
Streaming
Extended Pix Format
Media Driver Info:
Driver name : sun6i-isp
Model : Allwinner A31 ISP Device
Serial :
Bus info : platform:1cb8000.isp
Media version : 5.13.0
Hardware revision: 0x00000000 (0)
Driver version : 5.13.0
Interface Info:
ID : 0x0300001d
Type : V4L Video
Entity Info:
ID : 0x0000001b (27)
Name : sun6i-isp-params
Function : V4L2 I/O
Pad 0x0100001c : 0: Source, Must Connect
Link 0x0200001f: to remote pad 0x1000003 of entity 'sun6i-isp-proc': Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video2 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 17 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for sun6i-isp device /dev/video2: 45, Succeeded: 45, Failed: 0, Warnings: 0
Kévin L'hôpital (1):
ARM: dts: sun8i: a83t: bananapi-m3: Enable MIPI CSI-2 with OV8865
Paul Kocialkowski (65):
ARM: dts: sun8i: v3s: Move the csi1 block to follow address order
dt-bindings: interconnect: sunxi: Add V3s mbus compatible
clk: sunxi-ng: v3s: Export the MBUS clock to the public header
ARM: dts: sun8i: v3s: Add mbus node to represent the interconnect
dt-bindings: sun6i-a31-mipi-dphy: Add optional direction property
phy: allwinner: phy-sun6i-mipi-dphy: Support D-PHY Rx mode for MIPI
CSI-2
dt-bindings: media: sun6i-a31-csi: Add MIPI CSI-2 input port
dt-bindings: media: Add Allwinner A31 MIPI CSI-2 bindings
documentation
media: sunxi: Add support for the A31 MIPI CSI-2 controller
MAINTAINERS: Add entry for the Allwinner A31 MIPI CSI-2 bridge driver
ARM: dts: sun8i: v3s: Add nodes for MIPI CSI-2 support
dt-bindings: media: Add Allwinner A83T MIPI CSI-2 bindings
documentation
media: sunxi: Add support for the A83T MIPI CSI-2 controller
MAINTAINERS: Add entry for the Allwinner A83T MIPI CSI-2 bridge
ARM: dts: sun8i: a83t: Add MIPI CSI-2 controller node
media: sun6i-csi: Define and use driver name and (reworked)
description
media: sun6i-csi: Refactor main driver data structures
media: sun6i-csi: Grab bus clock instead of passing it to regmap
media: sun6i-csi: Tidy up platform code
media: sun6i-csi: Always set exclusive module clock rate
media: sun6i-csi: Use runtime pm for clocks and reset
media: sun6i-csi: Tidy up v4l2 code
media: sun6i-csi: Tidy up video code
media: sun6i-csi: Pass and store csi device directly in video code
media: sun6i-csi: Register the media device after creation
media: sun6i-csi: Add media ops with link notify callback
media: sun6i-csi: Introduce and use video helper functions
media: sun6i-csi: Move csi buffer definition to main header file
media: sun6i-csi: Add bridge v4l2 subdev with port management
media: sun6i-csi: Rename sun6i_video to sun6i_csi_capture
media: sun6i-csi: Add capture state using vsync for page flip
media: sun6i-csi: Rework register definitions, invert misleading
fields
media: sun6i-csi: Add dimensions and format helpers to capture
media: sun6i-csi: Implement address configuration without indirection
media: sun6i-csi: Split stream sequences and irq code in capture
media: sun6i-csi: Move power management to runtime pm in capture
media: sun6i-csi: Move register configuration to capture
media: sun6i-csi: Rework capture format management with helper
media: sun6i-csi: Remove custom format helper and rework configure
media: sun6i-csi: Add bridge dimensions and format helpers
media: sun6i-csi: Get mbus code from bridge instead of storing it
media: sun6i-csi: Tidy capture configure code
media: sun6i-csi: Introduce bridge format structure, list and helper
media: sun6i-csi: Introduce capture format structure, list and helper
media: sun6i-csi: Configure registers from format tables
media: sun6i-csi: Introduce format match structure, list and helper
media: sun6i-csi: Implement capture link validation with logic
media: sun6i-csi: Get bridge subdev directly in capture stream ops
media: sun6i-csi: Move hardware control to the bridge
media: sun6i-csi: Unset bridge source on capture streamon fail
media: sun6i-csi: Rename the capture video device to sun6i-csi-capture
media: sun6i-csi: Cleanup headers and includes, update copyright lines
media: sun6i-csi: Add support for MIPI CSI-2 to the bridge code
media: sun6i-csi: Only configure capture when streaming
media: sun6i-csi: Add extra checks to the interrupt routine
media: sun6i-csi: Request a shared interrupt
media: sun6i-csi: Detect the availability of the ISP
media: sun6i-csi: Add support for hooking to the isp devices
MAINTAINERS: Add myself as sun6i-csi maintainer and rename/move entry
dt-bindings: media: Add Allwinner A31 ISP bindings documentation
dt-bindings: media: sun6i-a31-csi: Add ISP output port
staging: media: Add support for the Allwinner A31 ISP
MAINTAINERS: Add entry for the Allwinner A31 ISP driver
ARM: dts: sun8i: v3s: Add support for the ISP
of: Mark interconnects property supplier as optional
.../arm/sunxi/allwinner,sun4i-a10-mbus.yaml | 1 +
.../media/allwinner,sun6i-a31-csi.yaml | 74 +-
.../media/allwinner,sun6i-a31-isp.yaml | 117 ++
.../media/allwinner,sun6i-a31-mipi-csi2.yaml | 142 +++
.../media/allwinner,sun8i-a83t-mipi-csi2.yaml | 133 ++
.../phy/allwinner,sun6i-a31-mipi-dphy.yaml | 12 +
MAINTAINERS | 42 +-
arch/arm/boot/dts/sun8i-a83t-bananapi-m3.dts | 102 ++
arch/arm/boot/dts/sun8i-a83t.dtsi | 26 +
arch/arm/boot/dts/sun8i-v3s.dtsi | 134 +-
drivers/clk/sunxi-ng/ccu-sun8i-v3s.h | 2 -
drivers/media/platform/sunxi/Kconfig | 2 +
drivers/media/platform/sunxi/Makefile | 2 +
.../media/platform/sunxi/sun6i-csi/Makefile | 2 +-
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 1054 +++++-----------
.../platform/sunxi/sun6i-csi/sun6i_csi.h | 152 +--
.../sunxi/sun6i-csi/sun6i_csi_bridge.c | 885 +++++++++++++
.../sunxi/sun6i-csi/sun6i_csi_bridge.h | 64 +
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 1102 +++++++++++++++++
.../sunxi/sun6i-csi/sun6i_csi_capture.h | 89 ++
.../platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 362 +++---
.../platform/sunxi/sun6i-csi/sun6i_video.c | 681 ----------
.../platform/sunxi/sun6i-csi/sun6i_video.h | 38 -
.../platform/sunxi/sun6i-mipi-csi2/Kconfig | 12 +
.../platform/sunxi/sun6i-mipi-csi2/Makefile | 4 +
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c | 746 +++++++++++
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h | 52 +
.../sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h | 82 ++
.../sunxi/sun8i-a83t-mipi-csi2/Kconfig | 11 +
.../sunxi/sun8i-a83t-mipi-csi2/Makefile | 4 +
.../sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.c | 72 ++
.../sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.h | 39 +
.../sun8i_a83t_mipi_csi2.c | 812 ++++++++++++
.../sun8i_a83t_mipi_csi2.h | 55 +
.../sun8i_a83t_mipi_csi2_reg.h | 157 +++
drivers/of/property.c | 2 +-
drivers/phy/allwinner/phy-sun6i-mipi-dphy.c | 166 ++-
drivers/staging/media/sunxi/Kconfig | 1 +
drivers/staging/media/sunxi/Makefile | 1 +
drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
.../staging/media/sunxi/sun6i-isp/Makefile | 4 +
.../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++
.../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
.../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 +++++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
.../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 +
.../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 +++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 +
.../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 ++++
.../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
include/dt-bindings/clock/sun8i-v3s-ccu.h | 2 +-
52 files changed, 8720 insertions(+), 1823 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml
create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-mipi-csi2.yaml
create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun8i-a83t-mipi-csi2.yaml
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
delete mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
delete mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/Kconfig
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/Makefile
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.c
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.h
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.h
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2_reg.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
--
2.34.1
Given the substantial rework of the driver that I carried out and the
knowledge acquired about the hardware along the way, make myself a
maintainer of the sun6i-csi driver.
Also rename and move the entry while at it since the driver is not
specific to the V3s.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
MAINTAINERS | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 46582119e767..0e65b9e5123f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -751,6 +751,15 @@ T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/allwinner,sun4i-a10-csi.yaml
F: drivers/media/platform/sunxi/sun4i-csi/
+ALLWINNER A31 CSI DRIVER
+M: Yong Deng <[email protected]>
+M: Paul Kocialkowski <[email protected]>
+L: [email protected]
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
+F: drivers/media/platform/sunxi/sun6i-csi/
+
ALLWINNER A31 MIPI CSI-2 BRIDGE DRIVER
M: Paul Kocialkowski <[email protected]>
L: [email protected]
@@ -5064,14 +5073,6 @@ M: Jaya Kumar <[email protected]>
S: Maintained
F: sound/pci/cs5535audio/
-CSI DRIVERS FOR ALLWINNER V3s
-M: Yong Deng <[email protected]>
-L: [email protected]
-S: Maintained
-T: git git://linuxtv.org/media_tree.git
-F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
-F: drivers/media/platform/sunxi/sun6i-csi/
-
CW1200 WLAN driver
M: Solomon Peachy <[email protected]>
S: Maintained
--
2.34.1
In order to support the isp workflow, we need to be able to configure
the hardware from the bridge when the capture device is not used.
As a result, move all hardware configuration calls from capture to
the bridge. Only the window configuration part (which is specific
to using capture) remains there.
This effectively opens the way for hooking the bridge to the
isp in the future.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../sunxi/sun6i-csi/sun6i_csi_bridge.c | 229 ++++++++++++++++
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 249 +-----------------
.../sunxi/sun6i-csi/sun6i_csi_capture.h | 3 +
3 files changed, 239 insertions(+), 242 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
index bb6260f28c45..f5303842d169 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
@@ -174,11 +174,211 @@ sun6i_csi_bridge_format_find(u32 mbus_code)
return NULL;
}
+/* Bridge */
+
+static void sun6i_csi_bridge_irq_enable(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+
+ regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG,
+ SUN6I_CSI_CH_INT_EN_VS |
+ SUN6I_CSI_CH_INT_EN_HB_OF |
+ SUN6I_CSI_CH_INT_EN_FIFO2_OF |
+ SUN6I_CSI_CH_INT_EN_FIFO1_OF |
+ SUN6I_CSI_CH_INT_EN_FIFO0_OF |
+ SUN6I_CSI_CH_INT_EN_FD |
+ SUN6I_CSI_CH_INT_EN_CD);
+}
+
+static void sun6i_csi_bridge_irq_disable(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+
+ regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0);
+}
+
+static void sun6i_csi_bridge_irq_clear(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+
+ regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0);
+ regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG,
+ SUN6I_CSI_CH_INT_STA_CLEAR);
+}
+
+static void sun6i_csi_bridge_enable(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+
+ regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN,
+ SUN6I_CSI_EN_CSI_EN);
+
+ regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON,
+ SUN6I_CSI_CAP_VCAP_ON);
+}
+
+static void sun6i_csi_bridge_disable(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+
+ regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON, 0);
+ regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN, 0);
+}
+
+static void
+sun6i_csi_bridge_configure_interface(struct sun6i_csi_device *csi_dev)
+{
+ struct device *dev = csi_dev->dev;
+ struct regmap *regmap = csi_dev->regmap;
+ struct v4l2_fwnode_endpoint *endpoint =
+ &csi_dev->bridge.source->endpoint;
+ unsigned char bus_width = endpoint->bus.parallel.bus_width;
+ unsigned int flags = endpoint->bus.parallel.flags;
+ u32 field;
+ u32 value = SUN6I_CSI_IF_CFG_IF_CSI;
+
+ sun6i_csi_bridge_format(csi_dev, NULL, &field);
+
+ if (field == V4L2_FIELD_INTERLACED ||
+ field == V4L2_FIELD_INTERLACED_TB ||
+ field == V4L2_FIELD_INTERLACED_BT)
+ value |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
+ SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
+ SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
+ else
+ value |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
+
+ switch (endpoint->bus_type) {
+ case V4L2_MBUS_PARALLEL:
+ if (bus_width == 16)
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
+ else
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
+
+ if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
+ value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ else
+ value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+
+ if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
+ value |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
+ else
+ value |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
+
+ if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
+ value |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
+ else
+ value |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
+
+ if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ else
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ break;
+ case V4L2_MBUS_BT656:
+ if (bus_width == 16)
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
+ else
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
+
+ if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
+ value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ else
+ value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+
+ if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ else
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ break;
+ default:
+ dev_warn(dev, "unsupported bus type: %d\n", endpoint->bus_type);
+ break;
+ }
+
+ switch (bus_width) {
+ case 8:
+ /* 16-bit YUV formats use a doubled width in 8-bit mode. */
+ case 16:
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
+ break;
+ case 10:
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
+ break;
+ case 12:
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
+ break;
+ default:
+ dev_warn(dev, "unsupported bus width: %u\n", bus_width);
+ break;
+ }
+
+ regmap_write(regmap, SUN6I_CSI_IF_CFG_REG, value);
+}
+
+static void sun6i_csi_bridge_configure_format(struct sun6i_csi_device *csi_dev)
+{
+ struct regmap *regmap = csi_dev->regmap;
+ const struct sun6i_csi_bridge_format *bridge_format;
+ const struct sun6i_csi_capture_format *capture_format;
+ u32 mbus_code, field, pixelformat;
+ u8 input_format, input_yuv_seq, output_format;
+ u32 value = 0;
+
+ sun6i_csi_bridge_format(csi_dev, &mbus_code, &field);
+
+ bridge_format = sun6i_csi_bridge_format_find(mbus_code);
+ if (WARN_ON(!bridge_format))
+ return;
+
+ input_format = bridge_format->input_format;
+ input_yuv_seq = bridge_format->input_yuv_seq;
+
+ sun6i_csi_capture_format(csi_dev, &pixelformat, NULL);
+
+ capture_format = sun6i_csi_capture_format_find(pixelformat);
+ if (WARN_ON(!capture_format))
+ return;
+
+ if (capture_format->input_format_raw)
+ input_format = SUN6I_CSI_INPUT_FMT_RAW;
+
+ if (capture_format->input_yuv_seq_invert)
+ input_yuv_seq = bridge_format->input_yuv_seq_invert;
+
+ if (field == V4L2_FIELD_INTERLACED ||
+ field == V4L2_FIELD_INTERLACED_TB ||
+ field == V4L2_FIELD_INTERLACED_BT)
+ output_format = capture_format->output_format_field;
+ else
+ output_format = capture_format->output_format_frame;
+
+ value |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(output_format);
+ value |= SUN6I_CSI_CH_CFG_INPUT_FMT(input_format);
+ value |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(input_yuv_seq);
+
+ if (field == V4L2_FIELD_TOP)
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
+ else if (field == V4L2_FIELD_BOTTOM)
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
+ else
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
+
+ regmap_write(regmap, SUN6I_CSI_CH_CFG_REG, value);
+}
+
+static void sun6i_csi_bridge_configure(struct sun6i_csi_device *csi_dev)
+{
+ sun6i_csi_bridge_configure_interface(csi_dev);
+ sun6i_csi_bridge_configure_format(csi_dev);
+}
+
/* V4L2 Subdev */
static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on)
{
struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
+ struct device *dev = csi_dev->dev;
struct v4l2_subdev *source_subdev;
/* Initialize to 0 to use both in disable label (ret != 0) and off. */
int ret = 0;
@@ -195,6 +395,30 @@ static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on)
goto disable;
}
+ /* PM */
+
+ ret = pm_runtime_resume_and_get(dev);
+ if (ret < 0)
+ return ret;
+
+ /* Clear */
+
+ sun6i_csi_bridge_irq_clear(csi_dev);
+
+ /* Configure */
+
+ sun6i_csi_bridge_configure(csi_dev);
+ sun6i_csi_capture_configure(csi_dev);
+
+ /* State Update */
+
+ sun6i_csi_capture_state_update(csi_dev);
+
+ /* Enable */
+
+ sun6i_csi_bridge_irq_enable(csi_dev);
+ sun6i_csi_bridge_enable(csi_dev);
+
ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
if (ret && ret != -ENOIOCTLCMD)
goto disable;
@@ -202,8 +426,13 @@ static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on)
return 0;
disable:
+ sun6i_csi_bridge_irq_disable(csi_dev);
+ sun6i_csi_bridge_disable(csi_dev);
+
csi_dev->bridge.source = NULL;
+ pm_runtime_put(dev);
+
return ret;
}
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
index 2f9bf75e4e39..4315e197d53f 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
@@ -6,7 +6,6 @@
*/
#include <linux/of.h>
-#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <media/v4l2-device.h>
@@ -330,55 +329,6 @@ static bool sun6i_csi_capture_format_match(u32 pixelformat, u32 mbus_code)
/* Capture */
-static void sun6i_csi_capture_irq_enable(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
-
- regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG,
- SUN6I_CSI_CH_INT_EN_VS |
- SUN6I_CSI_CH_INT_EN_HB_OF |
- SUN6I_CSI_CH_INT_EN_FIFO2_OF |
- SUN6I_CSI_CH_INT_EN_FIFO1_OF |
- SUN6I_CSI_CH_INT_EN_FIFO0_OF |
- SUN6I_CSI_CH_INT_EN_FD |
- SUN6I_CSI_CH_INT_EN_CD);
-}
-
-static void sun6i_csi_capture_irq_disable(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
-
- regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0);
-}
-
-static void sun6i_csi_capture_irq_clear(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
-
- regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0);
- regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG,
- SUN6I_CSI_CH_INT_STA_CLEAR);
-}
-
-static void sun6i_csi_capture_enable(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
-
- regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN,
- SUN6I_CSI_EN_CSI_EN);
-
- regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON,
- SUN6I_CSI_CAP_VCAP_ON);
-}
-
-static void sun6i_csi_capture_disable(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
-
- regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON, 0);
- regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN, 0);
-}
-
static void
sun6i_csi_capture_buffer_configure(struct sun6i_csi_device *csi_dev,
struct sun6i_csi_buffer *csi_buffer)
@@ -420,148 +370,7 @@ sun6i_csi_capture_buffer_configure(struct sun6i_csi_device *csi_dev,
}
}
-static void
-sun6i_csi_capture_configure_interface(struct sun6i_csi_device *csi_dev)
-{
- struct device *dev = csi_dev->dev;
- struct regmap *regmap = csi_dev->regmap;
- struct v4l2_fwnode_endpoint *endpoint =
- &csi_dev->bridge.source->endpoint;
- unsigned char bus_width = endpoint->bus.parallel.bus_width;
- unsigned int flags = endpoint->bus.parallel.flags;
- u32 pixelformat, field;
- u32 value = SUN6I_CSI_IF_CFG_IF_CSI;
-
- sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
-
- if (field == V4L2_FIELD_INTERLACED ||
- field == V4L2_FIELD_INTERLACED_TB ||
- field == V4L2_FIELD_INTERLACED_BT)
- value |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
- SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
- SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
- else
- value |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
-
- switch (endpoint->bus_type) {
- case V4L2_MBUS_PARALLEL:
- if (bus_width == 16)
- value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
- else
- value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
-
- if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
- else
- value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
-
- if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
- value |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
- else
- value |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
-
- if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
- value |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
- else
- value |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
-
- if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
- value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
- else
- value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
- break;
- case V4L2_MBUS_BT656:
- if (bus_width == 16)
- value |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
- else
- value |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
-
- if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
- else
- value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
-
- if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
- value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
- else
- value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
- break;
- default:
- dev_warn(dev, "unsupported bus type: %d\n", endpoint->bus_type);
- break;
- }
-
- switch (bus_width) {
- case 8:
- /* 16-bit YUV formats use a doubled width in 8-bit mode. */
- case 16:
- value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
- break;
- case 10:
- value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
- break;
- case 12:
- value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
- break;
- default:
- dev_warn(dev, "unsupported bus width: %u\n", bus_width);
- break;
- }
-
- regmap_write(regmap, SUN6I_CSI_IF_CFG_REG, value);
-}
-
-static void sun6i_csi_capture_configure_format(struct sun6i_csi_device *csi_dev)
-{
- struct regmap *regmap = csi_dev->regmap;
- const struct sun6i_csi_bridge_format *bridge_format;
- const struct sun6i_csi_capture_format *capture_format;
- u32 mbus_code, pixelformat, field;
- u8 input_format, input_yuv_seq, output_format;
- u32 value = 0;
-
- sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
- sun6i_csi_bridge_format(csi_dev, &mbus_code, NULL);
-
- bridge_format = sun6i_csi_bridge_format_find(mbus_code);
- if (WARN_ON(!bridge_format))
- return;
-
- input_format = bridge_format->input_format;
- input_yuv_seq = bridge_format->input_yuv_seq;
-
- capture_format = sun6i_csi_capture_format_find(pixelformat);
- if (WARN_ON(!capture_format))
- return;
-
- if (capture_format->input_format_raw)
- input_format = SUN6I_CSI_INPUT_FMT_RAW;
-
- if (capture_format->input_yuv_seq_invert)
- input_yuv_seq = bridge_format->input_yuv_seq_invert;
-
- if (field == V4L2_FIELD_INTERLACED ||
- field == V4L2_FIELD_INTERLACED_TB ||
- field == V4L2_FIELD_INTERLACED_BT)
- output_format = capture_format->output_format_field;
- else
- output_format = capture_format->output_format_frame;
-
- value |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(output_format);
- value |= SUN6I_CSI_CH_CFG_INPUT_FMT(input_format);
- value |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(input_yuv_seq);
-
- if (field == V4L2_FIELD_TOP)
- value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
- else if (field == V4L2_FIELD_BOTTOM)
- value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
- else
- value |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
-
- regmap_write(regmap, SUN6I_CSI_CH_CFG_REG, value);
-}
-
-static void sun6i_csi_capture_configure_window(struct sun6i_csi_device *csi_dev)
+void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev)
{
struct regmap *regmap = csi_dev->regmap;
const struct sun6i_csi_capture_format *format;
@@ -624,13 +433,6 @@ static void sun6i_csi_capture_configure_window(struct sun6i_csi_device *csi_dev)
SUN6I_CSI_CH_BUF_LEN_LUMA_LINE(luma_line));
}
-static void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev)
-{
- sun6i_csi_capture_configure_interface(csi_dev);
- sun6i_csi_capture_configure_format(csi_dev);
- sun6i_csi_capture_configure_window(csi_dev);
-}
-
/* State */
static void sun6i_csi_capture_state_cleanup(struct sun6i_csi_device *csi_dev,
@@ -670,7 +472,7 @@ static void sun6i_csi_capture_state_cleanup(struct sun6i_csi_device *csi_dev,
spin_unlock_irqrestore(&state->lock, flags);
}
-static void sun6i_csi_capture_state_update(struct sun6i_csi_device *csi_dev)
+void sun6i_csi_capture_state_update(struct sun6i_csi_device *csi_dev)
{
struct sun6i_csi_capture_state *state = &csi_dev->capture.state;
struct sun6i_csi_buffer *csi_buffer;
@@ -803,11 +605,9 @@ static int sun6i_csi_capture_start_streaming(struct vb2_queue *queue,
unsigned int count)
{
struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue);
- struct sun6i_csi_capture *capture = &csi_dev->capture;
- struct sun6i_csi_capture_state *state = &capture->state;
- struct video_device *video_dev = &capture->video_dev;
+ struct sun6i_csi_capture_state *state = &csi_dev->capture.state;
+ struct video_device *video_dev = &csi_dev->capture.video_dev;
struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
- struct device *dev = csi_dev->dev;
int ret;
state->sequence = 0;
@@ -816,41 +616,12 @@ static int sun6i_csi_capture_start_streaming(struct vb2_queue *queue,
if (ret < 0)
goto error_state;
- /* PM */
-
- ret = pm_runtime_resume_and_get(dev);
- if (ret < 0)
- goto error_media_pipeline;
-
- /* Clear */
-
- sun6i_csi_capture_irq_clear(csi_dev);
-
- /* Configure */
-
- sun6i_csi_capture_configure(csi_dev);
-
- /* State Update */
-
- sun6i_csi_capture_state_update(csi_dev);
-
- /* Enable */
-
- sun6i_csi_capture_irq_enable(csi_dev);
- sun6i_csi_capture_enable(csi_dev);
-
ret = v4l2_subdev_call(subdev, video, s_stream, 1);
if (ret && ret != -ENOIOCTLCMD)
- goto error_stream;
+ goto error_media_pipeline;
return 0;
-error_stream:
- sun6i_csi_capture_disable(csi_dev);
- sun6i_csi_capture_irq_disable(csi_dev);
-
- pm_runtime_put(dev);
-
error_media_pipeline:
media_pipeline_stop(&video_dev->entity);
@@ -863,18 +634,12 @@ static int sun6i_csi_capture_start_streaming(struct vb2_queue *queue,
static void sun6i_csi_capture_stop_streaming(struct vb2_queue *queue)
{
struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue);
- struct sun6i_csi_capture *capture = &csi_dev->capture;
+ struct video_device *video_dev = &csi_dev->capture.video_dev;
struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
- struct device *dev = csi_dev->dev;
v4l2_subdev_call(subdev, video, s_stream, 0);
- sun6i_csi_capture_disable(csi_dev);
- sun6i_csi_capture_irq_disable(csi_dev);
-
- pm_runtime_put(dev);
-
- media_pipeline_stop(&capture->video_dev.entity);
+ media_pipeline_stop(&video_dev->entity);
sun6i_csi_capture_state_cleanup(csi_dev, true);
}
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
index 2605b16f091c..a61db3bc72e5 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
@@ -63,6 +63,9 @@ void sun6i_csi_capture_format(struct sun6i_csi_device *csi_dev,
const
struct sun6i_csi_capture_format *sun6i_csi_capture_format_find(u32 pixelformat);
+void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev);
+void sun6i_csi_capture_state_update(struct sun6i_csi_device *csi_dev);
+
void sun6i_csi_capture_sync(struct sun6i_csi_device *csi_dev);
void sun6i_csi_capture_frame_done(struct sun6i_csi_device *csi_dev);
--
2.34.1
Since the bus clock alone is not enough to get access to the registers,
don't pass it to regmap and manage it instead just like the other
clocks.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 10 ++++++++--
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 1 +
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 5fbaa1e99412..dc79f3c14336 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -827,13 +827,19 @@ static int sun6i_csi_resource_request(struct sun6i_csi_device *csi_dev,
if (IS_ERR(io_base))
return PTR_ERR(io_base);
- csi_dev->regmap = devm_regmap_init_mmio_clk(&pdev->dev, "bus", io_base,
- &sun6i_csi_regmap_config);
+ csi_dev->regmap = devm_regmap_init_mmio(&pdev->dev, io_base,
+ &sun6i_csi_regmap_config);
if (IS_ERR(csi_dev->regmap)) {
dev_err(&pdev->dev, "Failed to init register map\n");
return PTR_ERR(csi_dev->regmap);
}
+ csi_dev->clk_bus = devm_clk_get(&pdev->dev, "bus");
+ if (IS_ERR(csi_dev->clk_bus)) {
+ dev_err(&pdev->dev, "Unable to acquire bus clock\n");
+ return PTR_ERR(csi_dev->clk_bus);
+ }
+
csi_dev->clk_mod = devm_clk_get(&pdev->dev, "mod");
if (IS_ERR(csi_dev->clk_mod)) {
dev_err(&pdev->dev, "Unable to acquire csi clock\n");
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
index e4e7ac6c869f..356661b413f8 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
@@ -51,6 +51,7 @@ struct sun6i_csi_device {
struct sun6i_video video;
struct regmap *regmap;
+ struct clk *clk_bus;
struct clk *clk_mod;
struct clk *clk_ram;
struct reset_control *reset;
--
2.34.1
Since the V3s uses the internal mbus, document its compatible.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
index 29c9961ee2d8..b67bf9261a6a 100644
--- a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
+++ b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
@@ -31,6 +31,7 @@ properties:
- allwinner,sun5i-a13-mbus
- allwinner,sun8i-h3-mbus
- allwinner,sun8i-r40-mbus
+ - allwinner,sun8i-v3s-mbus
- allwinner,sun50i-a64-mbus
reg:
--
2.34.1
Another instance of removing a duplicated variable and using common
helpers instead.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 18 +++++-------------
.../sunxi/sun6i-csi/sun6i_csi_capture.h | 1 -
2 files changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
index f977b89dcea2..12c02408d18e 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
@@ -17,6 +17,7 @@
#include <media/videobuf2-v4l2.h>
#include "sun6i_csi.h"
+#include "sun6i_csi_bridge.h"
#include "sun6i_csi_capture.h"
#include "sun6i_csi_reg.h"
@@ -455,20 +456,20 @@ sun6i_csi_capture_configure_interface(struct sun6i_csi_device *csi_dev)
static void sun6i_csi_capture_configure_format(struct sun6i_csi_device *csi_dev)
{
- struct sun6i_csi_capture *capture = &csi_dev->capture;
- u32 pixelformat, field;
+ u32 mbus_code, pixelformat, field;
u32 cfg = 0;
u32 val;
sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
+ sun6i_csi_bridge_format(csi_dev, &mbus_code, NULL);
- val = get_csi_input_format(csi_dev, capture->mbus_code, pixelformat);
+ val = get_csi_input_format(csi_dev, mbus_code, pixelformat);
cfg |= SUN6I_CSI_CH_CFG_INPUT_FMT(val);
val = get_csi_output_format(csi_dev, pixelformat, field);
cfg |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(val);
- val = get_csi_input_seq(csi_dev, capture->mbus_code, pixelformat);
+ val = get_csi_input_seq(csi_dev, mbus_code, pixelformat);
cfg |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(val);
if (field == V4L2_FIELD_TOP)
@@ -739,11 +740,6 @@ static int sun6i_csi_capture_start_streaming(struct vb2_queue *queue,
if (ret < 0)
goto error_state;
- if (capture->mbus_code == 0) {
- ret = -EINVAL;
- goto error_media_pipeline;
- }
-
subdev = sun6i_csi_capture_remote_subdev(capture, NULL);
if (!subdev) {
ret = -EINVAL;
@@ -1067,8 +1063,6 @@ static int sun6i_csi_capture_link_validate(struct media_link *link)
struct v4l2_subdev_format source_fmt;
int ret;
- capture->mbus_code = 0;
-
if (!media_entity_remote_pad(link->sink->entity->pads)) {
dev_info(csi_dev->dev, "capture node %s pad not connected\n",
vdev->name);
@@ -1100,8 +1094,6 @@ static int sun6i_csi_capture_link_validate(struct media_link *link)
return -EPIPE;
}
- capture->mbus_code = source_fmt.format.code;
-
return 0;
}
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
index 02bdf45f7ca5..3b9759e1563d 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
@@ -39,7 +39,6 @@ struct sun6i_csi_capture {
struct media_pad pad;
struct v4l2_format format;
- u32 mbus_code;
};
void sun6i_csi_capture_dimensions(struct sun6i_csi_device *csi_dev,
--
2.34.1
Add myself as maintainer of the Allwinner A31 ISP media driver.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
MAINTAINERS | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 0e65b9e5123f..709cf8aa1a8d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -760,6 +760,15 @@ T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
F: drivers/media/platform/sunxi/sun6i-csi/
+ALLWINNER A31 ISP DRIVER
+M: Paul Kocialkowski <[email protected]>
+L: [email protected]
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml
+F: drivers/staging/media/sunxi/sun6i-isp/
+F: drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
+
ALLWINNER A31 MIPI CSI-2 BRIDGE DRIVER
M: Paul Kocialkowski <[email protected]>
L: [email protected]
--
2.34.1
Check against the enabled bits and make sure capture is running before
serving an interrupt, to add extra safety in the process.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 8941104f611b..e3ac7dad86ae 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -111,13 +111,17 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi_device *csi_dev)
static irqreturn_t sun6i_csi_isr(int irq, void *private)
{
struct sun6i_csi_device *csi_dev = private;
+ bool capture_streaming = csi_dev->capture.state.streaming;
struct regmap *regmap = csi_dev->regmap;
- u32 status;
+ u32 status = 0, enable = 0;
regmap_read(regmap, SUN6I_CSI_CH_INT_STA_REG, &status);
+ regmap_read(regmap, SUN6I_CSI_CH_INT_EN_REG, &enable);
- if (!(status & 0xFF))
+ if (!status)
return IRQ_NONE;
+ else if (!(status & enable) || !capture_streaming)
+ goto complete;
if ((status & SUN6I_CSI_CH_INT_STA_FIFO0_OF) ||
(status & SUN6I_CSI_CH_INT_STA_FIFO1_OF) ||
@@ -138,6 +142,7 @@ static irqreturn_t sun6i_csi_isr(int irq, void *private)
if (status & SUN6I_CSI_CH_INT_STA_VS)
sun6i_csi_capture_sync(csi_dev);
+complete:
regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG, status);
return IRQ_HANDLED;
--
2.34.1
The buffer structure is a top-level definition, put it in the main header
to keep things tidy. No functional change intended.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 9 +++++++++
drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c | 8 --------
2 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
index 356661b413f8..4dd83e57bafa 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
@@ -11,12 +11,21 @@
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
#include <media/v4l2-fwnode.h>
+#include <media/videobuf2-v4l2.h>
#include "sun6i_video.h"
#define SUN6I_CSI_NAME "sun6i-csi"
#define SUN6I_CSI_DESCRIPTION "Allwinner A31 CSI Device"
+struct sun6i_csi_buffer {
+ struct vb2_v4l2_buffer v4l2_buffer;
+ struct list_head list;
+
+ dma_addr_t dma_addr;
+ bool queued_to_csi;
+};
+
/**
* struct sun6i_csi_config - configs for sun6i csi
* @pixelformat: v4l2 pixel format (V4L2_PIX_FMT_*)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index 6f5470a6859d..d32ff6b81f8a 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -23,14 +23,6 @@
#define MAX_WIDTH (4800)
#define MAX_HEIGHT (4800)
-struct sun6i_csi_buffer {
- struct vb2_v4l2_buffer v4l2_buffer;
- struct list_head list;
-
- dma_addr_t dma_addr;
- bool queued_to_csi;
-};
-
/* Helpers */
static struct v4l2_subdev *
--
2.34.1
This illegitimate change makes it possible for the sun6i csi and isp
drivers to probe without a proper interconnect driver available.
Without this change, the core will wait for an interconnect driver to
probe and return EPROBE_DEFER in our drivers. This ends up in these
drivers never being probed since no driver exists.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/of/property.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/of/property.c b/drivers/of/property.c
index a3483484a5a2..6f1b74a2f568 100644
--- a/drivers/of/property.c
+++ b/drivers/of/property.c
@@ -1352,7 +1352,7 @@ static struct device_node *parse_interrupts(struct device_node *np,
static const struct supplier_bindings of_supplier_bindings[] = {
{ .parse_prop = parse_clocks, },
- { .parse_prop = parse_interconnects, },
+ { .parse_prop = parse_interconnects, .optional = true,},
{ .parse_prop = parse_iommus, .optional = true, },
{ .parse_prop = parse_iommu_maps, .optional = true, },
{ .parse_prop = parse_mboxes, },
--
2.34.1
Add a helper to detect whether the ISP is available and connected
and store the indication in a driver-wide variable.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 33 +++++++++++++++++++
.../platform/sunxi/sun6i-csi/sun6i_csi.h | 3 ++
2 files changed, 36 insertions(+)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index b8e925f6bc09..70b2a8b95b40 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -24,6 +24,35 @@
#include "sun6i_csi_capture.h"
#include "sun6i_csi_reg.h"
+/* ISP */
+
+static bool sun6i_csi_isp_detect(struct sun6i_csi_device *csi_dev)
+{
+ struct device *dev = csi_dev->dev;
+ struct fwnode_handle *handle = NULL;
+
+ /* ISP is not available if disabled in kernel config. */
+ if (!IS_ENABLED(CONFIG_VIDEO_SUN6I_ISP))
+ return 0;
+
+ /*
+ * ISP is not available if not connected via fwnode graph.
+ * This weill also check that the remote parent node is available.
+ */
+ handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev),
+ SUN6I_CSI_PORT_ISP, 0,
+ FWNODE_GRAPH_ENDPOINT_NEXT);
+ if (!handle)
+ return 0;
+
+ fwnode_handle_put(handle);
+
+ dev_info(dev, "ISP link is available\n");
+ csi_dev->isp_available = true;
+
+ return 0;
+}
+
/* Media */
static const struct media_device_ops sun6i_csi_media_ops = {
@@ -328,6 +357,10 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)
if (ret)
return ret;
+ ret = sun6i_csi_isp_detect(csi_dev);
+ if (ret)
+ goto error_resources;
+
ret = sun6i_csi_v4l2_setup(csi_dev);
if (ret)
goto error_resources;
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
index 3c08b2712215..6cdaf98dc295 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
@@ -22,6 +22,7 @@
enum sun6i_csi_port {
SUN6I_CSI_PORT_PARALLEL = 0,
SUN6I_CSI_PORT_MIPI_CSI2 = 1,
+ SUN6I_CSI_PORT_ISP = 2,
};
struct sun6i_csi_buffer {
@@ -47,6 +48,8 @@ struct sun6i_csi_device {
struct clk *clk_mod;
struct clk *clk_ram;
struct reset_control *reset;
+
+ bool isp_available;
};
/* V4L2 */
--
2.34.1
Some Allwinner platforms come with an Image Signal Processor, which
supports various features in order to enhance and transform data
received by image sensors into good-looking pictures. In most cases,
the data is raw bayer, which gets internally converted to RGB and
finally YUV, which is what the hardware produces.
This driver supports ISPs that are similar to the A31 ISP, which was
the first standalone ISP found in Allwinner platforms. Simpler ISP
blocks were found in the A10 and A20, where they are tied to a CSI
controller. Newer generations of Allwinner SoCs (starting with the
H6, H616, etc) come with a new camera subsystem and revised ISP.
Even though these previous and next-generation ISPs are somewhat
similar to the A31 ISP, they have enough significant differences to
be out of the scope of this driver.
While the ISP supports many features, including 3A and many
enhancement blocks, this implementation is limited to the following:
- V3s (V3/S3) platform support;
- Bayer media bus formats as input;
- Semi-planar YUV (NV12/NV21) as output;
- Debayering with per-component gain and offset configuration;
- 2D noise filtering with configurable coefficients.
Since many features are missing from the associated uAPI, the driver
is aimed to integrate staging until all features are properly
described.
On the technical side, it uses the v4l2 and media controller APIs,
with a video node for capture, a processor subdev and a video node
for parameters submission. A specific uAPI structure and associated
v4l2 meta format are used to configure parameters of the supported
modules.
One particular thing about the hardware is that configuration for
module registers needs to be stored in a DMA buffer and gets copied
to actual registers by the hardware at the next vsync, when instructed
by a flag. This is handled by the "state" mechanism in the driver.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/staging/media/sunxi/Kconfig | 1 +
drivers/staging/media/sunxi/Makefile | 1 +
drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
.../staging/media/sunxi/sun6i-isp/Makefile | 4 +
.../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
.../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
.../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
.../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
.../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
.../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
.../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
.../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
14 files changed, 3109 insertions(+)
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
diff --git a/drivers/staging/media/sunxi/Kconfig b/drivers/staging/media/sunxi/Kconfig
index 4549a135741f..62a486aba88b 100644
--- a/drivers/staging/media/sunxi/Kconfig
+++ b/drivers/staging/media/sunxi/Kconfig
@@ -12,5 +12,6 @@ config VIDEO_SUNXI
if VIDEO_SUNXI
source "drivers/staging/media/sunxi/cedrus/Kconfig"
+source "drivers/staging/media/sunxi/sun6i-isp/Kconfig"
endif
diff --git a/drivers/staging/media/sunxi/Makefile b/drivers/staging/media/sunxi/Makefile
index b87140b0e15f..3d20b2f0e644 100644
--- a/drivers/staging/media/sunxi/Makefile
+++ b/drivers/staging/media/sunxi/Makefile
@@ -1,2 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += cedrus/
+obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp/
diff --git a/drivers/staging/media/sunxi/sun6i-isp/Kconfig b/drivers/staging/media/sunxi/sun6i-isp/Kconfig
new file mode 100644
index 000000000000..4a259c4e0046
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_SUN6I_ISP
+ tristate "Allwinner A31 Image Signal Processor (ISP) Driver"
+ depends on ARCH_SUNXI || COMPILE_TEST
+ depends on PM && COMMON_CLK && VIDEO_DEV && VIDEO_V4L2
+ depends on HAS_DMA
+ select REGMAP_MMIO
+ select MEDIA_CONTROLLER
+ select VIDEOBUF2_DMA_CONTIG
+ select VIDEOBUF2_VMALLOC
+ select V4L2_MEM2MEM_DEV
+ help
+ Support for the Allwinner A31 Image Signal Processor (ISP).
diff --git a/drivers/staging/media/sunxi/sun6i-isp/Makefile b/drivers/staging/media/sunxi/sun6i-isp/Makefile
new file mode 100644
index 000000000000..da1034785144
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+sun6i-isp-y += sun6i_isp.o sun6i_isp_proc.o sun6i_isp_capture.o sun6i_isp_params.o
+
+obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp.o
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
new file mode 100644
index 000000000000..ed60b69e2a51
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
@@ -0,0 +1,572 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-mc.h>
+
+#include "sun6i_isp.h"
+#include "sun6i_isp_capture.h"
+#include "sun6i_isp_params.h"
+#include "sun6i_isp_proc.h"
+#include "sun6i_isp_reg.h"
+
+/* Helpers */
+
+u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset)
+{
+ u32 *data = (u32 *)(isp_dev->tables.load.data + offset);
+
+ return *data;
+}
+
+void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset,
+ u32 value)
+{
+ u32 *data = (u32 *)(isp_dev->tables.load.data + offset);
+
+ *data = value;
+}
+
+/* State */
+
+/*
+ * The ISP works with a load buffer, which gets copied to the actual registers
+ * by the hardware before processing a frame when a specific flag is set.
+ * This is represented by tracking the ISP state in the different parts of
+ * the code with explicit sync points:
+ * - state update: to update the load buffer for the next frame if necessary;
+ * - state complete: to indicate that the state update was applied.
+ */
+
+static void sun6i_isp_state_ready(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+ u32 value;
+
+ regmap_read(regmap, SUN6I_ISP_FE_CTRL_REG, &value);
+ value |= SUN6I_ISP_FE_CTRL_PARA_READY;
+ regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, value);
+}
+
+static void sun6i_isp_state_complete(struct sun6i_isp_device *isp_dev)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&isp_dev->state_lock, flags);
+
+ sun6i_isp_capture_state_complete(isp_dev);
+ sun6i_isp_params_state_complete(isp_dev);
+
+ spin_unlock_irqrestore(&isp_dev->state_lock, flags);
+}
+
+void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold)
+{
+ bool update = false;
+ unsigned long flags;
+
+ spin_lock_irqsave(&isp_dev->state_lock, flags);
+
+ sun6i_isp_capture_state_update(isp_dev, &update);
+ sun6i_isp_params_state_update(isp_dev, &update);
+
+ if (update && !ready_hold)
+ sun6i_isp_state_ready(isp_dev);
+
+ spin_unlock_irqrestore(&isp_dev->state_lock, flags);
+}
+
+/* Tables */
+
+static int sun6i_isp_table_setup(struct sun6i_isp_device *isp_dev,
+ struct sun6i_isp_table *table)
+{
+ table->data = dma_alloc_coherent(isp_dev->dev, table->size,
+ &table->address, GFP_KERNEL);
+ if (!table->data)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void sun6i_isp_table_cleanup(struct sun6i_isp_device *isp_dev,
+ struct sun6i_isp_table *table)
+{
+ dma_free_coherent(isp_dev->dev, table->size, table->data,
+ table->address);
+}
+
+void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+
+ regmap_write(regmap, SUN6I_ISP_REG_LOAD_ADDR_REG,
+ SUN6I_ISP_ADDR_VALUE(isp_dev->tables.load.address));
+
+ regmap_write(regmap, SUN6I_ISP_REG_SAVE_ADDR_REG,
+ SUN6I_ISP_ADDR_VALUE(isp_dev->tables.save.address));
+
+ regmap_write(regmap, SUN6I_ISP_LUT_TABLE_ADDR_REG,
+ SUN6I_ISP_ADDR_VALUE(isp_dev->tables.lut.address));
+
+ regmap_write(regmap, SUN6I_ISP_DRC_TABLE_ADDR_REG,
+ SUN6I_ISP_ADDR_VALUE(isp_dev->tables.drc.address));
+
+ regmap_write(regmap, SUN6I_ISP_STATS_ADDR_REG,
+ SUN6I_ISP_ADDR_VALUE(isp_dev->tables.stats.address));
+}
+
+static int sun6i_isp_tables_setup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_tables *tables = &isp_dev->tables;
+ int ret;
+
+ /* Sizes are hardcoded for now but actually depend on the platform. */
+
+ tables->load.size = 0x1000;
+ ret = sun6i_isp_table_setup(isp_dev, &tables->load);
+ if (ret)
+ return ret;
+
+ tables->save.size = 0x1000;
+ ret = sun6i_isp_table_setup(isp_dev, &tables->save);
+ if (ret)
+ return ret;
+
+ tables->lut.size = 0xe00;
+ ret = sun6i_isp_table_setup(isp_dev, &tables->lut);
+ if (ret)
+ return ret;
+
+ tables->drc.size = 0x600;
+ ret = sun6i_isp_table_setup(isp_dev, &tables->drc);
+ if (ret)
+ return ret;
+
+ tables->stats.size = 0x2100;
+ ret = sun6i_isp_table_setup(isp_dev, &tables->stats);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void sun6i_isp_tables_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_tables *tables = &isp_dev->tables;
+
+ sun6i_isp_table_cleanup(isp_dev, &tables->stats);
+ sun6i_isp_table_cleanup(isp_dev, &tables->drc);
+ sun6i_isp_table_cleanup(isp_dev, &tables->lut);
+ sun6i_isp_table_cleanup(isp_dev, &tables->save);
+ sun6i_isp_table_cleanup(isp_dev, &tables->load);
+}
+
+/* Media */
+
+static const struct media_device_ops sun6i_isp_media_ops = {
+ .link_notify = v4l2_pipeline_link_notify,
+};
+
+/* V4L2 */
+
+static int sun6i_isp_v4l2_setup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2;
+ struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
+ struct media_device *media_dev = &v4l2->media_dev;
+ struct device *dev = isp_dev->dev;
+ int ret;
+
+ /* Media Device */
+
+ strscpy(media_dev->model, SUN6I_ISP_DESCRIPTION,
+ sizeof(media_dev->model));
+ snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+ "platform:%s", dev_name(dev));
+ media_dev->ops = &sun6i_isp_media_ops;
+ media_dev->hw_revision = 0;
+ media_dev->dev = dev;
+
+ media_device_init(media_dev);
+
+ ret = media_device_register(media_dev);
+ if (ret) {
+ dev_err(dev, "failed to register media device\n");
+ return ret;
+ }
+
+ /* V4L2 Control Handler */
+
+ ret = v4l2_ctrl_handler_init(&v4l2->ctrl_handler, 0);
+ if (ret) {
+ dev_err(dev, "failed to init v4l2 control handler\n");
+ goto error_media;
+ }
+
+ /* V4L2 Device */
+
+ v4l2_dev->mdev = media_dev;
+ v4l2_dev->ctrl_handler = &v4l2->ctrl_handler;
+
+ ret = v4l2_device_register(dev, v4l2_dev);
+ if (ret) {
+ dev_err(dev, "failed to register v4l2 device\n");
+ goto error_v4l2_ctrl;
+ }
+
+ return 0;
+
+error_v4l2_ctrl:
+ v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
+
+error_media:
+ media_device_unregister(media_dev);
+ media_device_cleanup(media_dev);
+
+ return ret;
+}
+
+static void sun6i_isp_v4l2_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2;
+
+ media_device_unregister(&v4l2->media_dev);
+ v4l2_device_unregister(&v4l2->v4l2_dev);
+ v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
+ media_device_cleanup(&v4l2->media_dev);
+}
+
+/* Platform */
+
+static irqreturn_t sun6i_isp_isr(int irq, void *private)
+{
+ struct sun6i_isp_device *isp_dev = private;
+ struct regmap *regmap = isp_dev->regmap;
+ u32 status = 0, enable = 0;
+
+ regmap_read(regmap, SUN6I_ISP_FE_INT_STA_REG, &status);
+ regmap_read(regmap, SUN6I_ISP_FE_INT_EN_REG, &enable);
+
+ if (!status)
+ return IRQ_NONE;
+ else if (!(status & enable))
+ goto complete;
+
+ /*
+ * The ISP working cycle starts with a params-load, which makes the
+ * state from the load buffer active. Then it starts processing the
+ * frame and gives a finish interrupt. Soon after that, the next state
+ * coming from the load buffer will be applied for the next frame,
+ * giving a params-load as well.
+ *
+ * Because both frame finish and params-load are received almost
+ * at the same time (one ISR call), handle them in chronology order.
+ */
+
+ if (status & SUN6I_ISP_FE_INT_STA_FINISH)
+ sun6i_isp_capture_finish(isp_dev);
+
+ if (status & SUN6I_ISP_FE_INT_STA_PARA_LOAD) {
+ sun6i_isp_state_complete(isp_dev);
+ sun6i_isp_state_update(isp_dev, false);
+ }
+
+complete:
+ regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG, status);
+
+ return IRQ_HANDLED;
+}
+
+static int sun6i_isp_suspend(struct device *dev)
+{
+ struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev);
+
+ reset_control_assert(isp_dev->reset);
+ clk_disable_unprepare(isp_dev->clk_ram);
+ clk_disable_unprepare(isp_dev->clk_mod);
+ clk_disable_unprepare(isp_dev->clk_bus);
+
+ return 0;
+}
+
+static int sun6i_isp_resume(struct device *dev)
+{
+ struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev);
+ int ret;
+
+ ret = reset_control_deassert(isp_dev->reset);
+ if (ret) {
+ dev_err(dev, "failed to deassert reset\n");
+ return ret;
+ }
+
+ ret = clk_prepare_enable(isp_dev->clk_bus);
+ if (ret) {
+ dev_err(dev, "failed to enable bus clock\n");
+ goto error_reset;
+ }
+
+ ret = clk_prepare_enable(isp_dev->clk_mod);
+ if (ret) {
+ dev_err(dev, "failed to enable module clock\n");
+ goto error_clk_bus;
+ }
+
+ ret = clk_prepare_enable(isp_dev->clk_ram);
+ if (ret) {
+ dev_err(dev, "failed to enable ram clock\n");
+ goto error_clk_mod;
+ }
+
+ return 0;
+
+error_clk_mod:
+ clk_disable_unprepare(isp_dev->clk_mod);
+
+error_clk_bus:
+ clk_disable_unprepare(isp_dev->clk_bus);
+
+error_reset:
+ reset_control_assert(isp_dev->reset);
+
+ return ret;
+}
+
+static const struct dev_pm_ops sun6i_isp_pm_ops = {
+ SET_RUNTIME_PM_OPS(sun6i_isp_suspend, sun6i_isp_resume, NULL)
+};
+
+static const struct regmap_config sun6i_isp_regmap_config = {
+ .reg_bits = 32,
+ .reg_stride = 4,
+ .val_bits = 32,
+ .max_register = 0x400,
+};
+
+static int sun6i_isp_resources_setup(struct sun6i_isp_device *isp_dev,
+ struct platform_device *platform_dev)
+{
+ struct device *dev = isp_dev->dev;
+ void __iomem *io_base;
+ int irq;
+ int ret;
+
+ /* Registers */
+
+ io_base = devm_platform_ioremap_resource(platform_dev, 0);
+ if (IS_ERR(io_base))
+ return PTR_ERR(io_base);
+
+ isp_dev->regmap = devm_regmap_init_mmio(dev, io_base,
+ &sun6i_isp_regmap_config);
+ if (IS_ERR(isp_dev->regmap)) {
+ dev_err(dev, "failed to init register map\n");
+ return PTR_ERR(isp_dev->regmap);
+ }
+
+ /* Clocks */
+
+ isp_dev->clk_bus = devm_clk_get(dev, "bus");
+ if (IS_ERR(isp_dev->clk_bus)) {
+ dev_err(dev, "failed to acquire bus clock\n");
+ return PTR_ERR(isp_dev->clk_bus);
+ }
+
+ isp_dev->clk_mod = devm_clk_get(dev, "mod");
+ if (IS_ERR(isp_dev->clk_mod)) {
+ dev_err(dev, "failed to acquire module clock\n");
+ return PTR_ERR(isp_dev->clk_mod);
+ }
+
+ isp_dev->clk_ram = devm_clk_get(dev, "ram");
+ if (IS_ERR(isp_dev->clk_ram)) {
+ dev_err(dev, "failed to acquire ram clock\n");
+ return PTR_ERR(isp_dev->clk_ram);
+ }
+
+ ret = clk_set_rate_exclusive(isp_dev->clk_mod, 297000000);
+ if (ret) {
+ dev_err(dev, "failed to set mod clock rate\n");
+ return ret;
+ }
+
+ /* Reset */
+
+ isp_dev->reset = devm_reset_control_get_shared(dev, NULL);
+ if (IS_ERR(isp_dev->reset)) {
+ dev_err(dev, "failed to acquire reset\n");
+ ret = PTR_ERR(isp_dev->reset);
+ goto error_clk_rate_exclusive;
+ }
+
+ /* Interrupt */
+
+ irq = platform_get_irq(platform_dev, 0);
+ if (irq < 0) {
+ dev_err(dev, "failed to get interrupt\n");
+ ret = -ENXIO;
+ goto error_clk_rate_exclusive;
+ }
+
+ ret = devm_request_irq(dev, irq, sun6i_isp_isr, IRQF_SHARED,
+ SUN6I_ISP_NAME, isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to request interrupt\n");
+ goto error_clk_rate_exclusive;
+ }
+
+ /* Runtime PM */
+
+ pm_runtime_enable(dev);
+ pm_runtime_set_suspended(dev);
+
+ return 0;
+
+error_clk_rate_exclusive:
+ clk_rate_exclusive_put(isp_dev->clk_mod);
+
+ return ret;
+}
+
+static void sun6i_isp_resources_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct device *dev = isp_dev->dev;
+
+ pm_runtime_disable(dev);
+ clk_rate_exclusive_put(isp_dev->clk_mod);
+}
+
+static int sun6i_isp_probe(struct platform_device *platform_dev)
+{
+ struct sun6i_isp_device *isp_dev;
+ struct device *dev = &platform_dev->dev;
+ int ret;
+
+ isp_dev = devm_kzalloc(dev, sizeof(*isp_dev), GFP_KERNEL);
+ if (!isp_dev)
+ return -ENOMEM;
+
+ isp_dev->dev = dev;
+ platform_set_drvdata(platform_dev, isp_dev);
+
+ spin_lock_init(&isp_dev->state_lock);
+
+ ret = sun6i_isp_resources_setup(isp_dev, platform_dev);
+ if (ret)
+ return ret;
+
+ ret = sun6i_isp_tables_setup(isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to setup tables\n");
+ goto error_resources;
+ }
+
+ ret = sun6i_isp_v4l2_setup(isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to setup v4l2\n");
+ goto error_tables;
+ }
+
+ ret = sun6i_isp_proc_setup(isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to setup proc\n");
+ goto error_v4l2;
+ }
+
+ ret = sun6i_isp_capture_setup(isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to setup capture\n");
+ goto error_proc;
+ }
+
+ ret = sun6i_isp_params_setup(isp_dev);
+ if (ret) {
+ dev_err(dev, "failed to setup params\n");
+ goto error_capture;
+ }
+
+ return 0;
+
+error_capture:
+ sun6i_isp_capture_cleanup(isp_dev);
+
+error_proc:
+ sun6i_isp_proc_cleanup(isp_dev);
+
+error_v4l2:
+ sun6i_isp_v4l2_cleanup(isp_dev);
+
+error_tables:
+ sun6i_isp_tables_cleanup(isp_dev);
+
+error_resources:
+ sun6i_isp_resources_cleanup(isp_dev);
+
+ return ret;
+}
+
+static int sun6i_isp_remove(struct platform_device *platform_dev)
+{
+ struct sun6i_isp_device *isp_dev = platform_get_drvdata(platform_dev);
+
+ sun6i_isp_params_cleanup(isp_dev);
+ sun6i_isp_capture_cleanup(isp_dev);
+ sun6i_isp_proc_cleanup(isp_dev);
+ sun6i_isp_v4l2_cleanup(isp_dev);
+ sun6i_isp_tables_cleanup(isp_dev);
+ sun6i_isp_resources_cleanup(isp_dev);
+
+ return 0;
+}
+
+/*
+ * History of sun6i-isp:
+ * - sun4i-a10-isp: initial ISP tied to the CSI0 controller,
+ * apparently unused in software implementations;
+ * - sun6i-a31-isp: separate ISP loosely based on sun4i-a10-isp,
+ * adding extra modules and features;
+ * - sun9i-a80-isp: based on sun6i-a31-isp with some register offset changes
+ * and new modules like saturation and cnr;
+ * - sun8i-a23-isp/sun8i-h3-isp: based on sun9i-a80-isp with most modules
+ * related to raw removed;
+ * - sun8i-a83t-isp: based on sun9i-a80-isp with some register offset changes
+ * - sun8i-v3s-isp: based on sun8i-a83t-isp with a new disc module;
+ */
+
+static const struct of_device_id sun6i_isp_of_match[] = {
+ { .compatible = "allwinner,sun8i-v3s-isp" },
+ {},
+};
+
+MODULE_DEVICE_TABLE(of, sun6i_isp_of_match);
+
+static struct platform_driver sun6i_isp_platform_driver = {
+ .probe = sun6i_isp_probe,
+ .remove = sun6i_isp_remove,
+ .driver = {
+ .name = SUN6I_ISP_NAME,
+ .of_match_table = of_match_ptr(sun6i_isp_of_match),
+ .pm = &sun6i_isp_pm_ops,
+ },
+};
+
+module_platform_driver(sun6i_isp_platform_driver);
+
+MODULE_DESCRIPTION("Allwinner A31 Image Signal Processor driver");
+MODULE_AUTHOR("Paul Kocialkowski <[email protected]>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
new file mode 100644
index 000000000000..d487d4d61727
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_ISP_H_
+#define _SUN6I_ISP_H_
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "sun6i_isp_capture.h"
+#include "sun6i_isp_params.h"
+#include "sun6i_isp_proc.h"
+
+#define SUN6I_ISP_NAME "sun6i-isp"
+#define SUN6I_ISP_DESCRIPTION "Allwinner A31 ISP Device"
+
+enum sun6i_isp_port {
+ SUN6I_ISP_PORT_CSI0 = 0,
+ SUN6I_ISP_PORT_CSI1 = 1,
+};
+
+struct sun6i_isp_buffer {
+ struct vb2_v4l2_buffer v4l2_buffer;
+ struct list_head list;
+};
+
+struct sun6i_isp_v4l2 {
+ struct v4l2_device v4l2_dev;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct media_device media_dev;
+};
+
+struct sun6i_isp_table {
+ void *data;
+ dma_addr_t address;
+ unsigned int size;
+};
+
+struct sun6i_isp_tables {
+ struct sun6i_isp_table load;
+ struct sun6i_isp_table save;
+
+ struct sun6i_isp_table lut;
+ struct sun6i_isp_table drc;
+ struct sun6i_isp_table stats;
+};
+
+struct sun6i_isp_device {
+ struct device *dev;
+
+ struct sun6i_isp_tables tables;
+
+ struct sun6i_isp_v4l2 v4l2;
+ struct sun6i_isp_proc proc;
+ struct sun6i_isp_capture capture;
+ struct sun6i_isp_params params;
+
+ struct regmap *regmap;
+ struct clk *clk_bus;
+ struct clk *clk_mod;
+ struct clk *clk_ram;
+ struct reset_control *reset;
+
+ spinlock_t state_lock; /* State helpers lock. */
+};
+
+/* Helpers */
+
+u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset);
+void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset,
+ u32 value);
+u32 sun6i_isp_address_value(dma_addr_t address);
+
+/* State */
+
+void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold);
+
+/* Tables */
+
+void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev);
+
+#endif
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
new file mode 100644
index 000000000000..4ccc828fb8b9
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
@@ -0,0 +1,751 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "sun6i_isp.h"
+#include "sun6i_isp_capture.h"
+#include "sun6i_isp_proc.h"
+#include "sun6i_isp_reg.h"
+
+/* Helpers */
+
+void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev,
+ unsigned int *width, unsigned int *height)
+{
+ if (width)
+ *width = isp_dev->capture.format.fmt.pix.width;
+ if (height)
+ *height = isp_dev->capture.format.fmt.pix.height;
+}
+
+void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev,
+ u32 *pixelformat)
+{
+ if (pixelformat)
+ *pixelformat = isp_dev->capture.format.fmt.pix.pixelformat;
+}
+
+/* Format */
+
+static const struct sun6i_isp_capture_format sun6i_isp_capture_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .output_format = SUN6I_ISP_OUTPUT_FMT_YUV420SP,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_NV21,
+ .output_format = SUN6I_ISP_OUTPUT_FMT_YVU420SP,
+ },
+};
+
+const struct sun6i_isp_capture_format *
+sun6i_isp_capture_format_find(u32 pixelformat)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(sun6i_isp_capture_formats); i++)
+ if (sun6i_isp_capture_formats[i].pixelformat == pixelformat)
+ return &sun6i_isp_capture_formats[i];
+
+ return NULL;
+}
+
+/* Capture */
+
+static void
+sun6i_isp_capture_buffer_configure(struct sun6i_isp_device *isp_dev,
+ struct sun6i_isp_buffer *isp_buffer)
+{
+ const struct v4l2_format_info *info;
+ struct vb2_buffer *vb2_buffer;
+ unsigned int width, height;
+ unsigned int width_aligned;
+ dma_addr_t address;
+ u32 pixelformat;
+
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+ address = vb2_dma_contig_plane_dma_addr(vb2_buffer, 0);
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_Y_ADDR0_REG,
+ SUN6I_ISP_ADDR_VALUE(address));
+
+ sun6i_isp_capture_dimensions(isp_dev, &width, &height);
+ sun6i_isp_capture_format(isp_dev, &pixelformat);
+
+ info = v4l2_format_info(pixelformat);
+ if (WARN_ON(!info))
+ return;
+
+ /* Stride needs to be aligned to 4. */
+ width_aligned = ALIGN(width, 2);
+
+ if (info->comp_planes > 1) {
+ address += info->bpp[0] * width_aligned * height;
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_U_ADDR0_REG,
+ SUN6I_ISP_ADDR_VALUE(address));
+ }
+
+ if (info->comp_planes > 2) {
+ address += info->bpp[1] *
+ DIV_ROUND_UP(width_aligned, info->hdiv) *
+ DIV_ROUND_UP(height, info->vdiv);
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_V_ADDR0_REG,
+ SUN6I_ISP_ADDR_VALUE(address));
+ }
+}
+
+void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev)
+{
+ unsigned int width, height;
+ unsigned int stride_luma, stride_chroma = 0;
+ unsigned int stride_luma_div4, stride_chroma_div4;
+ const struct sun6i_isp_capture_format *format;
+ const struct v4l2_format_info *info;
+ u32 pixelformat;
+
+ sun6i_isp_capture_dimensions(isp_dev, &width, &height);
+ sun6i_isp_capture_format(isp_dev, &pixelformat);
+
+ format = sun6i_isp_capture_format_find(pixelformat);
+ if (WARN_ON(!format))
+ return;
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_SIZE_CFG_REG,
+ SUN6I_ISP_MCH_SIZE_CFG_WIDTH(width) |
+ SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(height));
+
+ info = v4l2_format_info(pixelformat);
+ if (WARN_ON(!info))
+ return;
+
+ stride_luma = width * info->bpp[0];
+ stride_luma_div4 = DIV_ROUND_UP(stride_luma, 4);
+
+ if (info->comp_planes > 1) {
+ stride_chroma = width * info->bpp[1] / info->hdiv;
+ stride_chroma_div4 = DIV_ROUND_UP(stride_chroma, 4);
+ }
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_CFG_REG,
+ SUN6I_ISP_MCH_CFG_EN |
+ SUN6I_ISP_MCH_CFG_OUTPUT_FMT(format->output_format) |
+ SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(stride_luma_div4) |
+ SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(stride_chroma_div4));
+}
+
+/* State */
+
+static void sun6i_isp_capture_state_cleanup(struct sun6i_isp_device *isp_dev,
+ bool error)
+{
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ struct sun6i_isp_buffer **isp_buffer_states[] = {
+ &state->pending, &state->current, &state->complete,
+ };
+ struct sun6i_isp_buffer *isp_buffer;
+ struct vb2_buffer *vb2_buffer;
+ unsigned long flags;
+ unsigned int i;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ for (i = 0; i < ARRAY_SIZE(isp_buffer_states); i++) {
+ isp_buffer = *isp_buffer_states[i];
+ if (!isp_buffer)
+ continue;
+
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+ vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
+ VB2_BUF_STATE_QUEUED);
+
+ *isp_buffer_states[i] = NULL;
+ }
+
+ list_for_each_entry(isp_buffer, &state->queue, list) {
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+ vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
+ VB2_BUF_STATE_QUEUED);
+ }
+
+ INIT_LIST_HEAD(&state->queue);
+
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev,
+ bool *update)
+{
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ struct sun6i_isp_buffer *isp_buffer;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ if (list_empty(&state->queue))
+ goto complete;
+
+ if (state->pending)
+ goto complete;
+
+ isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer,
+ list);
+
+ sun6i_isp_capture_buffer_configure(isp_dev, isp_buffer);
+
+ list_del(&isp_buffer->list);
+
+ state->pending = isp_buffer;
+
+ if (update)
+ *update = true;
+
+complete:
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ if (!state->pending)
+ goto complete;
+
+ state->complete = state->current;
+ state->current = state->pending;
+ state->pending = NULL;
+
+ if (state->complete) {
+ struct sun6i_isp_buffer *isp_buffer = state->complete;
+ struct vb2_buffer *vb2_buffer =
+ &isp_buffer->v4l2_buffer.vb2_buf;
+
+ vb2_buffer->timestamp = ktime_get_ns();
+ isp_buffer->v4l2_buffer.sequence = state->sequence;
+
+ vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE);
+
+ state->complete = NULL;
+ }
+
+complete:
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+ state->sequence++;
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+/* Queue */
+
+static int sun6i_isp_capture_queue_setup(struct vb2_queue *queue,
+ unsigned int *buffers_count,
+ unsigned int *planes_count,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage;
+
+ if (*planes_count)
+ return sizes[0] < size ? -EINVAL : 0;
+
+ *planes_count = 1;
+ sizes[0] = size;
+
+ return 0;
+}
+
+static int sun6i_isp_capture_buffer_prepare(struct vb2_buffer *vb2_buffer)
+{
+ struct sun6i_isp_device *isp_dev =
+ vb2_get_drv_priv(vb2_buffer->vb2_queue);
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage;
+
+ if (vb2_plane_size(vb2_buffer, 0) < size) {
+ v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n",
+ vb2_plane_size(vb2_buffer, 0), size);
+ return -EINVAL;
+ }
+
+ vb2_set_plane_payload(vb2_buffer, 0, size);
+
+ return 0;
+}
+
+static void sun6i_isp_capture_buffer_queue(struct vb2_buffer *vb2_buffer)
+{
+ struct sun6i_isp_device *isp_dev =
+ vb2_get_drv_priv(vb2_buffer->vb2_queue);
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer);
+ struct sun6i_isp_buffer *isp_buffer =
+ container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+ list_add_tail(&isp_buffer->list, &state->queue);
+ spin_unlock_irqrestore(&state->lock, flags);
+
+ /* Update the state to schedule our buffer as soon as possible. */
+ if (state->streaming)
+ sun6i_isp_state_update(isp_dev, false);
+}
+
+static int sun6i_isp_capture_start_streaming(struct vb2_queue *queue,
+ unsigned int count)
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ struct video_device *video_dev = &isp_dev->capture.video_dev;
+ struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
+ int ret;
+
+ state->sequence = 0;
+
+ ret = media_pipeline_start(&video_dev->entity, &video_dev->pipe);
+ if (ret < 0)
+ goto error_state;
+
+ state->streaming = true;
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, 1);
+ if (ret && ret != -ENOIOCTLCMD)
+ goto error_streaming;
+
+ return 0;
+
+error_streaming:
+ state->streaming = false;
+
+ media_pipeline_stop(&video_dev->entity);
+
+error_state:
+ sun6i_isp_capture_state_cleanup(isp_dev, false);
+
+ isp_dev->proc.source = NULL;
+
+ return ret;
+}
+
+static void sun6i_isp_capture_stop_streaming(struct vb2_queue *queue)
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ struct sun6i_isp_capture_state *state = &isp_dev->capture.state;
+ struct video_device *video_dev = &isp_dev->capture.video_dev;
+ struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
+
+ v4l2_subdev_call(subdev, video, s_stream, 0);
+
+ state->streaming = false;
+
+ media_pipeline_stop(&video_dev->entity);
+
+ sun6i_isp_capture_state_cleanup(isp_dev, true);
+}
+
+static const struct vb2_ops sun6i_isp_capture_queue_ops = {
+ .queue_setup = sun6i_isp_capture_queue_setup,
+ .buf_prepare = sun6i_isp_capture_buffer_prepare,
+ .buf_queue = sun6i_isp_capture_buffer_queue,
+ .start_streaming = sun6i_isp_capture_start_streaming,
+ .stop_streaming = sun6i_isp_capture_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+/* Video Device */
+
+static void sun6i_isp_capture_format_prepare(struct v4l2_format *format)
+{
+ struct v4l2_pix_format *pix_format = &format->fmt.pix;
+ const struct v4l2_format_info *info;
+ unsigned int width, height;
+ unsigned int width_aligned;
+ unsigned int i;
+
+ v4l_bound_align_image(&pix_format->width, SUN6I_ISP_CAPTURE_WIDTH_MIN,
+ SUN6I_ISP_CAPTURE_WIDTH_MAX, 1,
+ &pix_format->height, SUN6I_ISP_CAPTURE_HEIGHT_MIN,
+ SUN6I_ISP_CAPTURE_HEIGHT_MAX, 1, 0);
+
+ if (!sun6i_isp_capture_format_find(pix_format->pixelformat))
+ pix_format->pixelformat =
+ sun6i_isp_capture_formats[0].pixelformat;
+
+ info = v4l2_format_info(pix_format->pixelformat);
+ if (WARN_ON(!info))
+ return;
+
+ width = pix_format->width;
+ height = pix_format->height;
+
+ /* Stride needs to be aligned to 4. */
+ width_aligned = ALIGN(width, 2);
+
+ pix_format->bytesperline = width_aligned * info->bpp[0];
+ pix_format->sizeimage = 0;
+
+ for (i = 0; i < info->comp_planes; i++) {
+ unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
+ unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
+
+ pix_format->sizeimage += info->bpp[i] *
+ DIV_ROUND_UP(width_aligned, hdiv) *
+ DIV_ROUND_UP(height, vdiv);
+ }
+
+ pix_format->field = V4L2_FIELD_NONE;
+
+ pix_format->colorspace = V4L2_COLORSPACE_RAW;
+ pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pix_format->quantization = V4L2_QUANTIZATION_DEFAULT;
+ pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int sun6i_isp_capture_querycap(struct file *file, void *private,
+ struct v4l2_capability *capability)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+ struct video_device *video_dev = &isp_dev->capture.video_dev;
+
+ strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver));
+ strscpy(capability->card, video_dev->name, sizeof(capability->card));
+ snprintf(capability->bus_info, sizeof(capability->bus_info),
+ "platform:%s", dev_name(isp_dev->dev));
+
+ return 0;
+}
+
+static int sun6i_isp_capture_enum_fmt(struct file *file, void *private,
+ struct v4l2_fmtdesc *fmtdesc)
+{
+ u32 index = fmtdesc->index;
+
+ if (index >= ARRAY_SIZE(sun6i_isp_capture_formats))
+ return -EINVAL;
+
+ fmtdesc->pixelformat = sun6i_isp_capture_formats[index].pixelformat;
+
+ return 0;
+}
+
+static int sun6i_isp_capture_g_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+
+ *format = isp_dev->capture.format;
+
+ return 0;
+}
+
+static int sun6i_isp_capture_s_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+
+ if (vb2_is_busy(&isp_dev->capture.queue))
+ return -EBUSY;
+
+ sun6i_isp_capture_format_prepare(format);
+
+ isp_dev->capture.format = *format;
+
+ return 0;
+}
+
+static int sun6i_isp_capture_try_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
+{
+ sun6i_isp_capture_format_prepare(format);
+
+ return 0;
+}
+
+static int sun6i_isp_capture_enum_input(struct file *file, void *private,
+ struct v4l2_input *input)
+{
+ if (input->index != 0)
+ return -EINVAL;
+
+ input->type = V4L2_INPUT_TYPE_CAMERA;
+ strscpy(input->name, "Camera", sizeof(input->name));
+
+ return 0;
+}
+
+static int sun6i_isp_capture_g_input(struct file *file, void *private,
+ unsigned int *index)
+{
+ *index = 0;
+
+ return 0;
+}
+
+static int sun6i_isp_capture_s_input(struct file *file, void *private,
+ unsigned int index)
+{
+ if (index != 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops sun6i_isp_capture_ioctl_ops = {
+ .vidioc_querycap = sun6i_isp_capture_querycap,
+
+ .vidioc_enum_fmt_vid_cap = sun6i_isp_capture_enum_fmt,
+ .vidioc_g_fmt_vid_cap = sun6i_isp_capture_g_fmt,
+ .vidioc_s_fmt_vid_cap = sun6i_isp_capture_s_fmt,
+ .vidioc_try_fmt_vid_cap = sun6i_isp_capture_try_fmt,
+
+ .vidioc_enum_input = sun6i_isp_capture_enum_input,
+ .vidioc_g_input = sun6i_isp_capture_g_input,
+ .vidioc_s_input = sun6i_isp_capture_s_input,
+
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_log_status = v4l2_ctrl_log_status,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int sun6i_isp_capture_open(struct file *file)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+ struct video_device *video_dev = &isp_dev->capture.video_dev;
+ struct mutex *lock = &isp_dev->capture.lock;
+ int ret;
+
+ if (mutex_lock_interruptible(lock))
+ return -ERESTARTSYS;
+
+ ret = v4l2_pipeline_pm_get(&video_dev->entity);
+ if (ret)
+ goto error_mutex;
+
+ ret = v4l2_fh_open(file);
+ if (ret)
+ goto error_pipeline;
+
+ mutex_unlock(lock);
+
+ return 0;
+
+error_pipeline:
+ v4l2_pipeline_pm_put(&video_dev->entity);
+
+error_mutex:
+ mutex_unlock(lock);
+
+ return ret;
+}
+
+static int sun6i_isp_capture_release(struct file *file)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+ struct video_device *video_dev = &isp_dev->capture.video_dev;
+ struct mutex *lock = &isp_dev->capture.lock;
+
+ mutex_lock(lock);
+
+ _vb2_fop_release(file, NULL);
+ v4l2_pipeline_pm_put(&video_dev->entity);
+
+ mutex_unlock(lock);
+
+ return 0;
+}
+
+static const struct v4l2_file_operations sun6i_isp_capture_fops = {
+ .owner = THIS_MODULE,
+ .open = sun6i_isp_capture_open,
+ .release = sun6i_isp_capture_release,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
+
+/* Media Entity */
+
+static int sun6i_isp_capture_link_validate(struct media_link *link)
+{
+ struct video_device *video_dev =
+ media_entity_to_video_device(link->sink->entity);
+ struct sun6i_isp_device *isp_dev = video_get_drvdata(video_dev);
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ unsigned int capture_width, capture_height;
+ unsigned int proc_width, proc_height;
+
+ sun6i_isp_capture_dimensions(isp_dev, &capture_width, &capture_height);
+ sun6i_isp_proc_dimensions(isp_dev, &proc_width, &proc_height);
+
+ /* No cropping/scaling is supported (yet). */
+ if (capture_width != proc_width || capture_height != proc_height) {
+ v4l2_err(v4l2_dev,
+ "invalid input/output dimensions: %ux%u/%ux%u\n",
+ proc_width, proc_height, capture_width,
+ capture_height);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct media_entity_operations sun6i_isp_capture_entity_ops = {
+ .link_validate = sun6i_isp_capture_link_validate,
+};
+
+/* Capture */
+
+int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_capture *capture = &isp_dev->capture;
+ struct sun6i_isp_capture_state *state = &capture->state;
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev;
+ struct video_device *video_dev = &capture->video_dev;
+ struct vb2_queue *queue = &capture->queue;
+ struct media_pad *pad = &capture->pad;
+ struct v4l2_format *format = &capture->format;
+ struct v4l2_pix_format *pix_format = &format->fmt.pix;
+ int ret;
+
+ /* State */
+
+ INIT_LIST_HEAD(&state->queue);
+ spin_lock_init(&state->lock);
+
+ /* Media Entity */
+
+ video_dev->entity.ops = &sun6i_isp_capture_entity_ops;
+
+ /* Media Pads */
+
+ pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT;
+
+ ret = media_entity_pads_init(&video_dev->entity, 1, pad);
+ if (ret)
+ goto error_mutex;
+
+ /* Queue */
+
+ mutex_init(&capture->lock);
+
+ queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ queue->io_modes = VB2_MMAP | VB2_DMABUF;
+ queue->buf_struct_size = sizeof(struct sun6i_isp_buffer);
+ queue->ops = &sun6i_isp_capture_queue_ops;
+ queue->mem_ops = &vb2_dma_contig_memops;
+ queue->min_buffers_needed = 2;
+ queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ queue->lock = &capture->lock;
+ queue->dev = isp_dev->dev;
+ queue->drv_priv = isp_dev;
+
+ ret = vb2_queue_init(queue);
+ if (ret) {
+ v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret);
+ goto error_media_entity;
+ }
+
+ /* V4L2 Format */
+
+ format->type = queue->type;
+ pix_format->pixelformat = sun6i_isp_capture_formats[0].pixelformat;
+ pix_format->width = 1280;
+ pix_format->height = 720;
+
+ sun6i_isp_capture_format_prepare(format);
+
+ /* Video Device */
+
+ strscpy(video_dev->name, SUN6I_ISP_CAPTURE_NAME,
+ sizeof(video_dev->name));
+ video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+ video_dev->vfl_dir = VFL_DIR_RX;
+ video_dev->release = video_device_release_empty;
+ video_dev->fops = &sun6i_isp_capture_fops;
+ video_dev->ioctl_ops = &sun6i_isp_capture_ioctl_ops;
+ video_dev->v4l2_dev = v4l2_dev;
+ video_dev->queue = queue;
+ video_dev->lock = &capture->lock;
+
+ video_set_drvdata(video_dev, isp_dev);
+
+ ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ v4l2_err(v4l2_dev, "failed to register video device: %d\n",
+ ret);
+ goto error_media_entity;
+ }
+
+ v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
+ video_device_node_name(video_dev));
+
+ /* Media Pad Link */
+
+ ret = media_create_pad_link(&proc_subdev->entity,
+ SUN6I_ISP_PROC_PAD_SOURCE,
+ &video_dev->entity, 0,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret < 0) {
+ v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
+ proc_subdev->entity.name, SUN6I_ISP_PROC_PAD_SOURCE,
+ video_dev->entity.name, 0);
+ goto error_video_device;
+ }
+
+ return 0;
+
+error_video_device:
+ vb2_video_unregister_device(video_dev);
+
+error_media_entity:
+ media_entity_cleanup(&video_dev->entity);
+
+error_mutex:
+ mutex_destroy(&capture->lock);
+
+ return ret;
+}
+
+void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_capture *capture = &isp_dev->capture;
+ struct video_device *video_dev = &capture->video_dev;
+
+ vb2_video_unregister_device(video_dev);
+ media_entity_cleanup(&video_dev->entity);
+ mutex_destroy(&capture->lock);
+}
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
new file mode 100644
index 000000000000..5d54060433f5
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_ISP_CAPTURE_H_
+#define _SUN6I_ISP_CAPTURE_H_
+
+#include <media/v4l2-device.h>
+
+#define SUN6I_ISP_CAPTURE_NAME "sun6i-isp-capture"
+
+#define SUN6I_ISP_CAPTURE_WIDTH_MIN 16
+#define SUN6I_ISP_CAPTURE_WIDTH_MAX 3264
+#define SUN6I_ISP_CAPTURE_HEIGHT_MIN 16
+#define SUN6I_ISP_CAPTURE_HEIGHT_MAX 2448
+
+struct sun6i_isp_device;
+
+struct sun6i_isp_capture_format {
+ u32 pixelformat;
+ u8 output_format;
+};
+
+#undef current
+struct sun6i_isp_capture_state {
+ struct list_head queue;
+ spinlock_t lock; /* Queue and buffers lock. */
+
+ struct sun6i_isp_buffer *pending;
+ struct sun6i_isp_buffer *current;
+ struct sun6i_isp_buffer *complete;
+
+ unsigned int sequence;
+ bool streaming;
+};
+
+struct sun6i_isp_capture {
+ struct sun6i_isp_capture_state state;
+
+ struct video_device video_dev;
+ struct vb2_queue queue;
+ struct mutex lock; /* Queue lock. */
+ struct media_pad pad;
+
+ struct v4l2_format format;
+};
+
+/* Helpers */
+
+void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev,
+ unsigned int *width, unsigned int *height);
+void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev,
+ u32 *pixelformat);
+
+/* Format */
+
+const struct sun6i_isp_capture_format *
+sun6i_isp_capture_format_find(u32 pixelformat);
+
+/* Capture */
+
+void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev);
+
+/* State */
+
+void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev,
+ bool *update);
+void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev);
+void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev);
+
+/* Capture */
+
+int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev);
+void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev);
+
+#endif
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
new file mode 100644
index 000000000000..49eca3b09107
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
@@ -0,0 +1,573 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-vmalloc.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "sun6i_isp.h"
+#include "sun6i_isp_params.h"
+#include "sun6i_isp_reg.h"
+#include "uapi/sun6i-isp-config.h"
+
+/* Params */
+
+static const struct sun6i_isp_params_config sun6i_isp_params_config_default = {
+ .modules_used = SUN6I_ISP_MODULE_BAYER,
+
+ .bayer = {
+ .offset_r = 32,
+ .offset_gr = 32,
+ .offset_gb = 32,
+ .offset_b = 32,
+
+ .gain_r = 256,
+ .gain_gr = 256,
+ .gain_gb = 256,
+ .gain_b = 256,
+
+ },
+
+ .bdnf = {
+ .in_dis_min = 8,
+ .in_dis_max = 16,
+
+ .coefficients_g = { 15, 4, 1 },
+ .coefficients_rb = { 15, 4 },
+ },
+};
+
+static void sun6i_isp_params_configure_ob(struct sun6i_isp_device *isp_dev)
+{
+ unsigned int width, height;
+
+ sun6i_isp_proc_dimensions(isp_dev, &width, &height);
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SIZE_REG,
+ SUN6I_ISP_OB_SIZE_WIDTH(width) |
+ SUN6I_ISP_OB_SIZE_HEIGHT(height));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_VALID_REG,
+ SUN6I_ISP_OB_VALID_WIDTH(width) |
+ SUN6I_ISP_OB_VALID_HEIGHT(height));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SRC0_VALID_START_REG,
+ SUN6I_ISP_OB_SRC0_VALID_START_HORZ(0) |
+ SUN6I_ISP_OB_SRC0_VALID_START_VERT(0));
+}
+
+static void sun6i_isp_params_configure_ae(struct sun6i_isp_device *isp_dev)
+{
+ /* These are default values that need to be set to get an output. */
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_AE_CFG_REG,
+ SUN6I_ISP_AE_CFG_LOW_BRI_TH(0xff) |
+ SUN6I_ISP_AE_CFG_HORZ_NUM(8) |
+ SUN6I_ISP_AE_CFG_HIGH_BRI_TH(0xf00) |
+ SUN6I_ISP_AE_CFG_VERT_NUM(8));
+}
+
+static void
+sun6i_isp_params_configure_bayer(struct sun6i_isp_device *isp_dev,
+ const struct sun6i_isp_params_config *config)
+{
+ const struct sun6i_isp_params_config_bayer *bayer = &config->bayer;
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET0_REG,
+ SUN6I_ISP_BAYER_OFFSET0_R(bayer->offset_r) |
+ SUN6I_ISP_BAYER_OFFSET0_GR(bayer->offset_gr));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET1_REG,
+ SUN6I_ISP_BAYER_OFFSET1_GB(bayer->offset_gb) |
+ SUN6I_ISP_BAYER_OFFSET1_B(bayer->offset_b));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN0_REG,
+ SUN6I_ISP_BAYER_GAIN0_R(bayer->gain_r) |
+ SUN6I_ISP_BAYER_GAIN0_GR(bayer->gain_gr));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN1_REG,
+ SUN6I_ISP_BAYER_GAIN1_GB(bayer->gain_gb) |
+ SUN6I_ISP_BAYER_GAIN1_B(bayer->gain_b));
+}
+
+static void sun6i_isp_params_configure_wb(struct sun6i_isp_device *isp_dev)
+{
+ /* These are default values that need to be set to get an output. */
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN0_REG,
+ SUN6I_ISP_WB_GAIN0_R(256) |
+ SUN6I_ISP_WB_GAIN0_GR(256));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN1_REG,
+ SUN6I_ISP_WB_GAIN1_GB(256) |
+ SUN6I_ISP_WB_GAIN1_B(256));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_CFG_REG,
+ SUN6I_ISP_WB_CFG_CLIP(0xfff));
+}
+
+static void sun6i_isp_params_configure_base(struct sun6i_isp_device *isp_dev)
+{
+ sun6i_isp_params_configure_ae(isp_dev);
+ sun6i_isp_params_configure_ob(isp_dev);
+ sun6i_isp_params_configure_wb(isp_dev);
+}
+
+static void
+sun6i_isp_params_configure_bdnf(struct sun6i_isp_device *isp_dev,
+ const struct sun6i_isp_params_config *config)
+{
+ const struct sun6i_isp_params_config_bdnf *bdnf = &config->bdnf;
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_CFG_REG,
+ SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(bdnf->in_dis_min) |
+ SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(bdnf->in_dis_max));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_RB_REG,
+ SUN6I_ISP_BDNF_COEF_RB(0, bdnf->coefficients_rb[0]) |
+ SUN6I_ISP_BDNF_COEF_RB(1, bdnf->coefficients_rb[1]) |
+ SUN6I_ISP_BDNF_COEF_RB(2, bdnf->coefficients_rb[2]) |
+ SUN6I_ISP_BDNF_COEF_RB(3, bdnf->coefficients_rb[3]) |
+ SUN6I_ISP_BDNF_COEF_RB(4, bdnf->coefficients_rb[4]));
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_G_REG,
+ SUN6I_ISP_BDNF_COEF_G(0, bdnf->coefficients_g[0]) |
+ SUN6I_ISP_BDNF_COEF_G(1, bdnf->coefficients_g[1]) |
+ SUN6I_ISP_BDNF_COEF_G(2, bdnf->coefficients_g[2]) |
+ SUN6I_ISP_BDNF_COEF_G(3, bdnf->coefficients_g[3]) |
+ SUN6I_ISP_BDNF_COEF_G(4, bdnf->coefficients_g[4]) |
+ SUN6I_ISP_BDNF_COEF_G(5, bdnf->coefficients_g[5]) |
+ SUN6I_ISP_BDNF_COEF_G(6, bdnf->coefficients_g[6]));
+}
+
+static void
+sun6i_isp_params_configure_modules(struct sun6i_isp_device *isp_dev,
+ const struct sun6i_isp_params_config *config)
+{
+ u32 value;
+
+ if (config->modules_used & SUN6I_ISP_MODULE_BDNF)
+ sun6i_isp_params_configure_bdnf(isp_dev, config);
+
+ if (config->modules_used & SUN6I_ISP_MODULE_BAYER)
+ sun6i_isp_params_configure_bayer(isp_dev, config);
+
+ value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG);
+ /* Clear all modules but keep input configuration. */
+ value &= SUN6I_ISP_MODULE_EN_SRC0 | SUN6I_ISP_MODULE_EN_SRC1;
+
+ if (config->modules_used & SUN6I_ISP_MODULE_BDNF)
+ value |= SUN6I_ISP_MODULE_EN_BDNF;
+
+ /* Bayer stage is always enabled. */
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value);
+}
+
+void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ sun6i_isp_params_configure_base(isp_dev);
+
+ /* Default config is only applied at the very first stream start. */
+ if (state->configured)
+ goto complete;
+
+ sun6i_isp_params_configure_modules(isp_dev,
+ &sun6i_isp_params_config_default);
+
+ state->configured = true;
+
+complete:
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+/* State */
+
+static void sun6i_isp_params_state_cleanup(struct sun6i_isp_device *isp_dev,
+ bool error)
+{
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ struct sun6i_isp_buffer *isp_buffer;
+ struct vb2_buffer *vb2_buffer;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ if (state->pending) {
+ vb2_buffer = &state->pending->v4l2_buffer.vb2_buf;
+ vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
+ VB2_BUF_STATE_QUEUED);
+ }
+
+ list_for_each_entry(isp_buffer, &state->queue, list) {
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+ vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
+ VB2_BUF_STATE_QUEUED);
+ }
+
+ INIT_LIST_HEAD(&state->queue);
+
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev,
+ bool *update)
+{
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ struct sun6i_isp_buffer *isp_buffer;
+ struct vb2_buffer *vb2_buffer;
+ const struct sun6i_isp_params_config *config;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ if (list_empty(&state->queue))
+ goto complete;
+
+ if (state->pending)
+ goto complete;
+
+ isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer,
+ list);
+
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+ config = vb2_plane_vaddr(vb2_buffer, 0);
+
+ sun6i_isp_params_configure_modules(isp_dev, config);
+
+ list_del(&isp_buffer->list);
+
+ state->pending = isp_buffer;
+
+ if (update)
+ *update = true;
+
+complete:
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ struct sun6i_isp_buffer *isp_buffer;
+ struct vb2_buffer *vb2_buffer;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+
+ if (!state->pending)
+ goto complete;
+
+ isp_buffer = state->pending;
+ vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf;
+
+ vb2_buffer->timestamp = ktime_get_ns();
+
+ /* Parameters will be applied starting from the next frame. */
+ isp_buffer->v4l2_buffer.sequence = isp_dev->capture.state.sequence + 1;
+
+ vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE);
+
+ state->pending = NULL;
+
+complete:
+ spin_unlock_irqrestore(&state->lock, flags);
+}
+
+/* Queue */
+
+static int sun6i_isp_params_queue_setup(struct vb2_queue *queue,
+ unsigned int *buffers_count,
+ unsigned int *planes_count,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ unsigned int size = isp_dev->params.format.fmt.meta.buffersize;
+
+ if (*planes_count)
+ return sizes[0] < size ? -EINVAL : 0;
+
+ *planes_count = 1;
+ sizes[0] = size;
+
+ return 0;
+}
+
+static int sun6i_isp_params_buffer_prepare(struct vb2_buffer *vb2_buffer)
+{
+ struct sun6i_isp_device *isp_dev =
+ vb2_get_drv_priv(vb2_buffer->vb2_queue);
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ unsigned int size = isp_dev->params.format.fmt.meta.buffersize;
+
+ if (vb2_plane_size(vb2_buffer, 0) < size) {
+ v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n",
+ vb2_plane_size(vb2_buffer, 0), size);
+ return -EINVAL;
+ }
+
+ vb2_set_plane_payload(vb2_buffer, 0, size);
+
+ return 0;
+}
+
+static void sun6i_isp_params_buffer_queue(struct vb2_buffer *vb2_buffer)
+{
+ struct sun6i_isp_device *isp_dev =
+ vb2_get_drv_priv(vb2_buffer->vb2_queue);
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer);
+ struct sun6i_isp_buffer *isp_buffer =
+ container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer);
+ bool capture_streaming = isp_dev->capture.state.streaming;
+ unsigned long flags;
+
+ spin_lock_irqsave(&state->lock, flags);
+ list_add_tail(&isp_buffer->list, &state->queue);
+ spin_unlock_irqrestore(&state->lock, flags);
+
+ if (state->streaming && capture_streaming)
+ sun6i_isp_state_update(isp_dev, false);
+}
+
+static int sun6i_isp_params_start_streaming(struct vb2_queue *queue,
+ unsigned int count)
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+ bool capture_streaming = isp_dev->capture.state.streaming;
+
+ state->streaming = true;
+
+ /*
+ * Update the state as soon as possible if capture is streaming,
+ * otherwise it will be applied when capture starts streaming.
+ */
+
+ if (capture_streaming)
+ sun6i_isp_state_update(isp_dev, false);
+
+ return 0;
+}
+
+static void sun6i_isp_params_stop_streaming(struct vb2_queue *queue)
+{
+ struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue);
+ struct sun6i_isp_params_state *state = &isp_dev->params.state;
+
+ state->streaming = false;
+ sun6i_isp_params_state_cleanup(isp_dev, true);
+}
+
+static const struct vb2_ops sun6i_isp_params_queue_ops = {
+ .queue_setup = sun6i_isp_params_queue_setup,
+ .buf_prepare = sun6i_isp_params_buffer_prepare,
+ .buf_queue = sun6i_isp_params_buffer_queue,
+ .start_streaming = sun6i_isp_params_start_streaming,
+ .stop_streaming = sun6i_isp_params_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+/* Video Device */
+
+static int sun6i_isp_params_querycap(struct file *file, void *private,
+ struct v4l2_capability *capability)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+ struct video_device *video_dev = &isp_dev->params.video_dev;
+
+ strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver));
+ strscpy(capability->card, video_dev->name, sizeof(capability->card));
+ snprintf(capability->bus_info, sizeof(capability->bus_info),
+ "platform:%s", dev_name(isp_dev->dev));
+
+ return 0;
+}
+
+static int sun6i_isp_params_enum_fmt(struct file *file, void *private,
+ struct v4l2_fmtdesc *fmtdesc)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+ struct v4l2_meta_format *params_format =
+ &isp_dev->params.format.fmt.meta;
+
+ if (fmtdesc->index > 0)
+ return -EINVAL;
+
+ fmtdesc->pixelformat = params_format->dataformat;
+
+ return 0;
+}
+
+static int sun6i_isp_params_g_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
+{
+ struct sun6i_isp_device *isp_dev = video_drvdata(file);
+
+ *format = isp_dev->params.format;
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops sun6i_isp_params_ioctl_ops = {
+ .vidioc_querycap = sun6i_isp_params_querycap,
+
+ .vidioc_enum_fmt_meta_out = sun6i_isp_params_enum_fmt,
+ .vidioc_g_fmt_meta_out = sun6i_isp_params_g_fmt,
+ .vidioc_s_fmt_meta_out = sun6i_isp_params_g_fmt,
+ .vidioc_try_fmt_meta_out = sun6i_isp_params_g_fmt,
+
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_log_status = v4l2_ctrl_log_status,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_file_operations sun6i_isp_params_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .mmap = vb2_fop_mmap,
+ .poll = vb2_fop_poll,
+};
+
+/* Params */
+
+int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_params *params = &isp_dev->params;
+ struct sun6i_isp_params_state *state = ¶ms->state;
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev;
+ struct video_device *video_dev = ¶ms->video_dev;
+ struct vb2_queue *queue = &isp_dev->params.queue;
+ struct media_pad *pad = &isp_dev->params.pad;
+ struct v4l2_format *format = &isp_dev->params.format;
+ struct v4l2_meta_format *params_format = &format->fmt.meta;
+ int ret;
+
+ /* State */
+
+ INIT_LIST_HEAD(&state->queue);
+ spin_lock_init(&state->lock);
+
+ /* Media Pads */
+
+ pad->flags = MEDIA_PAD_FL_SOURCE | MEDIA_PAD_FL_MUST_CONNECT;
+
+ ret = media_entity_pads_init(&video_dev->entity, 1, pad);
+ if (ret)
+ goto error_mutex;
+
+ /* Queue */
+
+ mutex_init(¶ms->lock);
+
+ queue->type = V4L2_BUF_TYPE_META_OUTPUT;
+ queue->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+ queue->buf_struct_size = sizeof(struct sun6i_isp_buffer);
+ queue->ops = &sun6i_isp_params_queue_ops;
+ queue->mem_ops = &vb2_vmalloc_memops;
+ queue->min_buffers_needed = 1;
+ queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ queue->lock = ¶ms->lock;
+ queue->dev = isp_dev->dev;
+ queue->drv_priv = isp_dev;
+
+ ret = vb2_queue_init(queue);
+ if (ret) {
+ v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret);
+ goto error_media_entity;
+ }
+
+ /* V4L2 Format */
+
+ format->type = queue->type;
+ params_format->dataformat = V4L2_META_FMT_SUN6I_ISP_PARAMS;
+ params_format->buffersize = sizeof(struct sun6i_isp_params_config);
+
+ /* Video Device */
+
+ strscpy(video_dev->name, SUN6I_ISP_PARAMS_NAME,
+ sizeof(video_dev->name));
+ video_dev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING;
+ video_dev->vfl_dir = VFL_DIR_TX;
+ video_dev->release = video_device_release_empty;
+ video_dev->fops = &sun6i_isp_params_fops;
+ video_dev->ioctl_ops = &sun6i_isp_params_ioctl_ops;
+ video_dev->v4l2_dev = v4l2_dev;
+ video_dev->queue = queue;
+ video_dev->lock = ¶ms->lock;
+
+ video_set_drvdata(video_dev, isp_dev);
+
+ ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ v4l2_err(v4l2_dev, "failed to register video device: %d\n",
+ ret);
+ goto error_media_entity;
+ }
+
+ v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
+ video_device_node_name(video_dev));
+
+ /* Media Pad Link */
+
+ ret = media_create_pad_link(&video_dev->entity, 0,
+ &proc_subdev->entity,
+ SUN6I_ISP_PROC_PAD_SINK_PARAMS,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret < 0) {
+ v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
+ video_dev->entity.name, 0, proc_subdev->entity.name,
+ SUN6I_ISP_PROC_PAD_SINK_PARAMS);
+ goto error_video_device;
+ }
+
+ return 0;
+
+error_video_device:
+ vb2_video_unregister_device(video_dev);
+
+error_media_entity:
+ media_entity_cleanup(&video_dev->entity);
+
+error_mutex:
+ mutex_destroy(¶ms->lock);
+
+ return ret;
+}
+
+void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct sun6i_isp_params *params = &isp_dev->params;
+ struct video_device *video_dev = ¶ms->video_dev;
+
+ vb2_video_unregister_device(video_dev);
+ media_entity_cleanup(&video_dev->entity);
+ mutex_destroy(¶ms->lock);
+}
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
new file mode 100644
index 000000000000..f919b0a0cc9f
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_ISP_PARAMS_H_
+#define _SUN6I_ISP_PARAMS_H_
+
+#include <media/v4l2-device.h>
+
+#define SUN6I_ISP_PARAMS_NAME "sun6i-isp-params"
+
+struct sun6i_isp_device;
+
+struct sun6i_isp_params_state {
+ struct list_head queue; /* Queue and buffers lock. */
+ spinlock_t lock;
+
+ struct sun6i_isp_buffer *pending;
+
+ bool configured;
+ bool streaming;
+};
+
+struct sun6i_isp_params {
+ struct sun6i_isp_params_state state;
+
+ struct video_device video_dev;
+ struct vb2_queue queue;
+ struct mutex lock; /* Queue lock. */
+ struct media_pad pad;
+
+ struct v4l2_format format;
+};
+
+/* Params */
+
+void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev);
+
+/* State */
+
+void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev,
+ bool *update);
+void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev);
+
+/* Params */
+
+int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev);
+void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev);
+
+#endif
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
new file mode 100644
index 000000000000..704fab785b8c
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
@@ -0,0 +1,599 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#include "sun6i_isp.h"
+#include "sun6i_isp_capture.h"
+#include "sun6i_isp_params.h"
+#include "sun6i_isp_proc.h"
+#include "sun6i_isp_reg.h"
+
+/* Helpers */
+
+void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev,
+ unsigned int *width, unsigned int *height)
+{
+ if (width)
+ *width = isp_dev->proc.mbus_format.width;
+ if (height)
+ *height = isp_dev->proc.mbus_format.height;
+}
+
+/* Format */
+
+static const struct sun6i_isp_proc_format sun6i_isp_proc_formats[] = {
+ {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB,
+ },
+
+ {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB,
+ },
+};
+
+const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(sun6i_isp_proc_formats); i++)
+ if (sun6i_isp_proc_formats[i].mbus_code == mbus_code)
+ return &sun6i_isp_proc_formats[i];
+
+ return NULL;
+}
+
+/* Processor */
+
+static void sun6i_isp_proc_irq_enable(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+
+ regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG,
+ SUN6I_ISP_FE_INT_EN_FINISH |
+ SUN6I_ISP_FE_INT_EN_START |
+ SUN6I_ISP_FE_INT_EN_PARA_SAVE |
+ SUN6I_ISP_FE_INT_EN_PARA_LOAD |
+ SUN6I_ISP_FE_INT_EN_SRC0_FIFO |
+ SUN6I_ISP_FE_INT_EN_ROT_FINISH);
+}
+
+static void sun6i_isp_proc_irq_disable(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+
+ regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0);
+}
+
+static void sun6i_isp_proc_irq_clear(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+
+ regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0);
+ regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG,
+ SUN6I_ISP_FE_INT_STA_CLEAR);
+}
+
+static void sun6i_isp_proc_enable(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+ u8 mode;
+
+ /* Frontend */
+
+ if (isp_dev->proc.source == &isp_dev->proc.source_csi0)
+ mode = SUN6I_ISP_SRC_MODE_CSI(0);
+ else if (isp_dev->proc.source == &isp_dev->proc.source_csi1)
+ mode = SUN6I_ISP_SRC_MODE_CSI(1);
+
+ regmap_write(regmap, SUN6I_ISP_FE_CFG_REG,
+ SUN6I_ISP_FE_CFG_EN | SUN6I_ISP_FE_CFG_SRC0_MODE(mode));
+
+ regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG,
+ SUN6I_ISP_FE_CTRL_VCAP_EN | SUN6I_ISP_FE_CTRL_PARA_READY);
+}
+
+static void sun6i_isp_proc_disable(struct sun6i_isp_device *isp_dev)
+{
+ struct regmap *regmap = isp_dev->regmap;
+
+ /* Frontend */
+
+ regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, 0);
+ regmap_write(regmap, SUN6I_ISP_FE_CFG_REG, 0);
+}
+
+static void sun6i_isp_proc_configure(struct sun6i_isp_device *isp_dev)
+{
+ struct v4l2_mbus_framefmt *mbus_format = &isp_dev->proc.mbus_format;
+ const struct sun6i_isp_proc_format *format;
+ u32 value;
+
+ /* Module */
+
+ value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG);
+ value |= SUN6I_ISP_MODULE_EN_SRC0;
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value);
+
+ /* Input */
+
+ format = sun6i_isp_proc_format_find(mbus_format->code);
+ if (WARN_ON(!format))
+ return;
+
+ sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODE_REG,
+ SUN6I_ISP_MODE_INPUT_FMT(format->input_format) |
+ SUN6I_ISP_MODE_INPUT_YUV_SEQ(format->input_yuv_seq) |
+ SUN6I_ISP_MODE_SHARP(1) |
+ SUN6I_ISP_MODE_HIST(2));
+}
+
+/* V4L2 Subdev */
+
+static int sun6i_isp_proc_s_stream(struct v4l2_subdev *subdev, int on)
+{
+ struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
+ struct device *dev = isp_dev->dev;
+ struct v4l2_subdev *source_subdev;
+ /* Initialize to 0 to use both in disable label (ret != 0) and off. */
+ int ret = 0;
+
+ /* Source */
+
+ if (!isp_dev->proc.source)
+ return -ENODEV;
+
+ source_subdev = isp_dev->proc.source->subdev;
+
+ if (!on) {
+ sun6i_isp_proc_irq_disable(isp_dev);
+ v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ goto disable;
+ }
+
+ /* PM */
+
+ ret = pm_runtime_resume_and_get(dev);
+ if (ret < 0)
+ return ret;
+
+ /* Clear */
+
+ sun6i_isp_proc_irq_clear(isp_dev);
+
+ /* Configure */
+
+ sun6i_isp_tables_configure(isp_dev);
+ sun6i_isp_params_configure(isp_dev);
+ sun6i_isp_proc_configure(isp_dev);
+ sun6i_isp_capture_configure(isp_dev);
+
+ /* State Update */
+
+ sun6i_isp_state_update(isp_dev, true);
+
+ /* Enable */
+
+ sun6i_isp_proc_irq_enable(isp_dev);
+ sun6i_isp_proc_enable(isp_dev);
+
+ ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
+ if (ret && ret != -ENOIOCTLCMD) {
+ sun6i_isp_proc_irq_disable(isp_dev);
+ goto disable;
+ }
+
+ return 0;
+
+disable:
+ sun6i_isp_proc_disable(isp_dev);
+
+ isp_dev->proc.source = NULL;
+
+ pm_runtime_put(dev);
+
+ return ret;
+}
+
+static const struct v4l2_subdev_video_ops sun6i_isp_proc_video_ops = {
+ .s_stream = sun6i_isp_proc_s_stream,
+};
+
+static void
+sun6i_isp_proc_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format)
+{
+ if (!sun6i_isp_proc_format_find(mbus_format->code))
+ mbus_format->code = sun6i_isp_proc_formats[0].mbus_code;
+
+ mbus_format->field = V4L2_FIELD_NONE;
+ mbus_format->colorspace = V4L2_COLORSPACE_RAW;
+ mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT;
+ mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int sun6i_isp_proc_init_cfg(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state)
+{
+ unsigned int pad = SUN6I_ISP_PROC_PAD_SINK_CSI;
+ struct v4l2_mbus_framefmt *mbus_format =
+ v4l2_subdev_get_try_format(subdev, state, pad);
+
+ mbus_format->code = sun6i_isp_proc_formats[0].mbus_code;
+ mbus_format->width = 640;
+ mbus_format->height = 480;
+
+ sun6i_isp_proc_mbus_format_prepare(mbus_format);
+
+ return 0;
+}
+
+static int
+sun6i_isp_proc_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code_enum)
+{
+ if (code_enum->index >= ARRAY_SIZE(sun6i_isp_proc_formats))
+ return -EINVAL;
+
+ code_enum->code = sun6i_isp_proc_formats[code_enum->index].mbus_code;
+
+ return 0;
+}
+
+static int sun6i_isp_proc_get_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *mbus_format = *v4l2_subdev_get_try_format(subdev, state,
+ format->pad);
+ else
+ *mbus_format = isp_dev->proc.mbus_format;
+
+ return 0;
+}
+
+static int sun6i_isp_proc_set_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ sun6i_isp_proc_mbus_format_prepare(mbus_format);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *v4l2_subdev_get_try_format(subdev, state, format->pad) =
+ *mbus_format;
+ else
+ isp_dev->proc.mbus_format = *mbus_format;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops sun6i_isp_proc_pad_ops = {
+ .init_cfg = sun6i_isp_proc_init_cfg,
+ .enum_mbus_code = sun6i_isp_proc_enum_mbus_code,
+ .get_fmt = sun6i_isp_proc_get_fmt,
+ .set_fmt = sun6i_isp_proc_set_fmt,
+};
+
+const struct v4l2_subdev_ops sun6i_isp_proc_subdev_ops = {
+ .video = &sun6i_isp_proc_video_ops,
+ .pad = &sun6i_isp_proc_pad_ops,
+};
+
+/* Media Entity */
+
+static int sun6i_isp_proc_link_validate(struct media_link *link)
+{
+ struct v4l2_subdev *subdev =
+ media_entity_to_v4l2_subdev(link->sink->entity);
+ struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev);
+ struct sun6i_isp_proc *proc = &isp_dev->proc;
+ struct device *dev = isp_dev->dev;
+ struct v4l2_subdev *source_subdev =
+ media_entity_to_v4l2_subdev(link->source->entity);
+ int ret;
+
+ /* Only care about sink index 0, connected to CSI. */
+ if (link->sink->index > 0)
+ return 0;
+
+ /* Only support one enabled source at a time. */
+ if (proc->source) {
+ dev_err(dev, "more than one source is connected to proc\n");
+ return -EBUSY;
+ }
+
+ ret = v4l2_subdev_link_validate(link);
+ if (ret)
+ return ret;
+
+ if (source_subdev == proc->source_csi0.subdev)
+ proc->source = &proc->source_csi0;
+ else if (source_subdev == proc->source_csi1.subdev)
+ proc->source = &proc->source_csi1;
+ else
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct media_entity_operations sun6i_isp_proc_entity_ops = {
+ .link_validate = sun6i_isp_proc_link_validate,
+};
+
+/* V4L2 Async */
+
+static int sun6i_isp_proc_link(struct sun6i_isp_device *isp_dev,
+ int sink_pad_index,
+ struct v4l2_subdev *remote_subdev, bool enabled)
+{
+ struct device *dev = isp_dev->dev;
+ struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
+ struct media_entity *sink_entity = &subdev->entity;
+ struct media_entity *source_entity = &remote_subdev->entity;
+ int source_pad_index;
+ int ret;
+
+ /* Get the first remote source pad. */
+ ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (ret < 0) {
+ dev_err(dev, "missing source pad in external entity %s\n",
+ source_entity->name);
+ return -EINVAL;
+ }
+
+ source_pad_index = ret;
+
+ dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
+ source_pad_index, sink_entity->name, sink_pad_index);
+
+ ret = media_create_pad_link(source_entity, source_pad_index,
+ sink_entity, sink_pad_index,
+ enabled ? MEDIA_LNK_FL_ENABLED : 0);
+ if (ret < 0) {
+ dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
+ source_entity->name, source_pad_index,
+ sink_entity->name, sink_pad_index);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int sun6i_isp_proc_notifier_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *remote_subdev,
+ struct v4l2_async_subdev *async_subdev)
+{
+ struct sun6i_isp_device *isp_dev =
+ container_of(notifier, struct sun6i_isp_device, proc.notifier);
+ struct sun6i_isp_proc *proc = &isp_dev->proc;
+ struct sun6i_isp_proc_source *source = NULL;
+ struct fwnode_handle *fwnode = dev_fwnode(isp_dev->dev);
+ struct fwnode_handle *handle = NULL;
+ bool enabled;
+ int ret;
+
+ while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
+ struct fwnode_endpoint endpoint = { 0 };
+ struct fwnode_handle *remote_fwnode;
+
+ remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
+ if (!remote_fwnode)
+ continue;
+
+ if (remote_fwnode != remote_subdev->fwnode)
+ goto next;
+
+ ret = fwnode_graph_parse_endpoint(handle, &endpoint);
+ if (ret < 0)
+ goto next;
+
+ switch (endpoint.port) {
+ case SUN6I_ISP_PORT_CSI0:
+ source = &proc->source_csi0;
+ enabled = true;
+ break;
+ case SUN6I_ISP_PORT_CSI1:
+ source = &proc->source_csi1;
+ enabled = !proc->source_csi0.expected;
+ break;
+ default:
+ break;
+ }
+
+next:
+ fwnode_handle_put(remote_fwnode);
+ }
+
+ if (!source)
+ return -EINVAL;
+
+ source->subdev = remote_subdev;
+
+ return sun6i_isp_proc_link(isp_dev, SUN6I_ISP_PROC_PAD_SINK_CSI,
+ remote_subdev, enabled);
+}
+
+static int
+sun6i_isp_proc_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+ struct sun6i_isp_device *isp_dev =
+ container_of(notifier, struct sun6i_isp_device, proc.notifier);
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ int ret;
+
+ ret = v4l2_device_register_subdev_nodes(v4l2_dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct v4l2_async_notifier_operations
+sun6i_isp_proc_notifier_ops = {
+ .bound = sun6i_isp_proc_notifier_bound,
+ .complete = sun6i_isp_proc_notifier_complete,
+};
+
+/* Processor */
+
+static int sun6i_isp_proc_source_setup(struct sun6i_isp_device *isp_dev,
+ struct sun6i_isp_proc_source *source,
+ u32 port)
+{
+ struct device *dev = isp_dev->dev;
+ struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier;
+ struct v4l2_fwnode_endpoint *endpoint = &source->endpoint;
+ struct v4l2_async_subdev *async_subdev;
+ struct fwnode_handle *handle = NULL;
+ int ret;
+
+ handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0);
+ if (!handle)
+ return -ENODEV;
+
+ ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
+ if (ret)
+ goto complete;
+
+ async_subdev = v4l2_async_nf_add_fwnode_remote(notifier, handle,
+ struct v4l2_async_subdev);
+ if (IS_ERR(async_subdev)) {
+ ret = PTR_ERR(async_subdev);
+ goto complete;
+ }
+
+ source->expected = true;
+
+complete:
+ fwnode_handle_put(handle);
+
+ return ret;
+}
+
+int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev)
+{
+ struct device *dev = isp_dev->dev;
+ struct sun6i_isp_proc *proc = &isp_dev->proc;
+ struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev;
+ struct v4l2_async_notifier *notifier = &proc->notifier;
+ struct v4l2_subdev *subdev = &proc->subdev;
+ struct media_pad *pads = proc->pads;
+ int ret;
+
+ /* V4L2 Subdev */
+
+ v4l2_subdev_init(subdev, &sun6i_isp_proc_subdev_ops);
+ strscpy(subdev->name, SUN6I_ISP_PROC_NAME, sizeof(subdev->name));
+ subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ subdev->owner = THIS_MODULE;
+ subdev->dev = dev;
+
+ v4l2_set_subdevdata(subdev, isp_dev);
+
+ /* Media Entity */
+
+ subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
+ subdev->entity.ops = &sun6i_isp_proc_entity_ops;
+
+ /* Media Pads */
+
+ pads[SUN6I_ISP_PROC_PAD_SINK_CSI].flags = MEDIA_PAD_FL_SINK |
+ MEDIA_PAD_FL_MUST_CONNECT;
+ pads[SUN6I_ISP_PROC_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK |
+ MEDIA_PAD_FL_MUST_CONNECT;
+ pads[SUN6I_ISP_PROC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&subdev->entity, SUN6I_ISP_PROC_PAD_COUNT,
+ pads);
+ if (ret)
+ return ret;
+
+ /* V4L2 Subdev */
+
+ ret = v4l2_device_register_subdev(v4l2_dev, subdev);
+ if (ret < 0) {
+ v4l2_err(v4l2_dev, "failed to register v4l2 subdev: %d\n", ret);
+ goto error_media_entity;
+ }
+
+ /* V4L2 Async */
+
+ v4l2_async_nf_init(notifier);
+ notifier->ops = &sun6i_isp_proc_notifier_ops;
+
+ sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi0,
+ SUN6I_ISP_PORT_CSI0);
+ sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi1,
+ SUN6I_ISP_PORT_CSI1);
+
+ ret = v4l2_async_nf_register(v4l2_dev, notifier);
+ if (ret) {
+ v4l2_err(v4l2_dev,
+ "failed to register v4l2 async notifier: %d\n", ret);
+ goto error_v4l2_async_notifier;
+ }
+
+ return 0;
+
+error_v4l2_async_notifier:
+ v4l2_async_nf_cleanup(notifier);
+
+ v4l2_device_unregister_subdev(subdev);
+
+error_media_entity:
+ media_entity_cleanup(&subdev->entity);
+
+ return ret;
+}
+
+void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev)
+{
+ struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier;
+ struct v4l2_subdev *subdev = &isp_dev->proc.subdev;
+
+ v4l2_async_nf_unregister(notifier);
+ v4l2_async_nf_cleanup(notifier);
+
+ v4l2_device_unregister_subdev(subdev);
+ media_entity_cleanup(&subdev->entity);
+}
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
new file mode 100644
index 000000000000..10b4c0a40cb3
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_ISP_PROC_H_
+#define _SUN6I_ISP_PROC_H_
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#define SUN6I_ISP_PROC_NAME "sun6i-isp-proc"
+
+enum sun6i_isp_proc_pad {
+ SUN6I_ISP_PROC_PAD_SINK_CSI = 0,
+ SUN6I_ISP_PROC_PAD_SINK_PARAMS = 1,
+ SUN6I_ISP_PROC_PAD_SOURCE = 2,
+ SUN6I_ISP_PROC_PAD_COUNT = 3,
+};
+
+struct sun6i_isp_device;
+
+struct sun6i_isp_proc_format {
+ u32 mbus_code;
+ u8 input_format;
+ u8 input_yuv_seq;
+};
+
+struct sun6i_isp_proc_source {
+ struct v4l2_subdev *subdev;
+ struct v4l2_fwnode_endpoint endpoint;
+ bool expected;
+};
+
+struct sun6i_isp_proc {
+ struct v4l2_subdev subdev;
+ struct media_pad pads[3];
+ struct v4l2_async_notifier notifier;
+ struct v4l2_mbus_framefmt mbus_format;
+
+ struct sun6i_isp_proc_source source_csi0;
+ struct sun6i_isp_proc_source source_csi1;
+ struct sun6i_isp_proc_source *source;
+};
+
+/* Helpers */
+
+void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev,
+ unsigned int *width, unsigned int *height);
+
+/* Format */
+
+const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code);
+
+/* Proc */
+
+int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev);
+void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev);
+
+#endif
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
new file mode 100644
index 000000000000..1e6f42005235
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
@@ -0,0 +1,275 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_ISP_REG_H_
+#define _SUN6I_ISP_REG_H_
+
+#include <linux/kernel.h>
+
+#define SUN6I_ISP_ADDR_VALUE(a) ((a) >> 2)
+
+/* Frontend */
+
+#define SUN6I_ISP_SRC_MODE_DRAM 0
+#define SUN6I_ISP_SRC_MODE_CSI(n) (1 + (n))
+
+#define SUN6I_ISP_FE_CFG_REG 0x0
+#define SUN6I_ISP_FE_CFG_EN BIT(0)
+#define SUN6I_ISP_FE_CFG_SRC0_MODE(v) (((v) << 8) & GENMASK(9, 8))
+#define SUN6I_ISP_FE_CFG_SRC1_MODE(v) (((v) << 16) & GENMASK(17, 16))
+
+#define SUN6I_ISP_FE_CTRL_REG 0x4
+#define SUN6I_ISP_FE_CTRL_SCAP_EN BIT(0)
+#define SUN6I_ISP_FE_CTRL_VCAP_EN BIT(1)
+#define SUN6I_ISP_FE_CTRL_PARA_READY BIT(2)
+#define SUN6I_ISP_FE_CTRL_LUT_UPDATE BIT(3)
+#define SUN6I_ISP_FE_CTRL_LENS_UPDATE BIT(4)
+#define SUN6I_ISP_FE_CTRL_GAMMA_UPDATE BIT(5)
+#define SUN6I_ISP_FE_CTRL_DRC_UPDATE BIT(6)
+#define SUN6I_ISP_FE_CTRL_DISC_UPDATE BIT(7)
+#define SUN6I_ISP_FE_CTRL_OUTPUT_SPEED_CTRL(v) (((v) << 16) & GENMASK(17, 16))
+#define SUN6I_ISP_FE_CTRL_VCAP_READ_START BIT(31)
+
+#define SUN6I_ISP_FE_INT_EN_REG 0x8
+#define SUN6I_ISP_FE_INT_EN_FINISH BIT(0)
+#define SUN6I_ISP_FE_INT_EN_START BIT(1)
+#define SUN6I_ISP_FE_INT_EN_PARA_SAVE BIT(2)
+#define SUN6I_ISP_FE_INT_EN_PARA_LOAD BIT(3)
+#define SUN6I_ISP_FE_INT_EN_SRC0_FIFO BIT(4)
+#define SUN6I_ISP_FE_INT_EN_SRC1_FIFO BIT(5)
+#define SUN6I_ISP_FE_INT_EN_ROT_FINISH BIT(6)
+#define SUN6I_ISP_FE_INT_EN_LINE_NUM_START BIT(7)
+
+#define SUN6I_ISP_FE_INT_STA_REG 0xc
+#define SUN6I_ISP_FE_INT_STA_CLEAR 0xff
+#define SUN6I_ISP_FE_INT_STA_FINISH BIT(0)
+#define SUN6I_ISP_FE_INT_STA_START BIT(1)
+#define SUN6I_ISP_FE_INT_STA_PARA_SAVE BIT(2)
+#define SUN6I_ISP_FE_INT_STA_PARA_LOAD BIT(3)
+#define SUN6I_ISP_FE_INT_STA_SRC0_FIFO BIT(4)
+#define SUN6I_ISP_FE_INT_STA_SRC1_FIFO BIT(5)
+#define SUN6I_ISP_FE_INT_STA_ROT_FINISH BIT(6)
+#define SUN6I_ISP_FE_INT_STA_LINE_NUM_START BIT(7)
+
+/* Only since sun9i-a80-isp. */
+#define SUN6I_ISP_FE_INT_LINE_NUM_REG 0x18
+#define SUN6I_ISP_FE_ROT_OF_CFG_REG 0x1c
+
+/* Buffers/tables */
+
+#define SUN6I_ISP_REG_LOAD_ADDR_REG 0x20
+#define SUN6I_ISP_REG_SAVE_ADDR_REG 0x24
+
+#define SUN6I_ISP_LUT_TABLE_ADDR_REG 0x28
+#define SUN6I_ISP_DRC_TABLE_ADDR_REG 0x2c
+#define SUN6I_ISP_STATS_ADDR_REG 0x30
+
+/* SRAM */
+
+#define SUN6I_ISP_SRAM_RW_OFFSET_REG 0x38
+#define SUN6I_ISP_SRAM_RW_DATA_REG 0x3c
+
+/* Global */
+
+#define SUN6I_ISP_MODULE_EN_REG 0x40
+#define SUN6I_ISP_MODULE_EN_AE BIT(0)
+#define SUN6I_ISP_MODULE_EN_OBC BIT(1)
+#define SUN6I_ISP_MODULE_EN_DPC_LUT BIT(2)
+#define SUN6I_ISP_MODULE_EN_DPC_OTF BIT(3)
+#define SUN6I_ISP_MODULE_EN_BDNF BIT(4)
+#define SUN6I_ISP_MODULE_EN_AWB BIT(6)
+#define SUN6I_ISP_MODULE_EN_WB BIT(7)
+#define SUN6I_ISP_MODULE_EN_LSC BIT(8)
+#define SUN6I_ISP_MODULE_EN_BGC BIT(9)
+#define SUN6I_ISP_MODULE_EN_SAP BIT(10)
+#define SUN6I_ISP_MODULE_EN_AF BIT(11)
+#define SUN6I_ISP_MODULE_EN_RGB2RGB BIT(12)
+#define SUN6I_ISP_MODULE_EN_RGB_DRC BIT(13)
+#define SUN6I_ISP_MODULE_EN_TDNF BIT(15)
+#define SUN6I_ISP_MODULE_EN_AFS BIT(16)
+#define SUN6I_ISP_MODULE_EN_HIST BIT(17)
+#define SUN6I_ISP_MODULE_EN_YUV_GAIN_OFFSET BIT(18)
+#define SUN6I_ISP_MODULE_EN_YUV_DRC BIT(19)
+#define SUN6I_ISP_MODULE_EN_TG BIT(20)
+#define SUN6I_ISP_MODULE_EN_ROT BIT(21)
+#define SUN6I_ISP_MODULE_EN_CONTRAST BIT(22)
+#define SUN6I_ISP_MODULE_EN_SATU BIT(24)
+#define SUN6I_ISP_MODULE_EN_SRC1 BIT(30)
+#define SUN6I_ISP_MODULE_EN_SRC0 BIT(31)
+
+#define SUN6I_ISP_MODE_REG 0x44
+#define SUN6I_ISP_MODE_INPUT_FMT(v) ((v) & GENMASK(2, 0))
+#define SUN6I_ISP_MODE_INPUT_YUV_SEQ(v) (((v) << 3) & GENMASK(4, 3))
+#define SUN6I_ISP_MODE_OTF_DPC(v) (((v) << 16) & BIT(16))
+#define SUN6I_ISP_MODE_SHARP(v) (((v) << 17) & BIT(17))
+#define SUN6I_ISP_MODE_HIST(v) (((v) << 20) & GENMASK(21, 20))
+
+#define SUN6I_ISP_INPUT_FMT_YUV420 0
+#define SUN6I_ISP_INPUT_FMT_YUV422 1
+#define SUN6I_ISP_INPUT_FMT_RAW_BGGR 4
+#define SUN6I_ISP_INPUT_FMT_RAW_RGGB 5
+#define SUN6I_ISP_INPUT_FMT_RAW_GBRG 6
+#define SUN6I_ISP_INPUT_FMT_RAW_GRBG 7
+
+#define SUN6I_ISP_INPUT_YUV_SEQ_YUYV 0
+#define SUN6I_ISP_INPUT_YUV_SEQ_YVYU 1
+#define SUN6I_ISP_INPUT_YUV_SEQ_UYVY 2
+#define SUN6I_ISP_INPUT_YUV_SEQ_VYUY 3
+
+#define SUN6I_ISP_IN_CFG_REG 0x48
+#define SUN6I_ISP_IN_CFG_STRIDE_DIV16(v) ((v) & GENMASK(10, 0))
+
+#define SUN6I_ISP_IN_LUMA_RGB_ADDR0_REG 0x4c
+#define SUN6I_ISP_IN_CHROMA_ADDR0_REG 0x50
+#define SUN6I_ISP_IN_LUMA_RGB_ADDR1_REG 0x54
+#define SUN6I_ISP_IN_CHROMA_ADDR1_REG 0x58
+
+/* AE */
+
+#define SUN6I_ISP_AE_CFG_REG 0x60
+#define SUN6I_ISP_AE_CFG_LOW_BRI_TH(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_AE_CFG_HORZ_NUM(v) (((v) << 12) & GENMASK(15, 12))
+#define SUN6I_ISP_AE_CFG_HIGH_BRI_TH(v) (((v) << 16) & GENMASK(27, 16))
+#define SUN6I_ISP_AE_CFG_VERT_NUM(v) (((v) << 28) & GENMASK(31, 28))
+
+#define SUN6I_ISP_AE_SIZE_REG 0x64
+#define SUN6I_ISP_AE_SIZE_WIDTH(v) ((v) & GENMASK(10, 0))
+#define SUN6I_ISP_AE_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(26, 16))
+
+#define SUN6I_ISP_AE_POS_REG 0x68
+#define SUN6I_ISP_AE_POS_HORZ_START(v) ((v) & GENMASK(10, 0))
+#define SUN6I_ISP_AE_POS_VERT_START(v) (((v) << 16) & GENMASK(26, 16))
+
+/* OB */
+
+#define SUN6I_ISP_OB_SIZE_REG 0x78
+#define SUN6I_ISP_OB_SIZE_WIDTH(v) ((v) & GENMASK(13, 0))
+#define SUN6I_ISP_OB_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(29, 16))
+
+#define SUN6I_ISP_OB_VALID_REG 0x7c
+#define SUN6I_ISP_OB_VALID_WIDTH(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_OB_VALID_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_OB_SRC0_VALID_START_REG 0x80
+#define SUN6I_ISP_OB_SRC0_VALID_START_HORZ(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_OB_SRC0_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_OB_SRC1_VALID_START_REG 0x84
+#define SUN6I_ISP_OB_SRC1_VALID_START_HORZ(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_OB_SRC1_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_OB_SPRITE_REG 0x88
+#define SUN6I_ISP_OB_SPRITE_WIDTH(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_OB_SPRITE_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_OB_SPRITE_START_REG 0x8c
+#define SUN6I_ISP_OB_SPRITE_START_HORZ(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_OB_SPRITE_START_VERT(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_OB_CFG_REG 0x90
+#define SUN6I_ISP_OB_HORZ_POS_REG 0x94
+#define SUN6I_ISP_OB_VERT_PARA_REG 0x98
+#define SUN6I_ISP_OB_OFFSET_FIXED_REG 0x9c
+
+/* BDNF */
+
+#define SUN6I_ISP_BDNF_CFG_REG 0xcc
+#define SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(v) ((v) & GENMASK(7, 0))
+#define SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(v) (((v) << 16) & GENMASK(23, 16))
+
+#define SUN6I_ISP_BDNF_COEF_RB_REG 0xd0
+#define SUN6I_ISP_BDNF_COEF_RB(i, v) (((v) << (4 * (i))) & \
+ GENMASK(4 * (i) + 3, 4 * (i)))
+
+#define SUN6I_ISP_BDNF_COEF_G_REG 0xd4
+#define SUN6I_ISP_BDNF_COEF_G(i, v) (((v) << (4 * (i))) & \
+ GENMASK(4 * (i) + 3, 4 * (i)))
+
+/* Bayer */
+
+#define SUN6I_ISP_BAYER_OFFSET0_REG 0xe0
+#define SUN6I_ISP_BAYER_OFFSET0_R(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_BAYER_OFFSET0_GR(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_BAYER_OFFSET1_REG 0xe4
+#define SUN6I_ISP_BAYER_OFFSET1_GB(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_BAYER_OFFSET1_B(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_BAYER_GAIN0_REG 0xe8
+#define SUN6I_ISP_BAYER_GAIN0_R(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_BAYER_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_BAYER_GAIN1_REG 0xec
+#define SUN6I_ISP_BAYER_GAIN1_GB(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_BAYER_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16))
+
+/* WB */
+
+#define SUN6I_ISP_WB_GAIN0_REG 0x140
+#define SUN6I_ISP_WB_GAIN0_R(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_WB_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_WB_GAIN1_REG 0x144
+#define SUN6I_ISP_WB_GAIN1_GB(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_WB_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16))
+
+#define SUN6I_ISP_WB_CFG_REG 0x148
+#define SUN6I_ISP_WB_CFG_CLIP(v) ((v) & GENMASK(11, 0))
+
+/* Global */
+
+#define SUN6I_ISP_MCH_SIZE_CFG_REG 0x1e0
+#define SUN6I_ISP_MCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_MCH_SCALE_CFG_REG 0x1e4
+#define SUN6I_ISP_MCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_MCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16))
+#define SUN6I_ISP_MCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28))
+
+#define SUN6I_ISP_SCH_SIZE_CFG_REG 0x1e8
+#define SUN6I_ISP_SCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0))
+#define SUN6I_ISP_SCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16))
+
+#define SUN6I_ISP_SCH_SCALE_CFG_REG 0x1ec
+#define SUN6I_ISP_SCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0))
+#define SUN6I_ISP_SCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16))
+#define SUN6I_ISP_SCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28))
+
+#define SUN6I_ISP_MCH_CFG_REG 0x1f0
+#define SUN6I_ISP_MCH_CFG_EN BIT(0)
+#define SUN6I_ISP_MCH_CFG_SCALE_EN BIT(1)
+#define SUN6I_ISP_MCH_CFG_OUTPUT_FMT(v) (((v) << 2) & GENMASK(4, 2))
+#define SUN6I_ISP_MCH_CFG_MIRROR_EN BIT(5)
+#define SUN6I_ISP_MCH_CFG_FLIP_EN BIT(6)
+#define SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(v) (((v) << 8) & GENMASK(18, 8))
+#define SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(v) (((v) << 20) & GENMASK(30, 20))
+
+#define SUN6I_ISP_OUTPUT_FMT_YUV420SP 0
+#define SUN6I_ISP_OUTPUT_FMT_YUV422SP 1
+#define SUN6I_ISP_OUTPUT_FMT_YVU420SP 2
+#define SUN6I_ISP_OUTPUT_FMT_YVU422SP 3
+#define SUN6I_ISP_OUTPUT_FMT_YUV420P 4
+#define SUN6I_ISP_OUTPUT_FMT_YUV422P 5
+#define SUN6I_ISP_OUTPUT_FMT_YVU420P 6
+#define SUN6I_ISP_OUTPUT_FMT_YVU422P 7
+
+#define SUN6I_ISP_SCH_CFG_REG 0x1f4
+
+#define SUN6I_ISP_MCH_Y_ADDR0_REG 0x1f8
+#define SUN6I_ISP_MCH_U_ADDR0_REG 0x1fc
+#define SUN6I_ISP_MCH_V_ADDR0_REG 0x200
+#define SUN6I_ISP_MCH_Y_ADDR1_REG 0x204
+#define SUN6I_ISP_MCH_U_ADDR1_REG 0x208
+#define SUN6I_ISP_MCH_V_ADDR1_REG 0x20c
+#define SUN6I_ISP_SCH_Y_ADDR0_REG 0x210
+#define SUN6I_ISP_SCH_U_ADDR0_REG 0x214
+#define SUN6I_ISP_SCH_V_ADDR0_REG 0x218
+#define SUN6I_ISP_SCH_Y_ADDR1_REG 0x21c
+#define SUN6I_ISP_SCH_U_ADDR1_REG 0x220
+#define SUN6I_ISP_SCH_V_ADDR1_REG 0x224
+
+#endif
diff --git a/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
new file mode 100644
index 000000000000..fd2a0820aa98
--- /dev/null
+++ b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR MIT) */
+/*
+ * Allwinner A31 ISP Configuration
+ */
+
+#ifndef _UAPI_SUN6I_ISP_CONFIG_H
+#define _UAPI_SUN6I_ISP_CONFIG_H
+
+#include <linux/types.h>
+
+#define V4L2_META_FMT_SUN6I_ISP_PARAMS v4l2_fourcc('S', '6', 'I', 'P') /* Allwinner A31 ISP Parameters */
+
+#define SUN6I_ISP_MODULE_BAYER (1U << 0)
+#define SUN6I_ISP_MODULE_BDNF (1U << 1)
+
+struct sun6i_isp_params_config_bayer {
+ __u16 offset_r;
+ __u16 offset_gr;
+ __u16 offset_gb;
+ __u16 offset_b;
+
+ __u16 gain_r;
+ __u16 gain_gr;
+ __u16 gain_gb;
+ __u16 gain_b;
+};
+
+struct sun6i_isp_params_config_bdnf {
+ __u8 in_dis_min; // 8
+ __u8 in_dis_max; // 10
+
+ __u8 coefficients_g[7];
+ __u8 coefficients_rb[5];
+};
+
+struct sun6i_isp_params_config {
+ __u32 modules_used;
+
+ struct sun6i_isp_params_config_bayer bayer;
+ struct sun6i_isp_params_config_bdnf bdnf;
+};
+
+#endif /* _UAPI_SUN6I_ISP_CONFIG_H */
--
2.34.1
Continue moving things over to capture in tidy helpers.
Also take the occasion to remove the config struct, which is
unwelcome redundancy and use the capture helpers instead.
The code is only adapted to reflect the removal of the config
structure. No functional change intended.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 363 -----------------
.../platform/sunxi/sun6i-csi/sun6i_csi.h | 25 --
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 364 +++++++++++++++++-
3 files changed, 356 insertions(+), 396 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 7e3727c86e7a..d934b4b466ef 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -148,369 +148,6 @@ bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev,
return false;
}
-static enum csi_input_fmt get_csi_input_format(struct sun6i_csi_device *csi_dev,
- u32 mbus_code, u32 pixformat)
-{
- /* non-YUV */
- if ((mbus_code & 0xF000) != 0x2000)
- return CSI_INPUT_FORMAT_RAW;
-
- switch (pixformat) {
- case V4L2_PIX_FMT_YUYV:
- case V4L2_PIX_FMT_YVYU:
- case V4L2_PIX_FMT_UYVY:
- case V4L2_PIX_FMT_VYUY:
- return CSI_INPUT_FORMAT_RAW;
- default:
- break;
- }
-
- /* not support YUV420 input format yet */
- dev_dbg(csi_dev->dev, "Select YUV422 as default input format of CSI.\n");
- return CSI_INPUT_FORMAT_YUV422;
-}
-
-static enum csi_output_fmt
-get_csi_output_format(struct sun6i_csi_device *csi_dev, u32 pixformat,
- u32 field)
-{
- bool buf_interlaced = false;
-
- if (field == V4L2_FIELD_INTERLACED
- || field == V4L2_FIELD_INTERLACED_TB
- || field == V4L2_FIELD_INTERLACED_BT)
- buf_interlaced = true;
-
- switch (pixformat) {
- case V4L2_PIX_FMT_SBGGR8:
- case V4L2_PIX_FMT_SGBRG8:
- case V4L2_PIX_FMT_SGRBG8:
- case V4L2_PIX_FMT_SRGGB8:
- return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
- case V4L2_PIX_FMT_SBGGR10:
- case V4L2_PIX_FMT_SGBRG10:
- case V4L2_PIX_FMT_SGRBG10:
- case V4L2_PIX_FMT_SRGGB10:
- return buf_interlaced ? CSI_FRAME_RAW_10 : CSI_FIELD_RAW_10;
- case V4L2_PIX_FMT_SBGGR12:
- case V4L2_PIX_FMT_SGBRG12:
- case V4L2_PIX_FMT_SGRBG12:
- case V4L2_PIX_FMT_SRGGB12:
- return buf_interlaced ? CSI_FRAME_RAW_12 : CSI_FIELD_RAW_12;
-
- case V4L2_PIX_FMT_YUYV:
- case V4L2_PIX_FMT_YVYU:
- case V4L2_PIX_FMT_UYVY:
- case V4L2_PIX_FMT_VYUY:
- return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
-
- case V4L2_PIX_FMT_NV12_16L16:
- return buf_interlaced ? CSI_FRAME_MB_YUV420 :
- CSI_FIELD_MB_YUV420;
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV21:
- return buf_interlaced ? CSI_FRAME_UV_CB_YUV420 :
- CSI_FIELD_UV_CB_YUV420;
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YVU420:
- return buf_interlaced ? CSI_FRAME_PLANAR_YUV420 :
- CSI_FIELD_PLANAR_YUV420;
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_NV61:
- return buf_interlaced ? CSI_FRAME_UV_CB_YUV422 :
- CSI_FIELD_UV_CB_YUV422;
- case V4L2_PIX_FMT_YUV422P:
- return buf_interlaced ? CSI_FRAME_PLANAR_YUV422 :
- CSI_FIELD_PLANAR_YUV422;
-
- case V4L2_PIX_FMT_RGB565:
- case V4L2_PIX_FMT_RGB565X:
- return buf_interlaced ? CSI_FRAME_RGB565 : CSI_FIELD_RGB565;
-
- case V4L2_PIX_FMT_JPEG:
- return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
-
- default:
- dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x\n", pixformat);
- break;
- }
-
- return CSI_FIELD_RAW_8;
-}
-
-static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev,
- u32 mbus_code, u32 pixformat)
-{
- /* Input sequence does not apply to non-YUV formats */
- if ((mbus_code & 0xF000) != 0x2000)
- return 0;
-
- switch (pixformat) {
- case V4L2_PIX_FMT_NV12_16L16:
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YUV422P:
- switch (mbus_code) {
- case MEDIA_BUS_FMT_UYVY8_2X8:
- case MEDIA_BUS_FMT_UYVY8_1X16:
- return CSI_INPUT_SEQ_UYVY;
- case MEDIA_BUS_FMT_VYUY8_2X8:
- case MEDIA_BUS_FMT_VYUY8_1X16:
- return CSI_INPUT_SEQ_VYUY;
- case MEDIA_BUS_FMT_YUYV8_2X8:
- case MEDIA_BUS_FMT_YUYV8_1X16:
- return CSI_INPUT_SEQ_YUYV;
- case MEDIA_BUS_FMT_YVYU8_1X16:
- case MEDIA_BUS_FMT_YVYU8_2X8:
- return CSI_INPUT_SEQ_YVYU;
- default:
- dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n",
- mbus_code);
- break;
- }
- break;
- case V4L2_PIX_FMT_NV21:
- case V4L2_PIX_FMT_NV61:
- case V4L2_PIX_FMT_YVU420:
- switch (mbus_code) {
- case MEDIA_BUS_FMT_UYVY8_2X8:
- case MEDIA_BUS_FMT_UYVY8_1X16:
- return CSI_INPUT_SEQ_VYUY;
- case MEDIA_BUS_FMT_VYUY8_2X8:
- case MEDIA_BUS_FMT_VYUY8_1X16:
- return CSI_INPUT_SEQ_UYVY;
- case MEDIA_BUS_FMT_YUYV8_2X8:
- case MEDIA_BUS_FMT_YUYV8_1X16:
- return CSI_INPUT_SEQ_YVYU;
- case MEDIA_BUS_FMT_YVYU8_1X16:
- case MEDIA_BUS_FMT_YVYU8_2X8:
- return CSI_INPUT_SEQ_YUYV;
- default:
- dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n",
- mbus_code);
- break;
- }
- break;
-
- case V4L2_PIX_FMT_YUYV:
- return CSI_INPUT_SEQ_YUYV;
-
- default:
- dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x, defaulting to YUYV\n",
- pixformat);
- break;
- }
-
- return CSI_INPUT_SEQ_YUYV;
-}
-
-static void sun6i_csi_setup_bus(struct sun6i_csi_device *csi_dev)
-{
- struct v4l2_fwnode_endpoint *endpoint =
- &csi_dev->bridge.source->endpoint;
- struct sun6i_csi_config *config = &csi_dev->config;
- unsigned char bus_width;
- u32 flags;
- u32 cfg = 0;
- bool input_interlaced = false;
-
- if (config->field == V4L2_FIELD_INTERLACED
- || config->field == V4L2_FIELD_INTERLACED_TB
- || config->field == V4L2_FIELD_INTERLACED_BT)
- input_interlaced = true;
-
- bus_width = endpoint->bus.parallel.bus_width;
-
- if (input_interlaced)
- cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
- SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
- SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
- else
- cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
-
- switch (endpoint->bus_type) {
- case V4L2_MBUS_PARALLEL:
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
-
- flags = endpoint->bus.parallel.flags;
-
- if (bus_width == 16)
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
- else
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
-
- if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
- else
- cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
-
- if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
- cfg |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
- else
- cfg |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
-
- if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
- cfg |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
- else
- cfg |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
-
- if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
- else
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
- break;
- case V4L2_MBUS_BT656:
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
-
- flags = endpoint->bus.parallel.flags;
-
- if (bus_width == 16)
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
- else
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
-
- if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
- else
- cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
-
- if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
- else
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
- break;
- default:
- dev_warn(csi_dev->dev, "Unsupported bus type: %d\n",
- endpoint->bus_type);
- break;
- }
-
- switch (bus_width) {
- case 8:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
- break;
- case 10:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
- break;
- case 12:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
- break;
- case 16: /* No need to configure DATA_WIDTH for 16bit */
- break;
- default:
- dev_warn(csi_dev->dev, "Unsupported bus width: %u\n", bus_width);
- break;
- }
-
- regmap_write(csi_dev->regmap, SUN6I_CSI_IF_CFG_REG, cfg);
-}
-
-static void sun6i_csi_set_format(struct sun6i_csi_device *csi_dev)
-{
- struct sun6i_csi_config *config = &csi_dev->config;
- u32 cfg = 0;
- u32 val;
-
- val = get_csi_input_format(csi_dev, config->code,
- config->pixelformat);
- cfg |= SUN6I_CSI_CH_CFG_INPUT_FMT(val);
-
- val = get_csi_output_format(csi_dev, config->pixelformat,
- config->field);
- cfg |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(val);
-
- val = get_csi_input_seq(csi_dev, config->code,
- config->pixelformat);
- cfg |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(val);
-
- if (config->field == V4L2_FIELD_TOP)
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
- else if (config->field == V4L2_FIELD_BOTTOM)
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
- else
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
-
- regmap_write(csi_dev->regmap, SUN6I_CSI_CH_CFG_REG, cfg);
-}
-
-static void sun6i_csi_set_window(struct sun6i_csi_device *csi_dev)
-{
- struct sun6i_csi_config *config = &csi_dev->config;
- u32 bytesperline_y;
- u32 bytesperline_c;
- u32 width = config->width;
- u32 height = config->height;
- u32 hor_len = width;
-
- switch (config->pixelformat) {
- case V4L2_PIX_FMT_YUYV:
- case V4L2_PIX_FMT_YVYU:
- case V4L2_PIX_FMT_UYVY:
- case V4L2_PIX_FMT_VYUY:
- dev_dbg(csi_dev->dev,
- "Horizontal length should be 2 times of width for packed YUV formats!\n");
- hor_len = width * 2;
- break;
- default:
- break;
- }
-
- regmap_write(csi_dev->regmap, SUN6I_CSI_CH_HSIZE_REG,
- SUN6I_CSI_CH_HSIZE_LEN(hor_len) |
- SUN6I_CSI_CH_HSIZE_START(0));
- regmap_write(csi_dev->regmap, SUN6I_CSI_CH_VSIZE_REG,
- SUN6I_CSI_CH_VSIZE_LEN(height) |
- SUN6I_CSI_CH_VSIZE_START(0));
-
- switch (config->pixelformat) {
- case V4L2_PIX_FMT_NV12_16L16:
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV21:
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_NV61:
- bytesperline_y = width;
- bytesperline_c = width;
- break;
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YVU420:
- bytesperline_y = width;
- bytesperline_c = width / 2;
- break;
- case V4L2_PIX_FMT_YUV422P:
- bytesperline_y = width;
- bytesperline_c = width / 2;
- break;
- default: /* raw */
- dev_dbg(csi_dev->dev,
- "Calculating pixelformat(0x%x)'s bytesperline as a packed format\n",
- config->pixelformat);
- bytesperline_y = (sun6i_csi_get_bpp(config->pixelformat) *
- config->width) / 8;
- bytesperline_c = 0;
- break;
- }
-
- regmap_write(csi_dev->regmap, SUN6I_CSI_CH_BUF_LEN_REG,
- SUN6I_CSI_CH_BUF_LEN_CHROMA_LINE(bytesperline_c) |
- SUN6I_CSI_CH_BUF_LEN_LUMA_LINE(bytesperline_y));
-}
-
-int sun6i_csi_update_config(struct sun6i_csi_device *csi_dev,
- struct sun6i_csi_config *config)
-{
- if (!config)
- return -EINVAL;
-
- memcpy(&csi_dev->config, config, sizeof(csi_dev->config));
-
- sun6i_csi_setup_bus(csi_dev);
- sun6i_csi_set_format(csi_dev);
- sun6i_csi_set_window(csi_dev);
-
- return 0;
-}
-
/* Media */
static const struct media_device_ops sun6i_csi_media_ops = {
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
index 0271587e8520..c37bbab93cd5 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
@@ -28,22 +28,6 @@ struct sun6i_csi_buffer {
struct list_head list;
};
-/**
- * struct sun6i_csi_config - configs for sun6i csi
- * @pixelformat: v4l2 pixel format (V4L2_PIX_FMT_*)
- * @code: media bus format code (MEDIA_BUS_FMT_*)
- * @field: used interlacing type (enum v4l2_field)
- * @width: frame width
- * @height: frame height
- */
-struct sun6i_csi_config {
- u32 pixelformat;
- u32 code;
- u32 field;
- u32 width;
- u32 height;
-};
-
struct sun6i_csi_v4l2 {
struct v4l2_device v4l2_dev;
struct v4l2_ctrl_handler ctrl_handler;
@@ -53,7 +37,6 @@ struct sun6i_csi_v4l2 {
struct sun6i_csi_device {
struct device *dev;
- struct sun6i_csi_config config;
struct sun6i_csi_v4l2 v4l2;
struct sun6i_csi_bridge bridge;
struct sun6i_csi_capture capture;
@@ -74,14 +57,6 @@ struct sun6i_csi_device {
bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev,
u32 pixformat, u32 mbus_code);
-/**
- * sun6i_csi_update_config() - update the csi register settings
- * @csi: pointer to the csi
- * @config: see struct sun6i_csi_config
- */
-int sun6i_csi_update_config(struct sun6i_csi_device *csi_dev,
- struct sun6i_csi_config *config);
-
/* get bpp form v4l2 pixformat */
static inline int sun6i_csi_get_bpp(unsigned int pixformat)
{
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
index 05eb9aae2975..23b13b26b580 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
@@ -198,18 +198,366 @@ sun6i_csi_capture_buffer_configure(struct sun6i_csi_device *csi_dev,
}
}
-static void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev)
+static enum csi_input_fmt get_csi_input_format(struct sun6i_csi_device *csi_dev,
+ u32 mbus_code, u32 pixformat)
+{
+ /* non-YUV */
+ if ((mbus_code & 0xF000) != 0x2000)
+ return CSI_INPUT_FORMAT_RAW;
+
+ switch (pixformat) {
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_YVYU:
+ case V4L2_PIX_FMT_UYVY:
+ case V4L2_PIX_FMT_VYUY:
+ return CSI_INPUT_FORMAT_RAW;
+ default:
+ break;
+ }
+
+ /* not support YUV420 input format yet */
+ dev_dbg(csi_dev->dev, "Select YUV422 as default input format of CSI.\n");
+ return CSI_INPUT_FORMAT_YUV422;
+}
+
+static enum csi_output_fmt
+get_csi_output_format(struct sun6i_csi_device *csi_dev, u32 pixformat,
+ u32 field)
+{
+ bool buf_interlaced = false;
+
+ if (field == V4L2_FIELD_INTERLACED
+ || field == V4L2_FIELD_INTERLACED_TB
+ || field == V4L2_FIELD_INTERLACED_BT)
+ buf_interlaced = true;
+
+ switch (pixformat) {
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SRGGB10:
+ return buf_interlaced ? CSI_FRAME_RAW_10 : CSI_FIELD_RAW_10;
+ case V4L2_PIX_FMT_SBGGR12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SRGGB12:
+ return buf_interlaced ? CSI_FRAME_RAW_12 : CSI_FIELD_RAW_12;
+
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_YVYU:
+ case V4L2_PIX_FMT_UYVY:
+ case V4L2_PIX_FMT_VYUY:
+ return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
+
+ case V4L2_PIX_FMT_NV12_16L16:
+ return buf_interlaced ? CSI_FRAME_MB_YUV420 :
+ CSI_FIELD_MB_YUV420;
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV21:
+ return buf_interlaced ? CSI_FRAME_UV_CB_YUV420 :
+ CSI_FIELD_UV_CB_YUV420;
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YVU420:
+ return buf_interlaced ? CSI_FRAME_PLANAR_YUV420 :
+ CSI_FIELD_PLANAR_YUV420;
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_NV61:
+ return buf_interlaced ? CSI_FRAME_UV_CB_YUV422 :
+ CSI_FIELD_UV_CB_YUV422;
+ case V4L2_PIX_FMT_YUV422P:
+ return buf_interlaced ? CSI_FRAME_PLANAR_YUV422 :
+ CSI_FIELD_PLANAR_YUV422;
+
+ case V4L2_PIX_FMT_RGB565:
+ case V4L2_PIX_FMT_RGB565X:
+ return buf_interlaced ? CSI_FRAME_RGB565 : CSI_FIELD_RGB565;
+
+ case V4L2_PIX_FMT_JPEG:
+ return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8;
+
+ default:
+ dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x\n", pixformat);
+ break;
+ }
+
+ return CSI_FIELD_RAW_8;
+}
+
+static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev,
+ u32 mbus_code, u32 pixformat)
+{
+ /* Input sequence does not apply to non-YUV formats */
+ if ((mbus_code & 0xF000) != 0x2000)
+ return 0;
+
+ switch (pixformat) {
+ case V4L2_PIX_FMT_NV12_16L16:
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YUV422P:
+ switch (mbus_code) {
+ case MEDIA_BUS_FMT_UYVY8_2X8:
+ case MEDIA_BUS_FMT_UYVY8_1X16:
+ return CSI_INPUT_SEQ_UYVY;
+ case MEDIA_BUS_FMT_VYUY8_2X8:
+ case MEDIA_BUS_FMT_VYUY8_1X16:
+ return CSI_INPUT_SEQ_VYUY;
+ case MEDIA_BUS_FMT_YUYV8_2X8:
+ case MEDIA_BUS_FMT_YUYV8_1X16:
+ return CSI_INPUT_SEQ_YUYV;
+ case MEDIA_BUS_FMT_YVYU8_1X16:
+ case MEDIA_BUS_FMT_YVYU8_2X8:
+ return CSI_INPUT_SEQ_YVYU;
+ default:
+ dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n",
+ mbus_code);
+ break;
+ }
+ break;
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV61:
+ case V4L2_PIX_FMT_YVU420:
+ switch (mbus_code) {
+ case MEDIA_BUS_FMT_UYVY8_2X8:
+ case MEDIA_BUS_FMT_UYVY8_1X16:
+ return CSI_INPUT_SEQ_VYUY;
+ case MEDIA_BUS_FMT_VYUY8_2X8:
+ case MEDIA_BUS_FMT_VYUY8_1X16:
+ return CSI_INPUT_SEQ_UYVY;
+ case MEDIA_BUS_FMT_YUYV8_2X8:
+ case MEDIA_BUS_FMT_YUYV8_1X16:
+ return CSI_INPUT_SEQ_YVYU;
+ case MEDIA_BUS_FMT_YVYU8_1X16:
+ case MEDIA_BUS_FMT_YVYU8_2X8:
+ return CSI_INPUT_SEQ_YUYV;
+ default:
+ dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n",
+ mbus_code);
+ break;
+ }
+ break;
+
+ case V4L2_PIX_FMT_YUYV:
+ return CSI_INPUT_SEQ_YUYV;
+
+ default:
+ dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x, defaulting to YUYV\n",
+ pixformat);
+ break;
+ }
+
+ return CSI_INPUT_SEQ_YUYV;
+}
+
+static void
+sun6i_csi_capture_configure_interface(struct sun6i_csi_device *csi_dev)
+{
+ struct v4l2_fwnode_endpoint *endpoint =
+ &csi_dev->bridge.source->endpoint;
+ u32 pixelformat, field;
+ unsigned char bus_width;
+ u32 flags;
+ u32 cfg = 0;
+ bool input_interlaced = false;
+
+ sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
+
+ if (field == V4L2_FIELD_INTERLACED ||
+ field == V4L2_FIELD_INTERLACED_TB ||
+ field == V4L2_FIELD_INTERLACED_BT)
+ input_interlaced = true;
+
+ bus_width = endpoint->bus.parallel.bus_width;
+
+ if (input_interlaced)
+ cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
+ SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
+ SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
+
+ switch (endpoint->bus_type) {
+ case V4L2_MBUS_PARALLEL:
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
+
+ flags = endpoint->bus.parallel.flags;
+
+ if (bus_width == 16)
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
+
+ if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
+ cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+
+ if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
+ cfg |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
+
+ if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
+ cfg |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
+
+ if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
+ cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ break;
+ case V4L2_MBUS_BT656:
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
+
+ flags = endpoint->bus.parallel.flags;
+
+ if (bus_width == 16)
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
+
+ if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
+ cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+
+ if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
+ cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ else
+ cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ break;
+ default:
+ dev_warn(csi_dev->dev, "Unsupported bus type: %d\n",
+ endpoint->bus_type);
+ break;
+ }
+
+ switch (bus_width) {
+ case 8:
+ cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
+ break;
+ case 10:
+ cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
+ break;
+ case 12:
+ cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
+ break;
+ case 16: /* No need to configure DATA_WIDTH for 16bit */
+ break;
+ default:
+ dev_warn(csi_dev->dev, "Unsupported bus width: %u\n", bus_width);
+ break;
+ }
+
+ regmap_write(csi_dev->regmap, SUN6I_CSI_IF_CFG_REG, cfg);
+}
+
+static void sun6i_csi_capture_configure_format(struct sun6i_csi_device *csi_dev)
{
struct sun6i_csi_capture *capture = &csi_dev->capture;
- struct sun6i_csi_config config = { 0 };
+ u32 pixelformat, field;
+ u32 cfg = 0;
+ u32 val;
+
+ sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
+
+ val = get_csi_input_format(csi_dev, capture->mbus_code, pixelformat);
+ cfg |= SUN6I_CSI_CH_CFG_INPUT_FMT(val);
+
+ val = get_csi_output_format(csi_dev, pixelformat, field);
+ cfg |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(val);
+
+ val = get_csi_input_seq(csi_dev, capture->mbus_code, pixelformat);
+ cfg |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(val);
+
+ if (field == V4L2_FIELD_TOP)
+ cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
+ else if (field == V4L2_FIELD_BOTTOM)
+ cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
+ else
+ cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
+
+ regmap_write(csi_dev->regmap, SUN6I_CSI_CH_CFG_REG, cfg);
+}
- config.pixelformat = capture->format.fmt.pix.pixelformat;
- config.code = capture->mbus_code;
- config.field = capture->format.fmt.pix.field;
- config.width = capture->format.fmt.pix.width;
- config.height = capture->format.fmt.pix.height;
+static void sun6i_csi_capture_configure_window(struct sun6i_csi_device *csi_dev)
+{
+ u32 pixelformat, field;
+ u32 width, height;
+ u32 bytesperline_y;
+ u32 bytesperline_c;
+ u32 hor_len;
+
+ sun6i_csi_capture_dimensions(csi_dev, &width, &height);
+ sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
+
+ hor_len = width;
+
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_YVYU:
+ case V4L2_PIX_FMT_UYVY:
+ case V4L2_PIX_FMT_VYUY:
+ dev_dbg(csi_dev->dev,
+ "Horizontal length should be 2 times of width for packed YUV formats!\n");
+ hor_len = width * 2;
+ break;
+ default:
+ break;
+ }
+
+ regmap_write(csi_dev->regmap, SUN6I_CSI_CH_HSIZE_REG,
+ SUN6I_CSI_CH_HSIZE_LEN(hor_len) |
+ SUN6I_CSI_CH_HSIZE_START(0));
+ regmap_write(csi_dev->regmap, SUN6I_CSI_CH_VSIZE_REG,
+ SUN6I_CSI_CH_VSIZE_LEN(height) |
+ SUN6I_CSI_CH_VSIZE_START(0));
+
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_NV12_16L16:
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_NV61:
+ bytesperline_y = width;
+ bytesperline_c = width;
+ break;
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YVU420:
+ bytesperline_y = width;
+ bytesperline_c = width / 2;
+ break;
+ case V4L2_PIX_FMT_YUV422P:
+ bytesperline_y = width;
+ bytesperline_c = width / 2;
+ break;
+ default: /* raw */
+ dev_dbg(csi_dev->dev,
+ "Calculating pixelformat(0x%x)'s bytesperline as a packed format\n",
+ pixelformat);
+ bytesperline_y = (sun6i_csi_get_bpp(pixelformat) *
+ width) / 8;
+ bytesperline_c = 0;
+ break;
+ }
+
+ regmap_write(csi_dev->regmap, SUN6I_CSI_CH_BUF_LEN_REG,
+ SUN6I_CSI_CH_BUF_LEN_CHROMA_LINE(bytesperline_c) |
+ SUN6I_CSI_CH_BUF_LEN_LUMA_LINE(bytesperline_y));
+}
- sun6i_csi_update_config(csi_dev, &config);
+static void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev)
+{
+ sun6i_csi_capture_configure_interface(csi_dev);
+ sun6i_csi_capture_configure_format(csi_dev);
+ sun6i_csi_capture_configure_window(csi_dev);
}
/* State */
--
2.34.1
Some misc code cleanups and preparation for upcoming changes.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 105 ++++++++----------
1 file changed, 46 insertions(+), 59 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
index 12c02408d18e..726416d98c46 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
@@ -353,133 +353,120 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev,
static void
sun6i_csi_capture_configure_interface(struct sun6i_csi_device *csi_dev)
{
+ struct device *dev = csi_dev->dev;
+ struct regmap *regmap = csi_dev->regmap;
struct v4l2_fwnode_endpoint *endpoint =
&csi_dev->bridge.source->endpoint;
+ unsigned char bus_width = endpoint->bus.parallel.bus_width;
+ unsigned int flags = endpoint->bus.parallel.flags;
u32 pixelformat, field;
- unsigned char bus_width;
- u32 flags;
- u32 cfg = 0;
- bool input_interlaced = false;
+ u32 value = SUN6I_CSI_IF_CFG_IF_CSI;
sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
if (field == V4L2_FIELD_INTERLACED ||
field == V4L2_FIELD_INTERLACED_TB ||
field == V4L2_FIELD_INTERLACED_BT)
- input_interlaced = true;
-
- bus_width = endpoint->bus.parallel.bus_width;
-
- if (input_interlaced)
- cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
- SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
- SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
+ value |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED |
+ SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) |
+ SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC;
else
- cfg |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
+ value |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE;
switch (endpoint->bus_type) {
case V4L2_MBUS_PARALLEL:
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
-
- flags = endpoint->bus.parallel.flags;
-
if (bus_width == 16)
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED;
else
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW;
if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
else
- cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+ value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
- cfg |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
+ value |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE;
else
- cfg |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
+ value |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE;
if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
- cfg |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
+ value |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE;
else
- cfg |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
+ value |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE;
if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING)
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
else
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
break;
case V4L2_MBUS_BT656:
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI;
-
- flags = endpoint->bus.parallel.flags;
-
if (bus_width == 16)
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120;
else
- cfg |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
+ value |= SUN6I_CSI_IF_CFG_IF_CSI_BT656;
if (flags & V4L2_MBUS_FIELD_EVEN_LOW)
- cfg |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
+ value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE;
else
- cfg |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
+ value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE;
if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING;
else
- cfg |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
+ value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING;
break;
default:
- dev_warn(csi_dev->dev, "Unsupported bus type: %d\n",
- endpoint->bus_type);
+ dev_warn(dev, "unsupported bus type: %d\n", endpoint->bus_type);
break;
}
switch (bus_width) {
case 8:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
+ /* 16-bit YUV formats use a doubled width in 8-bit mode. */
+ case 16:
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8;
break;
case 10:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10;
break;
case 12:
- cfg |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
- break;
- case 16: /* No need to configure DATA_WIDTH for 16bit */
+ value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12;
break;
default:
- dev_warn(csi_dev->dev, "Unsupported bus width: %u\n", bus_width);
+ dev_warn(dev, "unsupported bus width: %u\n", bus_width);
break;
}
- regmap_write(csi_dev->regmap, SUN6I_CSI_IF_CFG_REG, cfg);
+ regmap_write(regmap, SUN6I_CSI_IF_CFG_REG, value);
}
static void sun6i_csi_capture_configure_format(struct sun6i_csi_device *csi_dev)
{
+ struct regmap *regmap = csi_dev->regmap;
u32 mbus_code, pixelformat, field;
- u32 cfg = 0;
- u32 val;
+ u8 input_format, input_yuv_seq, output_format;
+ u32 value = 0;
sun6i_csi_capture_format(csi_dev, &pixelformat, &field);
sun6i_csi_bridge_format(csi_dev, &mbus_code, NULL);
- val = get_csi_input_format(csi_dev, mbus_code, pixelformat);
- cfg |= SUN6I_CSI_CH_CFG_INPUT_FMT(val);
-
- val = get_csi_output_format(csi_dev, pixelformat, field);
- cfg |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(val);
+ input_format = get_csi_input_format(csi_dev, mbus_code, pixelformat);
+ input_yuv_seq = get_csi_input_seq(csi_dev, mbus_code, pixelformat);
+ output_format = get_csi_output_format(csi_dev, pixelformat, field);
- val = get_csi_input_seq(csi_dev, mbus_code, pixelformat);
- cfg |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(val);
+ value |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(output_format);
+ value |= SUN6I_CSI_CH_CFG_INPUT_FMT(input_format);
+ value |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(input_yuv_seq);
if (field == V4L2_FIELD_TOP)
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0;
else if (field == V4L2_FIELD_BOTTOM)
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1;
else
- cfg |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
+ value |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER;
- regmap_write(csi_dev->regmap, SUN6I_CSI_CH_CFG_REG, cfg);
+ regmap_write(regmap, SUN6I_CSI_CH_CFG_REG, value);
}
static void sun6i_csi_capture_configure_window(struct sun6i_csi_device *csi_dev)
--
2.34.1
The V3s (and related platforms) come with an instance of the A31 ISP.
Even though it is very close to the A31 ISP, it is not exactly
register-compatible and a dedicated compatible only is used as a
result.
Just like most other blocks of the camera pipeline, the ISP uses
the common CSI bus, module and ram clock as well as reset.
A port connection to the ISP is added to CSI0 for convenience since
CSI0 serves for MIPI CSI-2 interface support, which is likely to
receive raw data that will need to be processed by the ISP to produce
a final image.
While the interconnects property is used to inherit the proper dma
ranges, the associated index for the cell is set to 0 since no
particular meaning is attached to it. This might need to be changed
later on (when identifying a proper mbus channel becomes relevant,
e.g. for things like QoS).
Signed-off-by: Paul Kocialkowski <[email protected]>
---
arch/arm/boot/dts/sun8i-v3s.dtsi | 35 ++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-v3s.dtsi b/arch/arm/boot/dts/sun8i-v3s.dtsi
index f5f7dfec49f9..c2e65679b9ed 100644
--- a/arch/arm/boot/dts/sun8i-v3s.dtsi
+++ b/arch/arm/boot/dts/sun8i-v3s.dtsi
@@ -638,6 +638,14 @@ csi0_in_mipi_csi2: endpoint {
remote-endpoint = <&mipi_csi2_out_csi0>;
};
};
+
+ port@2 {
+ reg = <2>;
+
+ csi0_out_isp: endpoint {
+ remote-endpoint = <&isp_in_csi0>;
+ };
+ };
};
};
@@ -696,5 +704,32 @@ csi1: camera@1cb4000 {
resets = <&ccu RST_BUS_CSI>;
status = "disabled";
};
+
+ isp: isp@1cb8000 {
+ compatible = "allwinner,sun8i-v3s-isp";
+ reg = <0x01cb8000 0x1000>;
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_CSI>,
+ <&ccu CLK_CSI1_SCLK>,
+ <&ccu CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = <&ccu RST_BUS_CSI>;
+ interconnects = <&mbus 0>;
+ interconnect-names = "dma-mem";
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ isp_in_csi0: endpoint {
+ remote-endpoint = <&csi0_out_isp>;
+ };
+ };
+ };
+ };
};
};
--
2.34.1
Introduce a bridge v4l2 subdev to prepare for separation between the
processing part (bridge) and the dma engine, which is required to
properly support ths isp workflow later on.
Currently the bridge just manages fwnode mapping to media pads,
using an async notifier (which was previously in the main code).
The s_stream video op just forwards to the connected v4l2 subdev
(sensor or MIPI CSI-2 bridge).
The video capture device is now registered after the bridge and
attaches to it with a media link.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../media/platform/sunxi/sun6i-csi/Makefile | 2 +-
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 156 +-----
.../platform/sunxi/sun6i-csi/sun6i_csi.h | 12 +-
.../sunxi/sun6i-csi/sun6i_csi_bridge.c | 473 ++++++++++++++++++
.../sunxi/sun6i-csi/sun6i_csi_bridge.h | 44 ++
.../platform/sunxi/sun6i-csi/sun6i_video.c | 19 +
6 files changed, 571 insertions(+), 135 deletions(-)
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile
index e7e315347804..7a699580a641 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/Makefile
+++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-sun6i-csi-y += sun6i_video.o sun6i_csi.o
+sun6i-csi-y += sun6i_video.o sun6i_csi.o sun6i_csi_bridge.o
obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index c8fe31cc38b5..a1847ae3e88e 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -34,16 +34,17 @@
bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev,
u32 pixformat, u32 mbus_code)
{
- struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
+ struct v4l2_fwnode_endpoint *endpoint =
+ &csi_dev->bridge.source->endpoint;
/*
* Some video receivers have the ability to be compatible with
* 8bit and 16bit bus width.
* Identify the media bus format from device tree.
*/
- if ((v4l2->v4l2_ep.bus_type == V4L2_MBUS_PARALLEL
- || v4l2->v4l2_ep.bus_type == V4L2_MBUS_BT656)
- && v4l2->v4l2_ep.bus.parallel.bus_width == 16) {
+ if ((endpoint->bus_type == V4L2_MBUS_PARALLEL
+ || endpoint->bus_type == V4L2_MBUS_BT656)
+ && endpoint->bus.parallel.bus_width == 16) {
switch (pixformat) {
case V4L2_PIX_FMT_NV12_16L16:
case V4L2_PIX_FMT_NV12:
@@ -328,7 +329,8 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev,
static void sun6i_csi_setup_bus(struct sun6i_csi_device *csi_dev)
{
- struct v4l2_fwnode_endpoint *endpoint = &csi_dev->v4l2.v4l2_ep;
+ struct v4l2_fwnode_endpoint *endpoint =
+ &csi_dev->bridge.source->endpoint;
struct sun6i_csi_config *config = &csi_dev->config;
unsigned char bus_width;
u32 flags;
@@ -583,95 +585,11 @@ static const struct media_device_ops sun6i_csi_media_ops = {
/* V4L2 */
-static int sun6i_csi_link_entity(struct sun6i_csi_device *csi_dev,
- struct media_entity *entity,
- struct fwnode_handle *fwnode)
+int sun6i_csi_v4l2_complete(struct sun6i_csi_device *csi_dev)
{
- struct media_entity *sink;
- struct media_pad *sink_pad;
- int src_pad_index;
- int ret;
-
- ret = media_entity_get_fwnode_pad(entity, fwnode, MEDIA_PAD_FL_SOURCE);
- if (ret < 0) {
- dev_err(csi_dev->dev,
- "%s: no source pad in external entity %s\n", __func__,
- entity->name);
- return -EINVAL;
- }
-
- src_pad_index = ret;
-
- sink = &csi_dev->video.video_dev.entity;
- sink_pad = &csi_dev->video.pad;
-
- dev_dbg(csi_dev->dev, "creating %s:%u -> %s:%u link\n",
- entity->name, src_pad_index, sink->name, sink_pad->index);
- ret = media_create_pad_link(entity, src_pad_index, sink,
- sink_pad->index,
- MEDIA_LNK_FL_ENABLED |
- MEDIA_LNK_FL_IMMUTABLE);
- if (ret < 0) {
- dev_err(csi_dev->dev, "failed to create %s:%u -> %s:%u link\n",
- entity->name, src_pad_index,
- sink->name, sink_pad->index);
- return ret;
- }
-
- return 0;
-}
-
-static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier)
-{
- struct sun6i_csi_device *csi_dev =
- container_of(notifier, struct sun6i_csi_device,
- v4l2.notifier);
- struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
- struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
- struct v4l2_subdev *sd;
- int ret;
-
- dev_dbg(csi_dev->dev, "notify complete, all subdevs registered\n");
-
- sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list);
- if (!sd)
- return -EINVAL;
-
- ret = sun6i_csi_link_entity(csi_dev, &sd->entity, sd->fwnode);
- if (ret < 0)
- return ret;
-
- ret = v4l2_device_register_subdev_nodes(v4l2_dev);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-static const struct v4l2_async_notifier_operations sun6i_csi_async_ops = {
- .complete = sun6i_subdev_notify_complete,
-};
-
-static int sun6i_csi_fwnode_parse(struct device *dev,
- struct v4l2_fwnode_endpoint *vep,
- struct v4l2_async_subdev *asd)
-{
- struct sun6i_csi_device *csi_dev = dev_get_drvdata(dev);
+ struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
- if (vep->base.port || vep->base.id) {
- dev_warn(dev, "Only support a single port with one endpoint\n");
- return -ENOTCONN;
- }
-
- switch (vep->bus_type) {
- case V4L2_MBUS_PARALLEL:
- case V4L2_MBUS_BT656:
- csi_dev->v4l2.v4l2_ep = *vep;
- return 0;
- default:
- dev_err(dev, "Unsupported media bus type\n");
- return -ENOTCONN;
- }
+ return v4l2_device_register_subdev_nodes(v4l2_dev);
}
static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
@@ -679,7 +597,6 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
struct media_device *media_dev = &v4l2->media_dev;
struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
- struct v4l2_async_notifier *notifier = &v4l2->notifier;
struct device *dev = csi_dev->dev;
int ret;
@@ -720,42 +637,8 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
goto error_v4l2_ctrl;
}
- /* Video */
-
- ret = sun6i_video_setup(csi_dev);
- if (ret)
- goto error_v4l2_device;
-
- /* V4L2 Async */
-
- v4l2_async_nf_init(notifier);
- notifier->ops = &sun6i_csi_async_ops;
-
- ret = v4l2_async_nf_parse_fwnode_endpoints(dev, notifier,
- sizeof(struct
- v4l2_async_subdev),
- sun6i_csi_fwnode_parse);
- if (ret)
- goto error_video;
-
- ret = v4l2_async_nf_register(v4l2_dev, notifier);
- if (ret) {
- dev_err(dev, "failed to register v4l2 async notifier: %d\n",
- ret);
- goto error_v4l2_async_notifier;
- }
-
return 0;
-error_v4l2_async_notifier:
- v4l2_async_nf_cleanup(notifier);
-
-error_video:
- sun6i_video_cleanup(csi_dev);
-
-error_v4l2_device:
- v4l2_device_unregister(&v4l2->v4l2_dev);
-
error_v4l2_ctrl:
v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
@@ -771,9 +654,6 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi_device *csi_dev)
struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
media_device_unregister(&v4l2->media_dev);
- v4l2_async_nf_unregister(&v4l2->notifier);
- v4l2_async_nf_cleanup(&v4l2->notifier);
- sun6i_video_cleanup(csi_dev);
v4l2_device_unregister(&v4l2->v4l2_dev);
v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
media_device_cleanup(&v4l2->media_dev);
@@ -995,8 +875,22 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)
if (ret)
goto error_resources;
+ ret = sun6i_csi_bridge_setup(csi_dev);
+ if (ret)
+ goto error_v4l2;
+
+ ret = sun6i_video_setup(csi_dev);
+ if (ret)
+ goto error_bridge;
+
return 0;
+error_bridge:
+ sun6i_csi_bridge_cleanup(csi_dev);
+
+error_v4l2:
+ sun6i_csi_v4l2_cleanup(csi_dev);
+
error_resources:
sun6i_csi_resources_cleanup(csi_dev);
@@ -1007,6 +901,8 @@ static int sun6i_csi_remove(struct platform_device *pdev)
{
struct sun6i_csi_device *csi_dev = platform_get_drvdata(pdev);
+ sun6i_video_cleanup(csi_dev);
+ sun6i_csi_bridge_cleanup(csi_dev);
sun6i_csi_v4l2_cleanup(csi_dev);
sun6i_csi_resources_cleanup(csi_dev);
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
index 4dd83e57bafa..576c7f10289e 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
@@ -13,11 +13,16 @@
#include <media/v4l2-fwnode.h>
#include <media/videobuf2-v4l2.h>
+#include "sun6i_csi_bridge.h"
#include "sun6i_video.h"
#define SUN6I_CSI_NAME "sun6i-csi"
#define SUN6I_CSI_DESCRIPTION "Allwinner A31 CSI Device"
+enum sun6i_csi_port {
+ SUN6I_CSI_PORT_PARALLEL = 0,
+};
+
struct sun6i_csi_buffer {
struct vb2_v4l2_buffer v4l2_buffer;
struct list_head list;
@@ -46,10 +51,6 @@ struct sun6i_csi_v4l2 {
struct v4l2_device v4l2_dev;
struct v4l2_ctrl_handler ctrl_handler;
struct media_device media_dev;
-
- struct v4l2_async_notifier notifier;
- /* video port settings */
- struct v4l2_fwnode_endpoint v4l2_ep;
};
struct sun6i_csi_device {
@@ -57,6 +58,7 @@ struct sun6i_csi_device {
struct sun6i_csi_config config;
struct sun6i_csi_v4l2 v4l2;
+ struct sun6i_csi_bridge bridge;
struct sun6i_video video;
struct regmap *regmap;
@@ -156,4 +158,6 @@ static inline int sun6i_csi_get_bpp(unsigned int pixformat)
return 0;
}
+int sun6i_csi_v4l2_complete(struct sun6i_csi_device *csi_dev);
+
#endif /* __SUN6I_CSI_H__ */
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
new file mode 100644
index 000000000000..74706d883359
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
@@ -0,0 +1,473 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021-2022 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#include "sun6i_csi.h"
+#include "sun6i_csi_bridge.h"
+
+/* Format */
+
+static const u32 sun6i_csi_bridge_mbus_codes[] = {
+ /* Bayer */
+ MEDIA_BUS_FMT_SBGGR8_1X8,
+ MEDIA_BUS_FMT_SGBRG8_1X8,
+ MEDIA_BUS_FMT_SGRBG8_1X8,
+ MEDIA_BUS_FMT_SRGGB8_1X8,
+ MEDIA_BUS_FMT_SBGGR10_1X10,
+ MEDIA_BUS_FMT_SGBRG10_1X10,
+ MEDIA_BUS_FMT_SGRBG10_1X10,
+ MEDIA_BUS_FMT_SRGGB10_1X10,
+ MEDIA_BUS_FMT_SBGGR12_1X12,
+ MEDIA_BUS_FMT_SGBRG12_1X12,
+ MEDIA_BUS_FMT_SGRBG12_1X12,
+ MEDIA_BUS_FMT_SRGGB12_1X12,
+ /* RGB */
+ MEDIA_BUS_FMT_RGB565_2X8_LE,
+ MEDIA_BUS_FMT_RGB565_2X8_BE,
+ /* YUV422 */
+ MEDIA_BUS_FMT_YUYV8_2X8,
+ MEDIA_BUS_FMT_UYVY8_2X8,
+ MEDIA_BUS_FMT_YVYU8_2X8,
+ MEDIA_BUS_FMT_UYVY8_2X8,
+ MEDIA_BUS_FMT_VYUY8_2X8,
+ MEDIA_BUS_FMT_YUYV8_1X16,
+ MEDIA_BUS_FMT_UYVY8_1X16,
+ MEDIA_BUS_FMT_YVYU8_1X16,
+ MEDIA_BUS_FMT_UYVY8_1X16,
+ MEDIA_BUS_FMT_VYUY8_1X16,
+ /* Compressed */
+ MEDIA_BUS_FMT_JPEG_1X8,
+};
+
+static bool sun6i_csi_bridge_mbus_code_check(u32 mbus_code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(sun6i_csi_bridge_mbus_codes); i++)
+ if (sun6i_csi_bridge_mbus_codes[i] == mbus_code)
+ return true;
+
+ return false;
+}
+
+/* V4L2 Subdev */
+
+static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on)
+{
+ struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_subdev *source_subdev;
+ /* Initialize to 0 to use both in disable label (ret != 0) and off. */
+ int ret = 0;
+
+ /* Source */
+
+ if (!csi_dev->bridge.source)
+ return -ENODEV;
+
+ source_subdev = csi_dev->bridge.source->subdev;
+
+ if (!on) {
+ v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ goto disable;
+ }
+
+ ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
+ if (ret && ret != -ENOIOCTLCMD)
+ goto disable;
+
+ return 0;
+
+disable:
+ csi_dev->bridge.source = NULL;
+
+ return ret;
+}
+
+static const struct v4l2_subdev_video_ops sun6i_csi_bridge_video_ops = {
+ .s_stream = sun6i_csi_bridge_s_stream,
+};
+
+static void
+sun6i_csi_bridge_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format)
+{
+ if (!sun6i_csi_bridge_mbus_code_check(mbus_format->code))
+ mbus_format->code = sun6i_csi_bridge_mbus_codes[0];
+
+ mbus_format->field = V4L2_FIELD_NONE;
+ mbus_format->colorspace = V4L2_COLORSPACE_RAW;
+ mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT;
+ mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int sun6i_csi_bridge_init_cfg(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state)
+{
+ unsigned int pad = SUN6I_CSI_BRIDGE_PAD_SINK;
+ struct v4l2_mbus_framefmt *mbus_format =
+ v4l2_subdev_get_try_format(subdev, state, pad);
+
+ mbus_format->code = sun6i_csi_bridge_mbus_codes[0];
+ mbus_format->width = 1280;
+ mbus_format->height = 720;
+
+ sun6i_csi_bridge_mbus_format_prepare(mbus_format);
+
+ return 0;
+}
+
+static int
+sun6i_csi_bridge_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code_enum)
+{
+ if (code_enum->index >= ARRAY_SIZE(sun6i_csi_bridge_mbus_codes))
+ return -EINVAL;
+
+ code_enum->code = sun6i_csi_bridge_mbus_codes[code_enum->index];
+
+ return 0;
+}
+
+static int sun6i_csi_bridge_get_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *mbus_format = *v4l2_subdev_get_try_format(subdev, state,
+ format->pad);
+ else
+ *mbus_format = csi_dev->bridge.mbus_format;
+
+ return 0;
+}
+
+static int sun6i_csi_bridge_set_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ sun6i_csi_bridge_mbus_format_prepare(mbus_format);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *v4l2_subdev_get_try_format(subdev, state, format->pad) =
+ *mbus_format;
+ else
+ csi_dev->bridge.mbus_format = *mbus_format;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops sun6i_csi_bridge_pad_ops = {
+ .init_cfg = sun6i_csi_bridge_init_cfg,
+ .enum_mbus_code = sun6i_csi_bridge_enum_mbus_code,
+ .get_fmt = sun6i_csi_bridge_get_fmt,
+ .set_fmt = sun6i_csi_bridge_set_fmt,
+};
+
+const struct v4l2_subdev_ops sun6i_csi_bridge_subdev_ops = {
+ .video = &sun6i_csi_bridge_video_ops,
+ .pad = &sun6i_csi_bridge_pad_ops,
+};
+
+/* Media Entity */
+
+static int sun6i_csi_bridge_link_validate(struct media_link *link)
+{
+ struct v4l2_subdev *subdev =
+ media_entity_to_v4l2_subdev(link->sink->entity);
+ struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
+ struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
+ struct device *dev = csi_dev->dev;
+ struct v4l2_subdev *source_subdev =
+ media_entity_to_v4l2_subdev(link->source->entity);
+ int ret;
+
+ /* Only support one enabled source at a time. */
+ if (bridge->source) {
+ dev_err(dev, "multiple sources are connected to the bridge\n");
+ return -EBUSY;
+ }
+
+ ret = v4l2_subdev_link_validate(link);
+ if (ret)
+ return ret;
+
+ if (source_subdev == bridge->source_parallel.subdev)
+ bridge->source = &bridge->source_parallel;
+ else
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct media_entity_operations sun6i_csi_bridge_entity_ops = {
+ .link_validate = sun6i_csi_bridge_link_validate,
+};
+
+/* V4L2 Async */
+
+static int sun6i_csi_bridge_link(struct sun6i_csi_device *csi_dev,
+ int sink_pad_index,
+ struct v4l2_subdev *remote_subdev,
+ bool enabled)
+{
+ struct device *dev = csi_dev->dev;
+ struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
+ struct media_entity *sink_entity = &subdev->entity;
+ struct media_entity *source_entity = &remote_subdev->entity;
+ int source_pad_index;
+ int ret;
+
+ /* Get the first remote source pad. */
+ ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (ret < 0) {
+ dev_err(dev, "missing source pad in external entity %s\n",
+ source_entity->name);
+ return -EINVAL;
+ }
+
+ source_pad_index = ret;
+
+ dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
+ source_pad_index, sink_entity->name, sink_pad_index);
+
+ ret = media_create_pad_link(source_entity, source_pad_index,
+ sink_entity, sink_pad_index,
+ enabled ? MEDIA_LNK_FL_ENABLED : 0);
+ if (ret < 0) {
+ dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
+ source_entity->name, source_pad_index,
+ sink_entity->name, sink_pad_index);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *remote_subdev,
+ struct v4l2_async_subdev *async_subdev)
+{
+ struct sun6i_csi_device *csi_dev =
+ container_of(notifier, struct sun6i_csi_device,
+ bridge.notifier);
+ struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
+ struct sun6i_csi_bridge_source *source = NULL;
+ struct fwnode_handle *fwnode = dev_fwnode(csi_dev->dev);
+ struct fwnode_handle *handle = NULL;
+ bool enabled;
+ int ret;
+
+ while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
+ struct fwnode_endpoint endpoint = { 0 };
+ struct fwnode_handle *remote_fwnode;
+
+ remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
+ if (!remote_fwnode)
+ continue;
+
+ if (remote_fwnode != remote_subdev->fwnode)
+ goto next;
+
+ ret = fwnode_graph_parse_endpoint(handle, &endpoint);
+ if (ret < 0)
+ goto next;
+
+ switch (endpoint.port) {
+ case SUN6I_CSI_PORT_PARALLEL:
+ source = &bridge->source_parallel;
+ enabled = true;
+ break;
+ default:
+ break;
+ }
+
+next:
+ fwnode_handle_put(remote_fwnode);
+ }
+
+ if (!source)
+ return -EINVAL;
+
+ source->subdev = remote_subdev;
+
+ return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK,
+ remote_subdev, enabled);
+}
+
+static int
+sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier)
+{
+ struct sun6i_csi_device *csi_dev =
+ container_of(notifier, struct sun6i_csi_device,
+ bridge.notifier);
+
+ return sun6i_csi_v4l2_complete(csi_dev);
+}
+
+static const struct v4l2_async_notifier_operations
+sun6i_csi_bridge_notifier_ops = {
+ .bound = sun6i_csi_bridge_notifier_bound,
+ .complete = sun6i_csi_bridge_notifier_complete,
+};
+
+/* Bridge */
+
+static int sun6i_csi_bridge_source_setup(struct sun6i_csi_device *csi_dev,
+ struct sun6i_csi_bridge_source *source,
+ u32 port,
+ enum v4l2_mbus_type *bus_types)
+{
+ struct device *dev = csi_dev->dev;
+ struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
+ struct v4l2_fwnode_endpoint *endpoint = &source->endpoint;
+ struct v4l2_async_subdev *async_subdev;
+ struct fwnode_handle *handle;
+ int ret;
+
+ handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0);
+ if (!handle)
+ return -ENODEV;
+
+ ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
+ if (ret)
+ goto complete;
+
+ if (bus_types) {
+ bool valid = false;
+ unsigned int i;
+
+ for (i = 0; bus_types[i] != V4L2_MBUS_INVALID; i++) {
+ if (endpoint->bus_type == bus_types[i]) {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid) {
+ dev_err(dev, "unsupported bus type for port %d\n",
+ port);
+ ret = -EINVAL;
+ goto complete;
+ }
+ }
+
+ async_subdev = v4l2_async_nf_add_fwnode_remote(notifier, handle,
+ struct v4l2_async_subdev);
+ if (IS_ERR(async_subdev)) {
+ ret = PTR_ERR(async_subdev);
+ goto complete;
+ }
+
+ source->expected = true;
+
+complete:
+ fwnode_handle_put(handle);
+
+ return ret;
+}
+
+int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev)
+{
+ struct device *dev = csi_dev->dev;
+ struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
+ struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
+ struct v4l2_subdev *subdev = &bridge->subdev;
+ struct v4l2_async_notifier *notifier = &bridge->notifier;
+ struct media_pad *pads = bridge->pads;
+ enum v4l2_mbus_type parallel_mbus_types[] = {
+ V4L2_MBUS_PARALLEL,
+ V4L2_MBUS_BT656,
+ V4L2_MBUS_INVALID
+ };
+ int ret;
+
+ /* V4L2 Subdev */
+
+ v4l2_subdev_init(subdev, &sun6i_csi_bridge_subdev_ops);
+ strscpy(subdev->name, SUN6I_CSI_BRIDGE_NAME, sizeof(subdev->name));
+ subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ subdev->owner = THIS_MODULE;
+ subdev->dev = dev;
+
+ v4l2_set_subdevdata(subdev, csi_dev);
+
+ /* Media Entity */
+
+ subdev->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+ subdev->entity.ops = &sun6i_csi_bridge_entity_ops;
+
+ /* Media Pads */
+
+ pads[SUN6I_CSI_BRIDGE_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ pads[SUN6I_CSI_BRIDGE_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE |
+ MEDIA_PAD_FL_MUST_CONNECT;
+
+ ret = media_entity_pads_init(&subdev->entity,
+ SUN6I_CSI_BRIDGE_PAD_COUNT, pads);
+ if (ret < 0)
+ return ret;
+
+ /* V4L2 Subdev */
+
+ ret = v4l2_device_register_subdev(v4l2_dev, subdev);
+ if (ret) {
+ dev_err(dev, "failed to register v4l2 subdev: %d\n", ret);
+ goto error_media_entity;
+ }
+
+ /* V4L2 Async */
+
+ v4l2_async_nf_init(notifier);
+ notifier->ops = &sun6i_csi_bridge_notifier_ops;
+
+ sun6i_csi_bridge_source_setup(csi_dev, &bridge->source_parallel,
+ SUN6I_CSI_PORT_PARALLEL,
+ parallel_mbus_types);
+
+ ret = v4l2_async_nf_register(v4l2_dev, notifier);
+ if (ret) {
+ dev_err(dev, "failed to register v4l2 async notifier: %d\n",
+ ret);
+ goto error_v4l2_async_notifier;
+ }
+
+ return 0;
+
+error_v4l2_async_notifier:
+ v4l2_async_nf_cleanup(notifier);
+
+ v4l2_device_unregister_subdev(subdev);
+
+error_media_entity:
+ media_entity_cleanup(&subdev->entity);
+
+ return ret;
+}
+
+void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev)
+{
+ struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
+ struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
+
+ v4l2_async_nf_unregister(notifier);
+ v4l2_async_nf_cleanup(notifier);
+
+ v4l2_device_unregister_subdev(subdev);
+
+ media_entity_cleanup(&subdev->entity);
+}
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
new file mode 100644
index 000000000000..2ee7878102b6
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_CSI_BRIDGE_H_
+#define _SUN6I_CSI_BRIDGE_H_
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#define SUN6I_CSI_BRIDGE_NAME "sun6i-csi-bridge"
+
+enum sun6i_csi_bridge_pad {
+ SUN6I_CSI_BRIDGE_PAD_SINK = 0,
+ SUN6I_CSI_BRIDGE_PAD_SOURCE = 1,
+ SUN6I_CSI_BRIDGE_PAD_COUNT = 2,
+};
+
+struct sun6i_csi_device;
+
+struct sun6i_csi_bridge_source {
+ struct v4l2_subdev *subdev;
+ struct v4l2_fwnode_endpoint endpoint;
+ bool expected;
+};
+
+struct sun6i_csi_bridge {
+ struct v4l2_subdev subdev;
+ struct v4l2_async_notifier notifier;
+ struct media_pad pads[2];
+ struct v4l2_mbus_framefmt mbus_format;
+
+ struct sun6i_csi_bridge_source source_parallel;
+ struct sun6i_csi_bridge_source *source;
+};
+
+/* Bridge */
+
+int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev);
+void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev);
+
+#endif
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index d32ff6b81f8a..fa5bf3697ace 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -632,6 +632,7 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
{
struct sun6i_video *video = &csi_dev->video;
struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
+ struct v4l2_subdev *bridge_subdev = &csi_dev->bridge.subdev;
struct video_device *video_dev = &video->video_dev;
struct vb2_queue *queue = &video->queue;
struct media_pad *pad = &video->pad;
@@ -715,8 +716,26 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
video_device_node_name(video_dev));
+ /* Media Pad Link */
+
+ ret = media_create_pad_link(&bridge_subdev->entity,
+ SUN6I_CSI_BRIDGE_PAD_SOURCE,
+ &video_dev->entity, 0,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret < 0) {
+ v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
+ bridge_subdev->entity.name,
+ SUN6I_CSI_BRIDGE_PAD_SOURCE,
+ video_dev->entity.name, 0);
+ goto error_video_device;
+ }
+
return 0;
+error_video_device:
+ vb2_video_unregister_device(video_dev);
+
error_media_entity:
media_entity_cleanup(&video_dev->entity);
--
2.34.1
Request our interrupt shared since it is typically shared with the isp
block. The interrupt routine looks good to go for shared irq.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index e3ac7dad86ae..b8e925f6bc09 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -285,8 +285,8 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev,
goto error_clk_rate_exclusive;
}
- ret = devm_request_irq(dev, irq, sun6i_csi_isr, 0, SUN6I_CSI_NAME,
- csi_dev);
+ ret = devm_request_irq(dev, irq, sun6i_csi_isr, IRQF_SHARED,
+ SUN6I_CSI_NAME, csi_dev);
if (ret) {
dev_err(dev, "failed to request interrupt\n");
goto error_clk_rate_exclusive;
--
2.34.1
Hi Paul,
Dne ponedeljek, 07. februar 2022 ob 09:43:43 CET je Paul Kocialkowski
napisal(a):
> Hi,
>
> On Sat 05 Feb 22, 14:14, Samuel Holland wrote:
> > On 2/5/22 12:53 PM, Paul Kocialkowski wrote:
> > > Since the V3s uses the internal mbus, document its compatible.
> > >
> > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > ---
> > >
> > > .../devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml | 1 +
> > > 1 file changed, 1 insertion(+)
> > >
> > > diff --git
> > > a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > yaml
> > > b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > yaml index 29c9961ee2d8..b67bf9261a6a 100644
> > > ---
> > > a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > yaml +++
> > > b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > yaml> >
> > > @@ -31,6 +31,7 @@ properties:
> > > - allwinner,sun5i-a13-mbus
> > > - allwinner,sun8i-h3-mbus
> > > - allwinner,sun8i-r40-mbus
> > >
> > > + - allwinner,sun8i-v3s-mbus
> >
> > Please enable the expanded binding added in commit 245578ba9f03
> > ("dt-bindings: arm: sunxi: Expand MBUS binding")[1] by adding the new
> > compatible to the "if" block lower in the file. That way we can add V3S
> > devfreq support in the future without changing that binding.
>
> I had missed that new driver but surely I will expand the updated binding.
>
> By the way do you have an explanation about the cell index given to the
> interconnects (after &mbus)?
This is mbus channel. You can find appropriate one checking DRAM driver in U-
Boot, where mbus is configured.
Best regards,
Jernej
>
> Paul
>
> > Regards,
> > Samuel
> >
> > [1]: https://git.kernel.org/torvalds/c/245578ba9f03
> >
> > > - allwinner,sun50i-a64-mbus
> > >
> > > reg:
In some situations the default rate of the module clock is not the
required one for operation (for example when reconfiguring the clock
tree to use a different parent). As a result, always set the correct
rate for the clock (and take care of cleanup).
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 54 ++++++++++++++-----
1 file changed, 41 insertions(+), 13 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 8155e9560164..2355088fdc37 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -154,9 +154,6 @@ int sun6i_csi_set_power(struct sun6i_csi_device *csi_dev, bool enable)
regmap_update_bits(regmap, CSI_EN_REG, CSI_EN_CSI_EN, 0);
clk_disable_unprepare(csi_dev->clk_ram);
- if (of_device_is_compatible(dev->of_node,
- "allwinner,sun50i-a64-csi"))
- clk_rate_exclusive_put(csi_dev->clk_mod);
clk_disable_unprepare(csi_dev->clk_mod);
reset_control_assert(csi_dev->reset);
return 0;
@@ -168,9 +165,6 @@ int sun6i_csi_set_power(struct sun6i_csi_device *csi_dev, bool enable)
return ret;
}
- if (of_device_is_compatible(dev->of_node, "allwinner,sun50i-a64-csi"))
- clk_set_rate_exclusive(csi_dev->clk_mod, 300000000);
-
ret = clk_prepare_enable(csi_dev->clk_ram);
if (ret) {
dev_err(csi_dev->dev, "Enable clk_dram_csi clk err %d\n", ret);
@@ -190,8 +184,6 @@ int sun6i_csi_set_power(struct sun6i_csi_device *csi_dev, bool enable)
clk_ram_disable:
clk_disable_unprepare(csi_dev->clk_ram);
clk_mod_disable:
- if (of_device_is_compatible(dev->of_node, "allwinner,sun50i-a64-csi"))
- clk_rate_exclusive_put(csi_dev->clk_mod);
clk_disable_unprepare(csi_dev->clk_mod);
return ret;
}
@@ -819,6 +811,7 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev,
struct platform_device *platform_dev)
{
struct device *dev = csi_dev->dev;
+ unsigned long clk_mod_rate;
void __iomem *io_base;
int ret;
int irq;
@@ -856,28 +849,53 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev,
return PTR_ERR(csi_dev->clk_ram);
}
+ if (of_device_is_compatible(dev->of_node, "allwinner,sun50i-a64-csi"))
+ clk_mod_rate = 300000000;
+ else
+ clk_mod_rate = 297000000;
+
+ ret = clk_set_rate_exclusive(csi_dev->clk_mod, clk_mod_rate);
+ if (ret) {
+ dev_err(dev, "failed to set mod clock rate\n");
+ return ret;
+ }
+
/* Reset */
csi_dev->reset = devm_reset_control_get_shared(dev, NULL);
if (IS_ERR(csi_dev->reset)) {
dev_err(dev, "failed to acquire reset\n");
- return PTR_ERR(csi_dev->reset);
+ ret = PTR_ERR(csi_dev->reset);
+ goto error_clk_rate_exclusive;
}
/* Interrupt */
irq = platform_get_irq(platform_dev, 0);
- if (irq < 0)
- return -ENXIO;
+ if (irq < 0) {
+ dev_err(dev, "failed to get interrupt\n");
+ ret = -ENXIO;
+ goto error_clk_rate_exclusive;
+ }
ret = devm_request_irq(dev, irq, sun6i_csi_isr, 0, SUN6I_CSI_NAME,
csi_dev);
if (ret) {
dev_err(dev, "failed to request interrupt\n");
- return ret;
+ goto error_clk_rate_exclusive;
}
return 0;
+
+error_clk_rate_exclusive:
+ clk_rate_exclusive_put(csi_dev->clk_mod);
+
+ return ret;
+}
+
+static void sun6i_csi_resources_cleanup(struct sun6i_csi_device *csi_dev)
+{
+ clk_rate_exclusive_put(csi_dev->clk_mod);
}
static int sun6i_csi_probe(struct platform_device *platform_dev)
@@ -897,7 +915,16 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)
if (ret)
return ret;
- return sun6i_csi_v4l2_init(csi_dev);
+ ret = sun6i_csi_v4l2_init(csi_dev);
+ if (ret)
+ goto error_resources;
+
+ return 0;
+
+error_resources:
+ sun6i_csi_resources_cleanup(csi_dev);
+
+ return ret;
}
static int sun6i_csi_remove(struct platform_device *pdev)
@@ -905,6 +932,7 @@ static int sun6i_csi_remove(struct platform_device *pdev)
struct sun6i_csi_device *csi_dev = platform_get_drvdata(pdev);
sun6i_csi_v4l2_cleanup(csi_dev);
+ sun6i_csi_resources_cleanup(csi_dev);
return 0;
}
--
2.34.1
Define and export useful helpers to access dimensions and pixel format.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../sunxi/sun6i-csi/sun6i_csi_capture.c | 19 +++++++++++++++++++
.../sunxi/sun6i-csi/sun6i_csi_capture.h | 5 +++++
2 files changed, 24 insertions(+)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
index bcbb4ffbb517..7b8556a25c61 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c
@@ -25,6 +25,25 @@
/* Helpers */
+void sun6i_csi_capture_dimensions(struct sun6i_csi_device *csi_dev,
+ unsigned int *width, unsigned int *height)
+{
+ if (width)
+ *width = csi_dev->capture.format.fmt.pix.width;
+ if (height)
+ *height = csi_dev->capture.format.fmt.pix.height;
+}
+
+void sun6i_csi_capture_format(struct sun6i_csi_device *csi_dev,
+ u32 *pixelformat, u32 *field)
+{
+ if (pixelformat)
+ *pixelformat = csi_dev->capture.format.fmt.pix.pixelformat;
+
+ if (field)
+ *field = csi_dev->capture.format.fmt.pix.field;
+}
+
static struct v4l2_subdev *
sun6i_csi_capture_remote_subdev(struct sun6i_csi_capture *capture, u32 *pad)
{
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
index 7fa66a2af5ec..935f35b7049a 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h
@@ -37,6 +37,11 @@ struct sun6i_csi_capture {
u32 mbus_code;
};
+void sun6i_csi_capture_dimensions(struct sun6i_csi_device *csi_dev,
+ unsigned int *width, unsigned int *height);
+void sun6i_csi_capture_format(struct sun6i_csi_device *csi_dev,
+ u32 *pixelformat, u32 *field);
+
void sun6i_csi_capture_sync(struct sun6i_csi_device *csi_dev);
void sun6i_csi_capture_frame_done(struct sun6i_csi_device *csi_dev);
--
2.34.1
MIPI CSI-2 is supported on the V3s with an A31-based MIPI CSI-2 bridge
controller. The controller uses a separate D-PHY, which is the same
that is otherwise used for MIPI DSI, but used in Rx mode.
On the V3s, the CSI0 controller is dedicated to MIPI CSI-2 as it does
not have access to any parallel interface pins.
Add all the necessary nodes (CSI0, MIPI CSI-2 bridge and D-PHY) to
support the MIPI CSI-2 interface.
Note that a fwnode graph link is created between CSI0 and MIPI CSI-2
even when no sensor is connected. This will result in a probe failure
for the controller as long as no sensor is connected but this is fine
since no other interface is available.
While the interconnects property is used to inherit the proper dma
ranges, the associated index for the cell is set to 0 since no
particular meaning is attached to it. This might need to be changed
later on (when identifying a proper mbus channel becomes relevant,
e.g. for things like QoS).
Signed-off-by: Paul Kocialkowski <[email protected]>
---
arch/arm/boot/dts/sun8i-v3s.dtsi | 71 ++++++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-v3s.dtsi b/arch/arm/boot/dts/sun8i-v3s.dtsi
index 1b3048b6f197..f5f7dfec49f9 100644
--- a/arch/arm/boot/dts/sun8i-v3s.dtsi
+++ b/arch/arm/boot/dts/sun8i-v3s.dtsi
@@ -614,6 +614,77 @@ gic: interrupt-controller@1c81000 {
interrupts = <GIC_PPI 9 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
};
+ csi0: camera@1cb0000 {
+ compatible = "allwinner,sun8i-v3s-csi";
+ reg = <0x01cb0000 0x1000>;
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_CSI>,
+ <&ccu CLK_CSI1_SCLK>,
+ <&ccu CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = <&ccu RST_BUS_CSI>;
+ interconnects = <&mbus 0>;
+ interconnect-names = "dma-mem";
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@1 {
+ reg = <1>;
+
+ csi0_in_mipi_csi2: endpoint {
+ remote-endpoint = <&mipi_csi2_out_csi0>;
+ };
+ };
+ };
+ };
+
+ mipi_csi2: csi@1cb1000 {
+ compatible = "allwinner,sun8i-v3s-mipi-csi2",
+ "allwinner,sun6i-a31-mipi-csi2";
+ reg = <0x01cb1000 0x1000>;
+ interrupts = <GIC_SPI 90 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_CSI>,
+ <&ccu CLK_CSI1_SCLK>;
+ clock-names = "bus", "mod";
+ resets = <&ccu RST_BUS_CSI>;
+ status = "disabled";
+
+ phys = <&dphy>;
+ phy-names = "dphy";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mipi_csi2_in: port@0 {
+ reg = <0>;
+ };
+
+ mipi_csi2_out: port@1 {
+ reg = <1>;
+
+ mipi_csi2_out_csi0: endpoint {
+ remote-endpoint = <&csi0_in_mipi_csi2>;
+ };
+ };
+ };
+ };
+
+ dphy: d-phy@1cb2000 {
+ compatible = "allwinner,sun6i-a31-mipi-dphy";
+ reg = <0x01cb2000 0x1000>;
+ clocks = <&ccu CLK_BUS_CSI>,
+ <&ccu CLK_MIPI_CSI>;
+ clock-names = "bus", "mod";
+ resets = <&ccu RST_BUS_CSI>;
+ allwinner,direction = "rx";
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
csi1: camera@1cb4000 {
compatible = "allwinner,sun8i-v3s-csi";
reg = <0x01cb4000 0x3000>;
--
2.34.1
There is no particular need to register the media device in the
subdev notify complete callback.
Register it in the v4l2 code instead where it's more in-context.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 688288afae68..6f04f86504bf 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -638,7 +638,7 @@ static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier)
if (ret < 0)
return ret;
- return media_device_register(&v4l2->media_dev);
+ return 0;
}
static const struct v4l2_async_notifier_operations sun6i_csi_async_ops = {
@@ -687,6 +687,12 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
media_device_init(media_dev);
+ ret = media_device_register(media_dev);
+ if (ret) {
+ dev_err(dev, "failed to register media device: %d\n", ret);
+ goto error_media;
+ }
+
/* V4L2 Control Handler */
ret = v4l2_ctrl_handler_init(&v4l2->ctrl_handler, 0);
@@ -746,6 +752,7 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
error_media:
+ media_device_unregister(media_dev);
media_device_cleanup(media_dev);
return ret;
--
2.34.1
Some code cleanups, renames, variable lowerings and moving things around for
better organization. No functional change intended.
Signed-off-by: Paul Kocialkowski <[email protected]>
---
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 4 +-
.../platform/sunxi/sun6i-csi/sun6i_video.c | 508 ++++++++++--------
.../platform/sunxi/sun6i-csi/sun6i_video.h | 18 +-
3 files changed, 286 insertions(+), 244 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 622fb86c3170..9ff02f3d8037 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -595,7 +595,7 @@ static int sun6i_csi_link_entity(struct sun6i_csi_device *csi_dev,
src_pad_index = ret;
- sink = &csi_dev->video.vdev.entity;
+ sink = &csi_dev->video.video_dev.entity;
sink_pad = &csi_dev->video.pad;
dev_dbg(csi_dev->dev, "creating %s:%u -> %s:%u link\n",
@@ -708,7 +708,7 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
/* Video */
- ret = sun6i_video_init(&csi_dev->video, csi_dev, SUN6I_CSI_NAME);
+ ret = sun6i_video_setup(&csi_dev->video, csi_dev);
if (ret)
goto error_v4l2_device;
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index c152c0ceb6d5..427b23184a15 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -24,14 +24,34 @@
#define MAX_HEIGHT (4800)
struct sun6i_csi_buffer {
- struct vb2_v4l2_buffer vb;
+ struct vb2_v4l2_buffer v4l2_buffer;
struct list_head list;
dma_addr_t dma_addr;
bool queued_to_csi;
};
-static const u32 supported_pixformats[] = {
+/* Helpers */
+
+static struct v4l2_subdev *
+sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad)
+{
+ struct media_pad *remote;
+
+ remote = media_entity_remote_pad(&video->pad);
+
+ if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
+ return NULL;
+
+ if (pad)
+ *pad = remote->index;
+
+ return media_entity_to_v4l2_subdev(remote->entity);
+}
+
+/* Format */
+
+static const u32 sun6i_video_formats[] = {
V4L2_PIX_FMT_SBGGR8,
V4L2_PIX_FMT_SGBRG8,
V4L2_PIX_FMT_SGRBG8,
@@ -61,77 +81,80 @@ static const u32 supported_pixformats[] = {
V4L2_PIX_FMT_JPEG,
};
-static bool is_pixformat_valid(unsigned int pixformat)
+static bool sun6i_video_format_check(u32 format)
{
unsigned int i;
- for (i = 0; i < ARRAY_SIZE(supported_pixformats); i++)
- if (supported_pixformats[i] == pixformat)
+ for (i = 0; i < ARRAY_SIZE(sun6i_video_formats); i++)
+ if (sun6i_video_formats[i] == format)
return true;
return false;
}
-static struct v4l2_subdev *
-sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad)
-{
- struct media_pad *remote;
-
- remote = media_entity_remote_pad(&video->pad);
+/* Queue */
- if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
- return NULL;
-
- if (pad)
- *pad = remote->index;
-
- return media_entity_to_v4l2_subdev(remote->entity);
-}
-
-static int sun6i_video_queue_setup(struct vb2_queue *vq,
- unsigned int *nbuffers,
- unsigned int *nplanes,
+static int sun6i_video_queue_setup(struct vb2_queue *queue,
+ unsigned int *buffers_count,
+ unsigned int *planes_count,
unsigned int sizes[],
struct device *alloc_devs[])
{
- struct sun6i_video *video = vb2_get_drv_priv(vq);
- unsigned int size = video->fmt.fmt.pix.sizeimage;
+ struct sun6i_video *video = vb2_get_drv_priv(queue);
+ unsigned int size = video->format.fmt.pix.sizeimage;
- if (*nplanes)
+ if (*planes_count)
return sizes[0] < size ? -EINVAL : 0;
- *nplanes = 1;
+ *planes_count = 1;
sizes[0] = size;
return 0;
}
-static int sun6i_video_buffer_prepare(struct vb2_buffer *vb)
+static int sun6i_video_buffer_prepare(struct vb2_buffer *buffer)
{
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
- struct sun6i_csi_buffer *buf =
- container_of(vbuf, struct sun6i_csi_buffer, vb);
- struct sun6i_video *video = vb2_get_drv_priv(vb->vb2_queue);
- unsigned long size = video->fmt.fmt.pix.sizeimage;
-
- if (vb2_plane_size(vb, 0) < size) {
- v4l2_err(video->vdev.v4l2_dev, "buffer too small (%lu < %lu)\n",
- vb2_plane_size(vb, 0), size);
+ struct sun6i_video *video = vb2_get_drv_priv(buffer->vb2_queue);
+ struct sun6i_csi_device *csi_dev = video->csi_dev;
+ struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
+ struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer);
+ struct sun6i_csi_buffer *csi_buffer =
+ container_of(v4l2_buffer, struct sun6i_csi_buffer, v4l2_buffer);
+ unsigned long size = video->format.fmt.pix.sizeimage;
+
+ if (vb2_plane_size(buffer, 0) < size) {
+ v4l2_err(v4l2_dev, "buffer too small (%lu < %lu)\n",
+ vb2_plane_size(buffer, 0), size);
return -EINVAL;
}
- vb2_set_plane_payload(vb, 0, size);
-
- buf->dma_addr = vb2_dma_contig_plane_dma_addr(vb, 0);
+ vb2_set_plane_payload(buffer, 0, size);
- vbuf->field = video->fmt.fmt.pix.field;
+ csi_buffer->dma_addr = vb2_dma_contig_plane_dma_addr(buffer, 0);
+ v4l2_buffer->field = video->format.fmt.pix.field;
return 0;
}
-static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
+static void sun6i_video_buffer_queue(struct vb2_buffer *buffer)
+{
+ struct sun6i_video *video = vb2_get_drv_priv(buffer->vb2_queue);
+ struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer);
+ struct sun6i_csi_buffer *csi_buffer =
+ container_of(v4l2_buffer, struct sun6i_csi_buffer, v4l2_buffer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&video->dma_queue_lock, flags);
+ csi_buffer->queued_to_csi = false;
+ list_add_tail(&csi_buffer->list, &video->dma_queue);
+ spin_unlock_irqrestore(&video->dma_queue_lock, flags);
+}
+
+static int sun6i_video_start_streaming(struct vb2_queue *queue,
+ unsigned int count)
{
- struct sun6i_video *video = vb2_get_drv_priv(vq);
+ struct sun6i_video *video = vb2_get_drv_priv(queue);
+ struct video_device *video_dev = &video->video_dev;
struct sun6i_csi_buffer *buf;
struct sun6i_csi_buffer *next_buf;
struct sun6i_csi_config config;
@@ -141,30 +164,30 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
video->sequence = 0;
- ret = media_pipeline_start(&video->vdev.entity, &video->vdev.pipe);
+ ret = media_pipeline_start(&video_dev->entity, &video_dev->pipe);
if (ret < 0)
- goto clear_dma_queue;
+ goto error_dma_queue_flush;
if (video->mbus_code == 0) {
ret = -EINVAL;
- goto stop_media_pipeline;
+ goto error_media_pipeline;
}
subdev = sun6i_video_remote_subdev(video, NULL);
if (!subdev) {
ret = -EINVAL;
- goto stop_media_pipeline;
+ goto error_media_pipeline;
}
- config.pixelformat = video->fmt.fmt.pix.pixelformat;
+ config.pixelformat = video->format.fmt.pix.pixelformat;
config.code = video->mbus_code;
- config.field = video->fmt.fmt.pix.field;
- config.width = video->fmt.fmt.pix.width;
- config.height = video->fmt.fmt.pix.height;
+ config.field = video->format.fmt.pix.field;
+ config.width = video->format.fmt.pix.width;
+ config.height = video->format.fmt.pix.height;
ret = sun6i_csi_update_config(video->csi_dev, &config);
if (ret < 0)
- goto stop_media_pipeline;
+ goto error_media_pipeline;
spin_lock_irqsave(&video->dma_queue_lock, flags);
@@ -200,27 +223,30 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
ret = v4l2_subdev_call(subdev, video, s_stream, 1);
if (ret && ret != -ENOIOCTLCMD)
- goto stop_csi_stream;
+ goto error_stream;
return 0;
-stop_csi_stream:
+error_stream:
sun6i_csi_set_stream(video->csi_dev, false);
-stop_media_pipeline:
- media_pipeline_stop(&video->vdev.entity);
-clear_dma_queue:
+
+error_media_pipeline:
+ media_pipeline_stop(&video_dev->entity);
+
+error_dma_queue_flush:
spin_lock_irqsave(&video->dma_queue_lock, flags);
list_for_each_entry(buf, &video->dma_queue, list)
- vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
+ vb2_buffer_done(&buf->v4l2_buffer.vb2_buf,
+ VB2_BUF_STATE_QUEUED);
INIT_LIST_HEAD(&video->dma_queue);
spin_unlock_irqrestore(&video->dma_queue_lock, flags);
return ret;
}
-static void sun6i_video_stop_streaming(struct vb2_queue *vq)
+static void sun6i_video_stop_streaming(struct vb2_queue *queue)
{
- struct sun6i_video *video = vb2_get_drv_priv(vq);
+ struct sun6i_video *video = vb2_get_drv_priv(queue);
struct v4l2_subdev *subdev;
unsigned long flags;
struct sun6i_csi_buffer *buf;
@@ -231,35 +257,21 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq)
sun6i_csi_set_stream(video->csi_dev, false);
- media_pipeline_stop(&video->vdev.entity);
+ media_pipeline_stop(&video->video_dev.entity);
/* Release all active buffers */
spin_lock_irqsave(&video->dma_queue_lock, flags);
list_for_each_entry(buf, &video->dma_queue, list)
- vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ vb2_buffer_done(&buf->v4l2_buffer.vb2_buf, VB2_BUF_STATE_ERROR);
INIT_LIST_HEAD(&video->dma_queue);
spin_unlock_irqrestore(&video->dma_queue_lock, flags);
}
-static void sun6i_video_buffer_queue(struct vb2_buffer *vb)
-{
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
- struct sun6i_csi_buffer *buf =
- container_of(vbuf, struct sun6i_csi_buffer, vb);
- struct sun6i_video *video = vb2_get_drv_priv(vb->vb2_queue);
- unsigned long flags;
-
- spin_lock_irqsave(&video->dma_queue_lock, flags);
- buf->queued_to_csi = false;
- list_add_tail(&buf->list, &video->dma_queue);
- spin_unlock_irqrestore(&video->dma_queue_lock, flags);
-}
-
void sun6i_video_frame_done(struct sun6i_video *video)
{
struct sun6i_csi_buffer *buf;
struct sun6i_csi_buffer *next_buf;
- struct vb2_v4l2_buffer *vbuf;
+ struct vb2_v4l2_buffer *v4l2_buffer;
spin_lock(&video->dma_queue_lock);
@@ -267,7 +279,7 @@ void sun6i_video_frame_done(struct sun6i_video *video)
struct sun6i_csi_buffer, list);
if (list_is_last(&buf->list, &video->dma_queue)) {
dev_dbg(video->csi_dev->dev, "Frame dropped!\n");
- goto unlock;
+ goto complete;
}
next_buf = list_next_entry(buf, list);
@@ -280,14 +292,14 @@ void sun6i_video_frame_done(struct sun6i_video *video)
next_buf->queued_to_csi = true;
sun6i_csi_update_buf_addr(video->csi_dev, next_buf->dma_addr);
dev_dbg(video->csi_dev->dev, "Frame dropped!\n");
- goto unlock;
+ goto complete;
}
list_del(&buf->list);
- vbuf = &buf->vb;
- vbuf->vb2_buf.timestamp = ktime_get_ns();
- vbuf->sequence = video->sequence;
- vb2_buffer_done(&vbuf->vb2_buf, VB2_BUF_STATE_DONE);
+ v4l2_buffer = &buf->v4l2_buffer;
+ v4l2_buffer->vb2_buf.timestamp = ktime_get_ns();
+ v4l2_buffer->sequence = video->sequence;
+ vb2_buffer_done(&v4l2_buffer->vb2_buf, VB2_BUF_STATE_DONE);
/* Prepare buffer for next frame but one. */
if (!list_is_last(&next_buf->list, &video->dma_queue)) {
@@ -298,161 +310,169 @@ void sun6i_video_frame_done(struct sun6i_video *video)
dev_dbg(video->csi_dev->dev, "Next frame will be dropped!\n");
}
-unlock:
+complete:
video->sequence++;
spin_unlock(&video->dma_queue_lock);
}
-static const struct vb2_ops sun6i_csi_vb2_ops = {
+static const struct vb2_ops sun6i_video_queue_ops = {
.queue_setup = sun6i_video_queue_setup,
- .wait_prepare = vb2_ops_wait_prepare,
- .wait_finish = vb2_ops_wait_finish,
.buf_prepare = sun6i_video_buffer_prepare,
+ .buf_queue = sun6i_video_buffer_queue,
.start_streaming = sun6i_video_start_streaming,
.stop_streaming = sun6i_video_stop_streaming,
- .buf_queue = sun6i_video_buffer_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
};
-static int vidioc_querycap(struct file *file, void *priv,
- struct v4l2_capability *cap)
+/* V4L2 Device */
+
+static int sun6i_video_querycap(struct file *file, void *private,
+ struct v4l2_capability *capability)
{
struct sun6i_video *video = video_drvdata(file);
+ struct sun6i_csi_device *csi_dev = video->csi_dev;
+ struct video_device *video_dev = &video->video_dev;
- strscpy(cap->driver, "sun6i-video", sizeof(cap->driver));
- strscpy(cap->card, video->vdev.name, sizeof(cap->card));
- snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
- video->csi_dev->dev->of_node->name);
+ strscpy(capability->driver, SUN6I_CSI_NAME, sizeof(capability->driver));
+ strscpy(capability->card, video_dev->name, sizeof(capability->card));
+ snprintf(capability->bus_info, sizeof(capability->bus_info),
+ "platform:%s", dev_name(csi_dev->dev));
return 0;
}
-static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
+static int sun6i_video_enum_fmt(struct file *file, void *private,
+ struct v4l2_fmtdesc *fmtdesc)
{
- u32 index = f->index;
+ u32 index = fmtdesc->index;
- if (index >= ARRAY_SIZE(supported_pixformats))
+ if (index >= ARRAY_SIZE(sun6i_video_formats))
return -EINVAL;
- f->pixelformat = supported_pixformats[index];
+ fmtdesc->pixelformat = sun6i_video_formats[index];
return 0;
}
-static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *fmt)
+static int sun6i_video_g_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
{
struct sun6i_video *video = video_drvdata(file);
- *fmt = video->fmt;
+ *format = video->format;
return 0;
}
-static int sun6i_video_try_fmt(struct sun6i_video *video,
- struct v4l2_format *f)
+static int sun6i_video_format_try(struct sun6i_video *video,
+ struct v4l2_format *format)
{
- struct v4l2_pix_format *pixfmt = &f->fmt.pix;
+ struct v4l2_pix_format *pix_format = &format->fmt.pix;
int bpp;
- if (!is_pixformat_valid(pixfmt->pixelformat))
- pixfmt->pixelformat = supported_pixformats[0];
+ if (!sun6i_video_format_check(pix_format->pixelformat))
+ pix_format->pixelformat = sun6i_video_formats[0];
- v4l_bound_align_image(&pixfmt->width, MIN_WIDTH, MAX_WIDTH, 1,
- &pixfmt->height, MIN_HEIGHT, MAX_WIDTH, 1, 1);
+ v4l_bound_align_image(&pix_format->width, MIN_WIDTH, MAX_WIDTH, 1,
+ &pix_format->height, MIN_HEIGHT, MAX_WIDTH, 1, 1);
- bpp = sun6i_csi_get_bpp(pixfmt->pixelformat);
- pixfmt->bytesperline = (pixfmt->width * bpp) >> 3;
- pixfmt->sizeimage = pixfmt->bytesperline * pixfmt->height;
+ bpp = sun6i_csi_get_bpp(pix_format->pixelformat);
+ pix_format->bytesperline = (pix_format->width * bpp) >> 3;
+ pix_format->sizeimage = pix_format->bytesperline * pix_format->height;
- if (pixfmt->field == V4L2_FIELD_ANY)
- pixfmt->field = V4L2_FIELD_NONE;
+ if (pix_format->field == V4L2_FIELD_ANY)
+ pix_format->field = V4L2_FIELD_NONE;
- pixfmt->colorspace = V4L2_COLORSPACE_RAW;
- pixfmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
- pixfmt->quantization = V4L2_QUANTIZATION_DEFAULT;
- pixfmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ pix_format->colorspace = V4L2_COLORSPACE_RAW;
+ pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pix_format->quantization = V4L2_QUANTIZATION_DEFAULT;
+ pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
return 0;
}
-static int sun6i_video_set_fmt(struct sun6i_video *video, struct v4l2_format *f)
+static int sun6i_video_format_set(struct sun6i_video *video,
+ struct v4l2_format *format)
{
int ret;
- ret = sun6i_video_try_fmt(video, f);
+ ret = sun6i_video_format_try(video, format);
if (ret)
return ret;
- video->fmt = *f;
+ video->format = *format;
return 0;
}
-static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+static int sun6i_video_s_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
{
struct sun6i_video *video = video_drvdata(file);
- if (vb2_is_busy(&video->vb2_vidq))
+ if (vb2_is_busy(&video->queue))
return -EBUSY;
- return sun6i_video_set_fmt(video, f);
+ return sun6i_video_format_set(video, format);
}
-static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+static int sun6i_video_try_fmt(struct file *file, void *private,
+ struct v4l2_format *format)
{
struct sun6i_video *video = video_drvdata(file);
- return sun6i_video_try_fmt(video, f);
+ return sun6i_video_format_try(video, format);
}
-static int vidioc_enum_input(struct file *file, void *fh,
- struct v4l2_input *inp)
+static int sun6i_video_enum_input(struct file *file, void *private,
+ struct v4l2_input *input)
{
- if (inp->index != 0)
+ if (input->index != 0)
return -EINVAL;
- strscpy(inp->name, "camera", sizeof(inp->name));
- inp->type = V4L2_INPUT_TYPE_CAMERA;
+ input->type = V4L2_INPUT_TYPE_CAMERA;
+ strscpy(input->name, "Camera", sizeof(input->name));
return 0;
}
-static int vidioc_g_input(struct file *file, void *fh, unsigned int *i)
+static int sun6i_video_g_input(struct file *file, void *private,
+ unsigned int *index)
{
- *i = 0;
+ *index = 0;
return 0;
}
-static int vidioc_s_input(struct file *file, void *fh, unsigned int i)
+static int sun6i_video_s_input(struct file *file, void *private,
+ unsigned int index)
{
- if (i != 0)
+ if (index != 0)
return -EINVAL;
return 0;
}
static const struct v4l2_ioctl_ops sun6i_video_ioctl_ops = {
- .vidioc_querycap = vidioc_querycap,
- .vidioc_enum_fmt_vid_cap = vidioc_enum_fmt_vid_cap,
- .vidioc_g_fmt_vid_cap = vidioc_g_fmt_vid_cap,
- .vidioc_s_fmt_vid_cap = vidioc_s_fmt_vid_cap,
- .vidioc_try_fmt_vid_cap = vidioc_try_fmt_vid_cap,
+ .vidioc_querycap = sun6i_video_querycap,
+
+ .vidioc_enum_fmt_vid_cap = sun6i_video_enum_fmt,
+ .vidioc_g_fmt_vid_cap = sun6i_video_g_fmt,
+ .vidioc_s_fmt_vid_cap = sun6i_video_s_fmt,
+ .vidioc_try_fmt_vid_cap = sun6i_video_try_fmt,
- .vidioc_enum_input = vidioc_enum_input,
- .vidioc_s_input = vidioc_s_input,
- .vidioc_g_input = vidioc_g_input,
+ .vidioc_enum_input = sun6i_video_enum_input,
+ .vidioc_g_input = sun6i_video_g_input,
+ .vidioc_s_input = sun6i_video_s_input,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
.vidioc_reqbufs = vb2_ioctl_reqbufs,
.vidioc_querybuf = vb2_ioctl_querybuf,
- .vidioc_qbuf = vb2_ioctl_qbuf,
.vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
.vidioc_dqbuf = vb2_ioctl_dqbuf,
- .vidioc_create_bufs = vb2_ioctl_create_bufs,
- .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
.vidioc_streamon = vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
@@ -461,9 +481,8 @@ static const struct v4l2_ioctl_ops sun6i_video_ioctl_ops = {
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
-/* -----------------------------------------------------------------------------
- * V4L2 file operations
- */
+/* V4L2 File */
+
static int sun6i_video_open(struct file *file)
{
struct sun6i_video *video = video_drvdata(file);
@@ -474,44 +493,46 @@ static int sun6i_video_open(struct file *file)
ret = v4l2_fh_open(file);
if (ret < 0)
- goto unlock;
+ goto error_lock;
- ret = v4l2_pipeline_pm_get(&video->vdev.entity);
+ ret = v4l2_pipeline_pm_get(&video->video_dev.entity);
if (ret < 0)
- goto fh_release;
-
- /* check if already powered */
- if (!v4l2_fh_is_singular_file(file))
- goto unlock;
+ goto error_v4l2_fh;
- ret = sun6i_csi_set_power(video->csi_dev, true);
- if (ret < 0)
- goto fh_release;
+ /* Power on at first open. */
+ if (v4l2_fh_is_singular_file(file)) {
+ ret = sun6i_csi_set_power(video->csi_dev, true);
+ if (ret < 0)
+ goto error_v4l2_fh;
+ }
mutex_unlock(&video->lock);
+
return 0;
-fh_release:
+error_v4l2_fh:
v4l2_fh_release(file);
-unlock:
+
+error_lock:
mutex_unlock(&video->lock);
+
return ret;
}
static int sun6i_video_close(struct file *file)
{
struct sun6i_video *video = video_drvdata(file);
- bool last_fh;
+ bool last_close;
mutex_lock(&video->lock);
- last_fh = v4l2_fh_is_singular_file(file);
+ last_close = v4l2_fh_is_singular_file(file);
_vb2_fop_release(file, NULL);
+ v4l2_pipeline_pm_put(&video->video_dev.entity);
- v4l2_pipeline_pm_put(&video->vdev.entity);
-
- if (last_fh)
+ /* Power off at last close. */
+ if (last_close)
sun6i_csi_set_power(video->csi_dev, false);
mutex_unlock(&video->lock);
@@ -528,9 +549,8 @@ static const struct v4l2_file_operations sun6i_video_fops = {
.poll = vb2_fop_poll
};
-/* -----------------------------------------------------------------------------
- * Media Operations
- */
+/* Media Entity */
+
static int sun6i_video_link_validate_get_format(struct media_pad *pad,
struct v4l2_subdev_format *fmt)
{
@@ -567,20 +587,20 @@ static int sun6i_video_link_validate(struct media_link *link)
return ret;
if (!sun6i_csi_is_format_supported(video->csi_dev,
- video->fmt.fmt.pix.pixelformat,
+ video->format.fmt.pix.pixelformat,
source_fmt.format.code)) {
dev_err(video->csi_dev->dev,
"Unsupported pixformat: 0x%x with mbus code: 0x%x!\n",
- video->fmt.fmt.pix.pixelformat,
+ video->format.fmt.pix.pixelformat,
source_fmt.format.code);
return -EPIPE;
}
- if (source_fmt.format.width != video->fmt.fmt.pix.width ||
- source_fmt.format.height != video->fmt.fmt.pix.height) {
+ if (source_fmt.format.width != video->format.fmt.pix.width ||
+ source_fmt.format.height != video->format.fmt.pix.height) {
dev_err(video->csi_dev->dev,
"Wrong width or height %ux%u (%ux%u expected)\n",
- video->fmt.fmt.pix.width, video->fmt.fmt.pix.height,
+ video->format.fmt.pix.width, video->format.fmt.pix.height,
source_fmt.format.width, source_fmt.format.height);
return -EPIPE;
}
@@ -594,90 +614,112 @@ static const struct media_entity_operations sun6i_video_media_ops = {
.link_validate = sun6i_video_link_validate
};
-int sun6i_video_init(struct sun6i_video *video,
- struct sun6i_csi_device *csi_dev, const char *name)
+/* Video */
+
+int sun6i_video_setup(struct sun6i_video *video,
+ struct sun6i_csi_device *csi_dev)
{
- struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
- struct video_device *vdev = &video->vdev;
- struct vb2_queue *vidq = &video->vb2_vidq;
- struct v4l2_format fmt = { 0 };
+ struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
+ struct video_device *video_dev = &video->video_dev;
+ struct vb2_queue *queue = &video->queue;
+ struct media_pad *pad = &video->pad;
+ struct v4l2_format format = { 0 };
+ struct v4l2_pix_format *pix_format = &format.fmt.pix;
int ret;
video->csi_dev = csi_dev;
- /* Initialize the media entity... */
- video->pad.flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT;
- vdev->entity.ops = &sun6i_video_media_ops;
- ret = media_entity_pads_init(&vdev->entity, 1, &video->pad);
+ /* Media Entity */
+
+ video_dev->entity.ops = &sun6i_video_media_ops;
+
+ /* Media Pad */
+
+ pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT;
+
+ ret = media_entity_pads_init(&video_dev->entity, 1, pad);
if (ret < 0)
return ret;
- mutex_init(&video->lock);
+ /* DMA queue */
INIT_LIST_HEAD(&video->dma_queue);
spin_lock_init(&video->dma_queue_lock);
video->sequence = 0;
- /* Setup default format */
- fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
- fmt.fmt.pix.pixelformat = supported_pixformats[0];
- fmt.fmt.pix.width = 1280;
- fmt.fmt.pix.height = 720;
- fmt.fmt.pix.field = V4L2_FIELD_NONE;
- sun6i_video_set_fmt(video, &fmt);
-
- /* Initialize videobuf2 queue */
- vidq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
- vidq->io_modes = VB2_MMAP | VB2_DMABUF;
- vidq->drv_priv = video;
- vidq->buf_struct_size = sizeof(struct sun6i_csi_buffer);
- vidq->ops = &sun6i_csi_vb2_ops;
- vidq->mem_ops = &vb2_dma_contig_memops;
- vidq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
- vidq->lock = &video->lock;
- /* Make sure non-dropped frame */
- vidq->min_buffers_needed = 3;
- vidq->dev = csi_dev->dev;
-
- ret = vb2_queue_init(vidq);
+ /* Queue */
+
+ mutex_init(&video->lock);
+
+ queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ queue->io_modes = VB2_MMAP | VB2_DMABUF;
+ queue->buf_struct_size = sizeof(struct sun6i_csi_buffer);
+ queue->ops = &sun6i_video_queue_ops;
+ queue->mem_ops = &vb2_dma_contig_memops;
+ queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ queue->lock = &video->lock;
+ queue->dev = csi_dev->dev;
+ queue->drv_priv = video;
+
+ /* Make sure non-dropped frame. */
+ queue->min_buffers_needed = 3;
+
+ ret = vb2_queue_init(queue);
if (ret) {
- v4l2_err(&v4l2->v4l2_dev, "vb2_queue_init failed: %d\n",
- ret);
- goto clean_entity;
+ v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret);
+ goto error_media_entity;
}
- /* Register video device */
- strscpy(vdev->name, name, sizeof(vdev->name));
- vdev->release = video_device_release_empty;
- vdev->fops = &sun6i_video_fops;
- vdev->ioctl_ops = &sun6i_video_ioctl_ops;
- vdev->vfl_type = VFL_TYPE_VIDEO;
- vdev->vfl_dir = VFL_DIR_RX;
- vdev->v4l2_dev = &v4l2->v4l2_dev;
- vdev->queue = vidq;
- vdev->lock = &video->lock;
- vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
- video_set_drvdata(vdev, video);
-
- ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ /* V4L2 Format */
+
+ format.type = queue->type;
+ pix_format->pixelformat = sun6i_video_formats[0];
+ pix_format->width = 1280;
+ pix_format->height = 720;
+ pix_format->field = V4L2_FIELD_NONE;
+
+ sun6i_video_format_set(video, &format);
+
+ /* Video Device */
+
+ strscpy(video_dev->name, SUN6I_CSI_NAME, sizeof(video_dev->name));
+ video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+ video_dev->vfl_dir = VFL_DIR_RX;
+ video_dev->release = video_device_release_empty;
+ video_dev->fops = &sun6i_video_fops;
+ video_dev->ioctl_ops = &sun6i_video_ioctl_ops;
+ video_dev->v4l2_dev = v4l2_dev;
+ video_dev->queue = queue;
+ video_dev->lock = &video->lock;
+
+ video_set_drvdata(video_dev, video);
+
+ ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
if (ret < 0) {
- v4l2_err(&v4l2->v4l2_dev,
- "video_register_device failed: %d\n", ret);
- goto clean_entity;
+ v4l2_err(v4l2_dev, "failed to register video device: %d\n",
+ ret);
+ goto error_media_entity;
}
+ v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
+ video_device_node_name(video_dev));
+
return 0;
-clean_entity:
- media_entity_cleanup(&video->vdev.entity);
+error_media_entity:
+ media_entity_cleanup(&video_dev->entity);
+
mutex_destroy(&video->lock);
+
return ret;
}
void sun6i_video_cleanup(struct sun6i_video *video)
{
- vb2_video_unregister_device(&video->vdev);
- media_entity_cleanup(&video->vdev.entity);
+ struct video_device *video_dev = &video->video_dev;
+
+ vb2_video_unregister_device(video_dev);
+ media_entity_cleanup(&video_dev->entity);
mutex_destroy(&video->lock);
}
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h
index 30e37ee0d07f..7864f062d05b 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h
@@ -15,22 +15,22 @@ struct sun6i_csi_device;
struct sun6i_video {
struct sun6i_csi_device *csi_dev;
- struct video_device vdev;
- struct media_pad pad;
- struct mutex lock;
+ struct video_device video_dev;
+ struct vb2_queue queue;
+ struct mutex lock; /* Queue lock. */
+ struct media_pad pad;
- struct vb2_queue vb2_vidq;
- spinlock_t dma_queue_lock;
struct list_head dma_queue;
+ spinlock_t dma_queue_lock; /* DMA queue lock. */
- unsigned int sequence;
- struct v4l2_format fmt;
+ struct v4l2_format format;
u32 mbus_code;
+ unsigned int sequence;
};
-int sun6i_video_init(struct sun6i_video *video,
- struct sun6i_csi_device *csi_dev, const char *name);
+int sun6i_video_setup(struct sun6i_video *video,
+ struct sun6i_csi_device *csi_dev);
void sun6i_video_cleanup(struct sun6i_video *video);
void sun6i_video_frame_done(struct sun6i_video *video);
--
2.34.1
Hi Paul,
Thank you for the patch.
On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> Some Allwinner platforms come with an Image Signal Processor, which
> supports various features in order to enhance and transform data
> received by image sensors into good-looking pictures. In most cases,
> the data is raw bayer, which gets internally converted to RGB and
> finally YUV, which is what the hardware produces.
>
> This driver supports ISPs that are similar to the A31 ISP, which was
> the first standalone ISP found in Allwinner platforms. Simpler ISP
> blocks were found in the A10 and A20, where they are tied to a CSI
> controller. Newer generations of Allwinner SoCs (starting with the
> H6, H616, etc) come with a new camera subsystem and revised ISP.
> Even though these previous and next-generation ISPs are somewhat
> similar to the A31 ISP, they have enough significant differences to
> be out of the scope of this driver.
>
> While the ISP supports many features, including 3A and many
> enhancement blocks, this implementation is limited to the following:
> - V3s (V3/S3) platform support;
> - Bayer media bus formats as input;
Greyscale formats would also be nice to have, if the hardware can
support that (it mostly just requires the ability to disable the CFA
interpolation).
> - Semi-planar YUV (NV12/NV21) as output;
Packed YUV would also be useful if the hardware supports it.
> - Debayering with per-component gain and offset configuration;
> - 2D noise filtering with configurable coefficients.
>
> Since many features are missing from the associated uAPI, the driver
> is aimed to integrate staging until all features are properly
> described.
>
> On the technical side, it uses the v4l2 and media controller APIs,
> with a video node for capture, a processor subdev and a video node
> for parameters submission. A specific uAPI structure and associated
> v4l2 meta format are used to configure parameters of the supported
> modules.
>
> One particular thing about the hardware is that configuration for
> module registers needs to be stored in a DMA buffer and gets copied
> to actual registers by the hardware at the next vsync, when instructed
> by a flag. This is handled by the "state" mechanism in the driver.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
> ---
> drivers/staging/media/sunxi/Kconfig | 1 +
> drivers/staging/media/sunxi/Makefile | 1 +
> drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
Could you add a TODO file to list the issues that need to be fixed for
the driver to move out of staging ? I'll already propose one entry:
- Add support in libcamera
This isn't required to merge the driver in staging as long as ABI
compatibility doesn't need to be preserved until the driver is moved out
of staging.
> 14 files changed, 3109 insertions(+)
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
[snip]
--
Regards,
Laurent Pinchart
The A31 MIPI CSI-2 controller is a dedicated MIPI CSI-2 bridge
found on Allwinner SoCs such as the A31 and V3/V3s.
It is a standalone block, connected to the CSI controller on one side
and to the MIPI D-PHY block on the other. It has a dedicated address
space, interrupt line and clock.
It is represented as a V4L2 subdev to the CSI controller and takes a
MIPI CSI-2 sensor as its own subdev, all using the fwnode graph and
media controller API.
Only 8-bit and 10-bit Bayer formats are currently supported.
While up to 4 internal channels to the CSI controller exist, only one
is currently supported by this implementation.
Signed-off-by: Paul Kocialkowski <[email protected]>
Acked-by: Maxime Ripard <[email protected]>
---
drivers/media/platform/sunxi/Kconfig | 1 +
drivers/media/platform/sunxi/Makefile | 1 +
.../platform/sunxi/sun6i-mipi-csi2/Kconfig | 12 +
.../platform/sunxi/sun6i-mipi-csi2/Makefile | 4 +
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c | 746 ++++++++++++++++++
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h | 52 ++
.../sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h | 82 ++
7 files changed, 898 insertions(+)
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h
diff --git a/drivers/media/platform/sunxi/Kconfig b/drivers/media/platform/sunxi/Kconfig
index 7151cc249afa..9684e07454ad 100644
--- a/drivers/media/platform/sunxi/Kconfig
+++ b/drivers/media/platform/sunxi/Kconfig
@@ -2,3 +2,4 @@
source "drivers/media/platform/sunxi/sun4i-csi/Kconfig"
source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
+source "drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig"
diff --git a/drivers/media/platform/sunxi/Makefile b/drivers/media/platform/sunxi/Makefile
index fc537c9f5ca9..887a7cae8fca 100644
--- a/drivers/media/platform/sunxi/Makefile
+++ b/drivers/media/platform/sunxi/Makefile
@@ -2,5 +2,6 @@
obj-y += sun4i-csi/
obj-y += sun6i-csi/
+obj-y += sun6i-mipi-csi2/
obj-y += sun8i-di/
obj-y += sun8i-rotate/
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
new file mode 100644
index 000000000000..47f1bb0779a8
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_SUN6I_MIPI_CSI2
+ tristate "Allwinner A31 MIPI CSI-2 Controller Driver"
+ depends on ARCH_SUNXI || COMPILE_TEST
+ depends on PM && COMMON_CLK && VIDEO_V4L2
+ select REGMAP_MMIO
+ select PHY_SUN6I_MIPI_DPHY
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select V4L2_FWNODE
+ help
+ Support for the Allwinner A31 MIPI CSI-2 Controller.
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
new file mode 100644
index 000000000000..14e4e03818b5
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+sun6i-mipi-csi2-y += sun6i_mipi_csi2.o
+
+obj-$(CONFIG_VIDEO_SUN6I_MIPI_CSI2) += sun6i-mipi-csi2.o
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
new file mode 100644
index 000000000000..55ebb512ec21
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
@@ -0,0 +1,746 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2020-2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#include "sun6i_mipi_csi2.h"
+#include "sun6i_mipi_csi2_reg.h"
+
+/* Format */
+
+static const struct sun6i_mipi_csi2_format sun6i_mipi_csi2_formats[] = {
+ {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW8,
+ .bpp = 8,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW8,
+ .bpp = 8,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW8,
+ .bpp = 8,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW8,
+ .bpp = 8,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW10,
+ .bpp = 10,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW10,
+ .bpp = 10,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW10,
+ .bpp = 10,
+ },
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .data_type = MIPI_CSI2_DATA_TYPE_RAW10,
+ .bpp = 10,
+ },
+};
+
+static const struct sun6i_mipi_csi2_format *
+sun6i_mipi_csi2_format_find(u32 mbus_code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(sun6i_mipi_csi2_formats); i++)
+ if (sun6i_mipi_csi2_formats[i].mbus_code == mbus_code)
+ return &sun6i_mipi_csi2_formats[i];
+
+ return NULL;
+}
+
+/* Controller */
+
+static void sun6i_mipi_csi2_enable(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct regmap *regmap = csi2_dev->regmap;
+
+ regmap_update_bits(regmap, SUN6I_MIPI_CSI2_CTL_REG,
+ SUN6I_MIPI_CSI2_CTL_EN, SUN6I_MIPI_CSI2_CTL_EN);
+}
+
+static void sun6i_mipi_csi2_disable(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct regmap *regmap = csi2_dev->regmap;
+
+ regmap_update_bits(regmap, SUN6I_MIPI_CSI2_CTL_REG,
+ SUN6I_MIPI_CSI2_CTL_EN, 0);
+}
+
+static void sun6i_mipi_csi2_configure(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct regmap *regmap = csi2_dev->regmap;
+ unsigned int lanes_count =
+ csi2_dev->bridge.endpoint.bus.mipi_csi2.num_data_lanes;
+ struct v4l2_mbus_framefmt *mbus_format = &csi2_dev->bridge.mbus_format;
+ const struct sun6i_mipi_csi2_format *format;
+ struct device *dev = csi2_dev->dev;
+ u32 version = 0;
+
+ format = sun6i_mipi_csi2_format_find(mbus_format->code);
+ if (WARN_ON(!format))
+ return;
+
+ /*
+ * The enable flow in the Allwinner BSP is a bit different: the enable
+ * and reset bits are set together before starting the CSI controller.
+ *
+ * In mainline we enable the CSI controller first (due to subdev logic).
+ * One reliable way to make this work is to deassert reset, configure
+ * registers and enable the controller when everything's ready.
+ *
+ * However, setting the version enable bit and removing it afterwards
+ * appears necessary for capture to work reliably, while replacing it
+ * with a delay doesn't do the trick.
+ */
+ regmap_write(regmap, SUN6I_MIPI_CSI2_CTL_REG,
+ SUN6I_MIPI_CSI2_CTL_RESET_N |
+ SUN6I_MIPI_CSI2_CTL_VERSION_EN |
+ SUN6I_MIPI_CSI2_CTL_UNPK_EN);
+
+ regmap_read(regmap, SUN6I_MIPI_CSI2_VERSION_REG, &version);
+
+ regmap_update_bits(regmap, SUN6I_MIPI_CSI2_CTL_REG,
+ SUN6I_MIPI_CSI2_CTL_VERSION_EN, 0);
+
+ dev_dbg(dev, "A31 MIPI CSI-2 version: %04x\n", version);
+
+ regmap_write(regmap, SUN6I_MIPI_CSI2_CFG_REG,
+ SUN6I_MIPI_CSI2_CFG_CHANNEL_MODE(1) |
+ SUN6I_MIPI_CSI2_CFG_LANE_COUNT(lanes_count));
+
+ /*
+ * Only a single virtual channel (index 0) is currently supported.
+ * While the registers do mention multiple physical channels being
+ * available (which can be configured to match a specific virtual
+ * channel or data type), it's unclear whether channels > 0 are actually
+ * connected and available and the reference source code only makes use
+ * of channel 0.
+ *
+ * Using extra channels would also require matching channels to be
+ * available on the CSI (and ISP) side, which is also unsure although
+ * some CSI implementations are said to support multiple channels for
+ * BT656 time-sharing.
+ *
+ * We still configure virtual channel numbers to ensure that virtual
+ * channel 0 only goes to channel 0.
+ */
+
+ regmap_write(regmap, SUN6I_MIPI_CSI2_VCDT_RX_REG,
+ SUN6I_MIPI_CSI2_VCDT_RX_CH_VC(3, 3) |
+ SUN6I_MIPI_CSI2_VCDT_RX_CH_VC(2, 2) |
+ SUN6I_MIPI_CSI2_VCDT_RX_CH_VC(1, 1) |
+ SUN6I_MIPI_CSI2_VCDT_RX_CH_VC(0, 0) |
+ SUN6I_MIPI_CSI2_VCDT_RX_CH_DT(0, format->data_type));
+
+ regmap_write(regmap, SUN6I_MIPI_CSI2_CH_INT_PD_REG,
+ SUN6I_MIPI_CSI2_CH_INT_PD_CLEAR);
+}
+
+/* V4L2 Subdev */
+
+static int sun6i_mipi_csi2_s_stream(struct v4l2_subdev *subdev, int on)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_subdev *source_subdev = csi2_dev->bridge.source_subdev;
+ union phy_configure_opts dphy_opts = { 0 };
+ struct phy_configure_opts_mipi_dphy *dphy_cfg = &dphy_opts.mipi_dphy;
+ struct v4l2_mbus_framefmt *mbus_format = &csi2_dev->bridge.mbus_format;
+ const struct sun6i_mipi_csi2_format *format;
+ struct phy *dphy = csi2_dev->dphy;
+ struct device *dev = csi2_dev->dev;
+ struct v4l2_ctrl *ctrl;
+ unsigned int lanes_count =
+ csi2_dev->bridge.endpoint.bus.mipi_csi2.num_data_lanes;
+ unsigned long pixel_rate;
+ /* Initialize to 0 to use both in disable label (ret != 0) and off. */
+ int ret = 0;
+
+ if (!source_subdev)
+ return -ENODEV;
+
+ if (!on) {
+ v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ goto disable;
+ }
+
+ /* Runtime PM */
+
+ ret = pm_runtime_resume_and_get(dev);
+ if (ret < 0)
+ return ret;
+
+ /* Sensor Pixel Rate */
+
+ ctrl = v4l2_ctrl_find(source_subdev->ctrl_handler, V4L2_CID_PIXEL_RATE);
+ if (!ctrl) {
+ dev_err(dev, "missing sensor pixel rate\n");
+ ret = -ENODEV;
+ goto error_pm;
+ }
+
+ pixel_rate = (unsigned long)v4l2_ctrl_g_ctrl_int64(ctrl);
+ if (!pixel_rate) {
+ dev_err(dev, "missing (zero) sensor pixel rate\n");
+ ret = -ENODEV;
+ goto error_pm;
+ }
+
+ /* D-PHY */
+
+ if (!lanes_count) {
+ dev_err(dev, "missing (zero) MIPI CSI-2 lanes count\n");
+ ret = -ENODEV;
+ goto error_pm;
+ }
+
+ format = sun6i_mipi_csi2_format_find(mbus_format->code);
+ if (WARN_ON(!format)) {
+ ret = -ENODEV;
+ goto error_pm;
+ }
+
+ phy_mipi_dphy_get_default_config(pixel_rate, format->bpp, lanes_count,
+ dphy_cfg);
+
+ /*
+ * Note that our hardware is using DDR, which is not taken in account by
+ * phy_mipi_dphy_get_default_config when calculating hs_clk_rate from
+ * the pixel rate, lanes count and bpp.
+ *
+ * The resulting clock rate is basically the symbol rate over the whole
+ * link. The actual clock rate is calculated with division by two since
+ * DDR samples both on rising and falling edges.
+ */
+
+ dev_dbg(dev, "A31 MIPI CSI-2 config:\n");
+ dev_dbg(dev, "%ld pixels/s, %u bits/pixel, %u lanes, %lu Hz clock\n",
+ pixel_rate, format->bpp, lanes_count,
+ dphy_cfg->hs_clk_rate / 2);
+
+ ret = phy_reset(dphy);
+ if (ret) {
+ dev_err(dev, "failed to reset MIPI D-PHY\n");
+ goto error_pm;
+ }
+
+ ret = phy_configure(dphy, &dphy_opts);
+ if (ret) {
+ dev_err(dev, "failed to configure MIPI D-PHY\n");
+ goto error_pm;
+ }
+
+ /* Controller */
+
+ sun6i_mipi_csi2_configure(csi2_dev);
+ sun6i_mipi_csi2_enable(csi2_dev);
+
+ /* D-PHY */
+
+ ret = phy_power_on(dphy);
+ if (ret) {
+ dev_err(dev, "failed to power on MIPI D-PHY\n");
+ goto error_pm;
+ }
+
+ /* Source */
+
+ ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
+ if (ret && ret != -ENOIOCTLCMD)
+ goto disable;
+
+ return 0;
+
+disable:
+ phy_power_off(dphy);
+ sun6i_mipi_csi2_disable(csi2_dev);
+
+error_pm:
+ pm_runtime_put(dev);
+
+ return ret;
+}
+
+static const struct v4l2_subdev_video_ops sun6i_mipi_csi2_video_ops = {
+ .s_stream = sun6i_mipi_csi2_s_stream,
+};
+
+static void
+sun6i_mipi_csi2_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format)
+{
+ if (!sun6i_mipi_csi2_format_find(mbus_format->code))
+ mbus_format->code = sun6i_mipi_csi2_formats[0].mbus_code;
+
+ mbus_format->field = V4L2_FIELD_NONE;
+ mbus_format->colorspace = V4L2_COLORSPACE_RAW;
+ mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT;
+ mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int sun6i_mipi_csi2_init_cfg(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state)
+{
+ unsigned int pad = SUN6I_MIPI_CSI2_PAD_SINK;
+ struct v4l2_mbus_framefmt *mbus_format =
+ v4l2_subdev_get_try_format(subdev, state, pad);
+
+ mbus_format->code = sun6i_mipi_csi2_formats[0].mbus_code;
+ mbus_format->width = 640;
+ mbus_format->height = 480;
+
+ sun6i_mipi_csi2_mbus_format_prepare(mbus_format);
+
+ return 0;
+}
+
+static int
+sun6i_mipi_csi2_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code_enum)
+{
+ if (code_enum->index >= ARRAY_SIZE(sun6i_mipi_csi2_formats))
+ return -EINVAL;
+
+ code_enum->code = sun6i_mipi_csi2_formats[code_enum->index].mbus_code;
+
+ return 0;
+}
+
+static int sun6i_mipi_csi2_get_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *mbus_format = *v4l2_subdev_get_try_format(subdev, state,
+ format->pad);
+ else
+ *mbus_format = csi2_dev->bridge.mbus_format;
+
+ return 0;
+}
+
+static int sun6i_mipi_csi2_set_fmt(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev = v4l2_get_subdevdata(subdev);
+ struct v4l2_mbus_framefmt *mbus_format = &format->format;
+
+ sun6i_mipi_csi2_mbus_format_prepare(mbus_format);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ *v4l2_subdev_get_try_format(subdev, state, format->pad) =
+ *mbus_format;
+ else
+ csi2_dev->bridge.mbus_format = *mbus_format;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops sun6i_mipi_csi2_pad_ops = {
+ .init_cfg = sun6i_mipi_csi2_init_cfg,
+ .enum_mbus_code = sun6i_mipi_csi2_enum_mbus_code,
+ .get_fmt = sun6i_mipi_csi2_get_fmt,
+ .set_fmt = sun6i_mipi_csi2_set_fmt,
+};
+
+static const struct v4l2_subdev_ops sun6i_mipi_csi2_subdev_ops = {
+ .video = &sun6i_mipi_csi2_video_ops,
+ .pad = &sun6i_mipi_csi2_pad_ops,
+};
+
+/* Media Entity */
+
+static const struct media_entity_operations sun6i_mipi_csi2_entity_ops = {
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+/* V4L2 Async */
+
+static int
+sun6i_mipi_csi2_notifier_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *remote_subdev,
+ struct v4l2_async_subdev *async_subdev)
+{
+ struct v4l2_subdev *subdev = notifier->sd;
+ struct sun6i_mipi_csi2_device *csi2_dev =
+ container_of(notifier, struct sun6i_mipi_csi2_device,
+ bridge.notifier);
+ struct media_entity *sink_entity = &subdev->entity;
+ struct media_entity *source_entity = &remote_subdev->entity;
+ struct device *dev = csi2_dev->dev;
+ int sink_pad_index = 0;
+ int source_pad_index;
+ int ret;
+
+ ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (ret < 0) {
+ dev_err(dev, "missing source pad in external entity %s\n",
+ source_entity->name);
+ return -EINVAL;
+ }
+
+ source_pad_index = ret;
+
+ dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
+ source_pad_index, sink_entity->name, sink_pad_index);
+
+ ret = media_create_pad_link(source_entity, source_pad_index,
+ sink_entity, sink_pad_index,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret) {
+ dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
+ source_entity->name, source_pad_index,
+ sink_entity->name, sink_pad_index);
+ return ret;
+ }
+
+ csi2_dev->bridge.source_subdev = remote_subdev;
+
+ return 0;
+}
+
+static const struct v4l2_async_notifier_operations
+sun6i_mipi_csi2_notifier_ops = {
+ .bound = sun6i_mipi_csi2_notifier_bound,
+};
+
+/* Bridge */
+
+static int
+sun6i_mipi_csi2_bridge_source_setup(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct v4l2_async_notifier *notifier = &csi2_dev->bridge.notifier;
+ struct v4l2_fwnode_endpoint *endpoint = &csi2_dev->bridge.endpoint;
+ struct v4l2_async_subdev *subdev_async;
+ struct fwnode_handle *handle;
+ struct device *dev = csi2_dev->dev;
+ int ret;
+
+ handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0,
+ FWNODE_GRAPH_ENDPOINT_NEXT);
+ if (!handle)
+ return -ENODEV;
+
+ endpoint->bus_type = V4L2_MBUS_CSI2_DPHY;
+
+ ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
+ if (ret)
+ goto complete;
+
+ subdev_async = v4l2_async_nf_add_fwnode_remote(notifier, handle,
+ struct v4l2_async_subdev);
+ if (IS_ERR(subdev_async))
+ ret = PTR_ERR(subdev_async);
+
+complete:
+ fwnode_handle_put(handle);
+
+ return ret;
+}
+
+static int sun6i_mipi_csi2_bridge_setup(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct v4l2_subdev *subdev = &csi2_dev->bridge.subdev;
+ struct v4l2_async_notifier *notifier = &csi2_dev->bridge.notifier;
+ struct media_pad *pads = csi2_dev->bridge.pads;
+ struct device *dev = csi2_dev->dev;
+ int ret;
+
+ /* V4L2 Subdev */
+
+ v4l2_subdev_init(subdev, &sun6i_mipi_csi2_subdev_ops);
+ strscpy(subdev->name, SUN6I_MIPI_CSI2_NAME, sizeof(subdev->name));
+ subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ subdev->owner = THIS_MODULE;
+ subdev->dev = dev;
+
+ v4l2_set_subdevdata(subdev, csi2_dev);
+
+ /* Media Entity */
+
+ subdev->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+ subdev->entity.ops = &sun6i_mipi_csi2_entity_ops;
+
+ /* Media Pads */
+
+ pads[SUN6I_MIPI_CSI2_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ pads[SUN6I_MIPI_CSI2_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&subdev->entity, SUN6I_MIPI_CSI2_PAD_COUNT,
+ pads);
+ if (ret)
+ return ret;
+
+ /* V4L2 Async */
+
+ v4l2_async_nf_init(notifier);
+ notifier->ops = &sun6i_mipi_csi2_notifier_ops;
+
+ ret = sun6i_mipi_csi2_bridge_source_setup(csi2_dev);
+ if (ret)
+ goto error_v4l2_notifier_cleanup;
+
+ ret = v4l2_async_subdev_nf_register(subdev, notifier);
+ if (ret < 0)
+ goto error_v4l2_notifier_cleanup;
+
+ /* V4L2 Subdev */
+
+ ret = v4l2_async_register_subdev(subdev);
+ if (ret < 0)
+ goto error_v4l2_notifier_unregister;
+
+ return 0;
+
+error_v4l2_notifier_unregister:
+ v4l2_async_nf_unregister(notifier);
+
+error_v4l2_notifier_cleanup:
+ v4l2_async_nf_cleanup(notifier);
+
+ media_entity_cleanup(&subdev->entity);
+
+ return ret;
+}
+
+static void
+sun6i_mipi_csi2_bridge_cleanup(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ struct v4l2_subdev *subdev = &csi2_dev->bridge.subdev;
+ struct v4l2_async_notifier *notifier = &csi2_dev->bridge.notifier;
+
+ v4l2_async_unregister_subdev(subdev);
+ v4l2_async_nf_unregister(notifier);
+ v4l2_async_nf_cleanup(notifier);
+ media_entity_cleanup(&subdev->entity);
+}
+
+/* Platform */
+
+static int sun6i_mipi_csi2_suspend(struct device *dev)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(csi2_dev->clk_mod);
+ clk_disable_unprepare(csi2_dev->clk_bus);
+ reset_control_assert(csi2_dev->reset);
+
+ return 0;
+}
+
+static int sun6i_mipi_csi2_resume(struct device *dev)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev = dev_get_drvdata(dev);
+ int ret;
+
+ ret = reset_control_deassert(csi2_dev->reset);
+ if (ret) {
+ dev_err(dev, "failed to deassert reset\n");
+ return ret;
+ }
+
+ ret = clk_prepare_enable(csi2_dev->clk_bus);
+ if (ret) {
+ dev_err(dev, "failed to enable bus clock\n");
+ goto error_reset;
+ }
+
+ ret = clk_prepare_enable(csi2_dev->clk_mod);
+ if (ret) {
+ dev_err(dev, "failed to enable module clock\n");
+ goto error_clk_bus;
+ }
+
+ return 0;
+
+error_clk_bus:
+ clk_disable_unprepare(csi2_dev->clk_bus);
+
+error_reset:
+ reset_control_assert(csi2_dev->reset);
+
+ return ret;
+}
+
+static const struct dev_pm_ops sun6i_mipi_csi2_pm_ops = {
+ SET_RUNTIME_PM_OPS(sun6i_mipi_csi2_suspend, sun6i_mipi_csi2_resume,
+ NULL)
+};
+
+static const struct regmap_config sun6i_mipi_csi2_regmap_config = {
+ .reg_bits = 32,
+ .reg_stride = 4,
+ .val_bits = 32,
+ .max_register = 0x400,
+};
+
+static int
+sun6i_mipi_csi2_resources_setup(struct sun6i_mipi_csi2_device *csi2_dev,
+ struct platform_device *platform_dev)
+{
+ struct device *dev = csi2_dev->dev;
+ struct resource *res;
+ void __iomem *io_base;
+ int ret;
+
+ /* Registers */
+
+ res = platform_get_resource(platform_dev, IORESOURCE_MEM, 0);
+ io_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(io_base))
+ return PTR_ERR(io_base);
+
+ csi2_dev->regmap = devm_regmap_init_mmio(dev, io_base,
+ &sun6i_mipi_csi2_regmap_config);
+ if (IS_ERR(csi2_dev->regmap)) {
+ dev_err(dev, "failed to init register map\n");
+ return PTR_ERR(csi2_dev->regmap);
+ }
+
+ /* Clocks */
+
+ csi2_dev->clk_bus = devm_clk_get(dev, "bus");
+ if (IS_ERR(csi2_dev->clk_bus)) {
+ dev_err(dev, "failed to acquire bus clock\n");
+ return PTR_ERR(csi2_dev->clk_bus);
+ }
+
+ csi2_dev->clk_mod = devm_clk_get(dev, "mod");
+ if (IS_ERR(csi2_dev->clk_mod)) {
+ dev_err(dev, "failed to acquire mod clock\n");
+ return PTR_ERR(csi2_dev->clk_mod);
+ }
+
+ ret = clk_set_rate_exclusive(csi2_dev->clk_mod, 297000000);
+ if (ret) {
+ dev_err(dev, "failed to set mod clock rate\n");
+ return ret;
+ }
+
+ /* Reset */
+
+ csi2_dev->reset = devm_reset_control_get_shared(dev, NULL);
+ if (IS_ERR(csi2_dev->reset)) {
+ dev_err(dev, "failed to get reset controller\n");
+ return PTR_ERR(csi2_dev->reset);
+ }
+
+ /* D-PHY */
+
+ csi2_dev->dphy = devm_phy_get(dev, "dphy");
+ if (IS_ERR(csi2_dev->dphy)) {
+ dev_err(dev, "failed to get MIPI D-PHY\n");
+ return PTR_ERR(csi2_dev->dphy);
+ }
+
+ ret = phy_init(csi2_dev->dphy);
+ if (ret) {
+ dev_err(dev, "failed to initialize MIPI D-PHY\n");
+ return ret;
+ }
+
+ /* Runtime PM */
+
+ pm_runtime_enable(dev);
+ pm_runtime_set_suspended(dev);
+
+ return 0;
+}
+
+static void
+sun6i_mipi_csi2_resources_cleanup(struct sun6i_mipi_csi2_device *csi2_dev)
+{
+ pm_runtime_disable(csi2_dev->dev);
+ phy_exit(csi2_dev->dphy);
+ clk_rate_exclusive_put(csi2_dev->clk_mod);
+}
+
+static int sun6i_mipi_csi2_probe(struct platform_device *platform_dev)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev;
+ struct device *dev = &platform_dev->dev;
+ int ret;
+
+ csi2_dev = devm_kzalloc(dev, sizeof(*csi2_dev), GFP_KERNEL);
+ if (!csi2_dev)
+ return -ENOMEM;
+
+ csi2_dev->dev = dev;
+ platform_set_drvdata(platform_dev, csi2_dev);
+
+ ret = sun6i_mipi_csi2_resources_setup(csi2_dev, platform_dev);
+ if (ret)
+ return ret;
+
+ ret = sun6i_mipi_csi2_bridge_setup(csi2_dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int sun6i_mipi_csi2_remove(struct platform_device *platform_dev)
+{
+ struct sun6i_mipi_csi2_device *csi2_dev =
+ platform_get_drvdata(platform_dev);
+
+ sun6i_mipi_csi2_bridge_cleanup(csi2_dev);
+ sun6i_mipi_csi2_resources_cleanup(csi2_dev);
+
+ return 0;
+}
+
+static const struct of_device_id sun6i_mipi_csi2_of_match[] = {
+ { .compatible = "allwinner,sun6i-a31-mipi-csi2" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, sun6i_mipi_csi2_of_match);
+
+static struct platform_driver sun6i_mipi_csi2_platform_driver = {
+ .probe = sun6i_mipi_csi2_probe,
+ .remove = sun6i_mipi_csi2_remove,
+ .driver = {
+ .name = SUN6I_MIPI_CSI2_NAME,
+ .of_match_table = of_match_ptr(sun6i_mipi_csi2_of_match),
+ .pm = &sun6i_mipi_csi2_pm_ops,
+ },
+};
+module_platform_driver(sun6i_mipi_csi2_platform_driver);
+
+MODULE_DESCRIPTION("Allwinner A31 MIPI CSI-2 Controller Driver");
+MODULE_AUTHOR("Paul Kocialkowski <[email protected]>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
new file mode 100644
index 000000000000..25a8a0d0dfad
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2020-2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_MIPI_CSI2_H_
+#define _SUN6I_MIPI_CSI2_H_
+
+#include <linux/phy/phy.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#define SUN6I_MIPI_CSI2_NAME "sun6i-mipi-csi2"
+
+enum sun6i_mipi_csi2_pad {
+ SUN6I_MIPI_CSI2_PAD_SINK = 0,
+ SUN6I_MIPI_CSI2_PAD_SOURCE = 1,
+ SUN6I_MIPI_CSI2_PAD_COUNT = 2,
+};
+
+struct sun6i_mipi_csi2_format {
+ u32 mbus_code;
+ u8 data_type;
+ u32 bpp;
+};
+
+struct sun6i_mipi_csi2_bridge {
+ struct v4l2_subdev subdev;
+ struct media_pad pads[SUN6I_MIPI_CSI2_PAD_COUNT];
+ struct v4l2_fwnode_endpoint endpoint;
+ struct v4l2_async_notifier notifier;
+ struct v4l2_mbus_framefmt mbus_format;
+
+ struct v4l2_subdev *source_subdev;
+};
+
+struct sun6i_mipi_csi2_device {
+ struct device *dev;
+
+ struct regmap *regmap;
+ struct clk *clk_bus;
+ struct clk *clk_mod;
+ struct reset_control *reset;
+ struct phy *dphy;
+
+ struct sun6i_mipi_csi2_bridge bridge;
+};
+
+#endif
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h
new file mode 100644
index 000000000000..aafda934f5ef
--- /dev/null
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2_reg.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2020-2021 Bootlin
+ * Author: Paul Kocialkowski <[email protected]>
+ */
+
+#ifndef _SUN6I_MIPI_CSI2_REG_H_
+#define _SUN6I_MIPI_CSI2_REG_H_
+
+#define SUN6I_MIPI_CSI2_CTL_REG 0x0
+#define SUN6I_MIPI_CSI2_CTL_RESET_N BIT(31)
+#define SUN6I_MIPI_CSI2_CTL_VERSION_EN BIT(30)
+#define SUN6I_MIPI_CSI2_CTL_UNPK_EN BIT(1)
+#define SUN6I_MIPI_CSI2_CTL_EN BIT(0)
+
+#define SUN6I_MIPI_CSI2_CFG_REG 0x4
+#define SUN6I_MIPI_CSI2_CFG_CHANNEL_MODE(v) ((((v) - 1) << 8) & \
+ GENMASK(9, 8))
+#define SUN6I_MIPI_CSI2_CFG_LANE_COUNT(v) (((v) - 1) & GENMASK(1, 0))
+
+#define SUN6I_MIPI_CSI2_VCDT_RX_REG 0x8
+#define SUN6I_MIPI_CSI2_VCDT_RX_CH_VC(ch, vc) (((vc) & GENMASK(1, 0)) << \
+ ((ch) * 8 + 6))
+#define SUN6I_MIPI_CSI2_VCDT_RX_CH_DT(ch, t) (((t) & GENMASK(5, 0)) << \
+ ((ch) * 8))
+#define SUN6I_MIPI_CSI2_RX_PKT_NUM_REG 0xc
+
+#define SUN6I_MIPI_CSI2_VERSION_REG 0x3c
+
+#define SUN6I_MIPI_CSI2_CH_CFG_REG 0x40
+#define SUN6I_MIPI_CSI2_CH_INT_EN_REG 0x50
+#define SUN6I_MIPI_CSI2_CH_INT_EN_EOT_ERR BIT(29)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_CHKSUM_ERR BIT(28)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_ECC_WRN BIT(27)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_ECC_ERR BIT(26)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_LINE_SYNC_ERR BIT(25)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_FRAME_SYNC_ERR BIT(24)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_EMB_DATA BIT(18)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_PF BIT(17)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_PH_UPDATE BIT(16)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_LINE_START_SYNC BIT(11)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_LINE_END_SYNC BIT(10)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_FRAME_START_SYNC BIT(9)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_FRAME_END_SYNC BIT(8)
+#define SUN6I_MIPI_CSI2_CH_INT_EN_FIFO_OVER BIT(0)
+
+#define SUN6I_MIPI_CSI2_CH_INT_PD_REG 0x58
+#define SUN6I_MIPI_CSI2_CH_INT_PD_CLEAR 0xff
+#define SUN6I_MIPI_CSI2_CH_INT_PD_EOT_ERR BIT(29)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_CHKSUM_ERR BIT(28)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_ECC_WRN BIT(27)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_ECC_ERR BIT(26)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_LINE_SYNC_ERR BIT(25)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_FRAME_SYNC_ERR BIT(24)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_EMB_DATA BIT(18)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_PF BIT(17)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_PH_UPDATE BIT(16)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_LINE_START_SYNC BIT(11)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_LINE_END_SYNC BIT(10)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_FRAME_START_SYNC BIT(9)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_FRAME_END_SYNC BIT(8)
+#define SUN6I_MIPI_CSI2_CH_INT_PD_FIFO_OVER BIT(0)
+
+#define SUN6I_MIPI_CSI2_CH_DT_TRIGGER_REG 0x60
+#define SUN6I_MIPI_CSI2_CH_CUR_PH_REG 0x70
+#define SUN6I_MIPI_CSI2_CH_ECC_REG 0x74
+#define SUN6I_MIPI_CSI2_CH_CKS_REG 0x78
+#define SUN6I_MIPI_CSI2_CH_FRAME_NUM_REG 0x7c
+#define SUN6I_MIPI_CSI2_CH_LINE_NUM_REG 0x80
+
+#define SUN6I_MIPI_CSI2_CH_OFFSET 0x100
+
+#define SUN6I_MIPI_CSI2_CH_REG(reg, ch) \
+ (SUN6I_MIPI_CSI2_CH_OFFSET * (ch) + (reg))
+
+enum mipi_csi2_data_type {
+ MIPI_CSI2_DATA_TYPE_RAW8 = 0x2a,
+ MIPI_CSI2_DATA_TYPE_RAW10 = 0x2b,
+ MIPI_CSI2_DATA_TYPE_RAW12 = 0x2c,
+};
+
+#endif
--
2.34.1
The A31 CSI controller supports two distinct input interfaces:
parallel and an external MIPI CSI-2 bridge. The parallel interface
is often connected to a set of hardware pins while the MIPI CSI-2
bridge is an internal FIFO-ish link. As a result, these two inputs
are distinguished as two different ports.
Note that only one of the two may be present on a controller instance.
For example, the V3s has one controller dedicated to MIPI-CSI2 and one
dedicated to parallel.
Update the binding with an explicit ports node that holds two distinct
port nodes: one for parallel input and one for MIPI CSI-2.
This is backward-compatible with the single-port approach that was
previously taken for representing the parallel interface port, which
stays enumerated as fwnode port 0.
Note that additional ports may be added in the future, especially to
support feeding the CSI controller's output to the ISP.
Signed-off-by: Paul Kocialkowski <[email protected]>
Reviewed-by: Rob Herring <[email protected]>
Acked-by: Maxime Ripard <[email protected]>
---
.../media/allwinner,sun6i-a31-csi.yaml | 60 +++++++++++++++----
1 file changed, 47 insertions(+), 13 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
index 8b568072a069..3cc61866ea89 100644
--- a/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
+++ b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml
@@ -61,6 +61,34 @@ properties:
additionalProperties: false
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: "#/properties/port"
+ unevaluatedProperties: false
+
+ port@1:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ description: MIPI CSI-2 bridge input port
+
+ properties:
+ reg:
+ const: 1
+
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ additionalProperties: false
+
+ anyOf:
+ - required:
+ - port@0
+ - required:
+ - port@1
+
required:
- compatible
- reg
@@ -89,19 +117,25 @@ examples:
"ram";
resets = <&ccu RST_BUS_CSI>;
- port {
- /* Parallel bus endpoint */
- csi1_ep: endpoint {
- remote-endpoint = <&adv7611_ep>;
- bus-width = <16>;
-
- /*
- * If hsync-active/vsync-active are missing,
- * embedded BT.656 sync is used.
- */
- hsync-active = <0>; /* Active low */
- vsync-active = <0>; /* Active low */
- pclk-sample = <1>; /* Rising */
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ /* Parallel bus endpoint */
+ csi1_ep: endpoint {
+ remote-endpoint = <&adv7611_ep>;
+ bus-width = <16>;
+
+ /*
+ * If hsync-active/vsync-active are missing,
+ * embedded BT.656 sync is used.
+ */
+ hsync-active = <0>; /* Active low */
+ vsync-active = <0>; /* Active low */
+ pclk-sample = <1>; /* Rising */
+ };
};
};
};
--
2.34.1
Hi Jernej,
On Mon 07 Feb 22, 09:50, Jernej Škrabec wrote:
> Hi Paul,
>
> Dne ponedeljek, 07. februar 2022 ob 09:43:43 CET je Paul Kocialkowski
> napisal(a):
> > Hi,
> >
> > On Sat 05 Feb 22, 14:14, Samuel Holland wrote:
> > > On 2/5/22 12:53 PM, Paul Kocialkowski wrote:
> > > > Since the V3s uses the internal mbus, document its compatible.
> > > >
> > > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > > ---
> > > >
> > > > .../devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml | 1 +
> > > > 1 file changed, 1 insertion(+)
> > > >
> > > > diff --git
> > > > a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > > yaml
> > > > b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > > yaml index 29c9961ee2d8..b67bf9261a6a 100644
> > > > ---
> > > > a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > > yaml +++
> > > > b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.
> > > > yaml> >
> > > > @@ -31,6 +31,7 @@ properties:
> > > > - allwinner,sun5i-a13-mbus
> > > > - allwinner,sun8i-h3-mbus
> > > > - allwinner,sun8i-r40-mbus
> > > >
> > > > + - allwinner,sun8i-v3s-mbus
> > >
> > > Please enable the expanded binding added in commit 245578ba9f03
> > > ("dt-bindings: arm: sunxi: Expand MBUS binding")[1] by adding the new
> > > compatible to the "if" block lower in the file. That way we can add V3S
> > > devfreq support in the future without changing that binding.
> >
> > I had missed that new driver but surely I will expand the updated binding.
> >
> > By the way do you have an explanation about the cell index given to the
> > interconnects (after &mbus)?
>
> This is mbus channel. You can find appropriate one checking DRAM driver in U-
> Boot, where mbus is configured.
Thanks, that's exactly what I was looking for! Looks like in my case
MBUS_PORT_CSI will be used both for CSI and ISP.
For the record it's also defined in the BSP kernel at:
include/linux/sunxi_mbus.h
Thanks,
Paul
> Best regards,
> Jernej
>
> >
> > Paul
> >
> > > Regards,
> > > Samuel
> > >
> > > [1]: https://git.kernel.org/torvalds/c/245578ba9f03
> > >
> > > > - allwinner,sun50i-a64-mbus
> > > >
> > > > reg:
>
>
>
>
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
On Sat, Feb 05, 2022 at 07:53:47PM +0100, Paul Kocialkowski wrote:
> Some code cleanups, renames, variable lowerings and moving things around for
> better organization. No functional change intended.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
Reviewed-by: Maxime Ripard <[email protected]>
Maxime
On Sat, Feb 05, 2022 at 07:53:52PM +0100, Paul Kocialkowski wrote:
> The buffer structure is a top-level definition, put it in the main header
> to keep things tidy. No functional change intended.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
Reviewed-by: Maxime Ripard <[email protected]>
Maxime
Hi,
On Sat, Feb 05, 2022 at 07:53:42PM +0100, Paul Kocialkowski wrote:
> Since the bus clock alone is not enough to get access to the registers,
> don't pass it to regmap and manage it instead just like the other
> clocks.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
I'm not really sure about this one.
Sure, there's more resources involved than the clock to get access to
the registers (like reset), but you can't have a functional device
without the reset line deasserted.
You can however have a functional device without the bus clock enabled,
so it just seems much better to let regmap manage it.
Unless there's a drawback to it of course, but it should be in your
commit log if so.
Maxime
On Sat, Feb 05, 2022 at 07:53:53PM +0100, Paul Kocialkowski wrote:
> Introduce a bridge v4l2 subdev to prepare for separation between the
> processing part (bridge) and the dma engine, which is required to
> properly support ths isp workflow later on.
>
> Currently the bridge just manages fwnode mapping to media pads,
> using an async notifier (which was previously in the main code).
> The s_stream video op just forwards to the connected v4l2 subdev
> (sensor or MIPI CSI-2 bridge).
>
> The video capture device is now registered after the bridge and
> attaches to it with a media link.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
There's a bunch of checkpatch --strict warnings that need to be fixed
Maxime
On Fri, Feb 11, 2022 at 04:43:51PM +0100, Paul Kocialkowski wrote:
> Hi,
>
> On Wed 09 Feb 22, 10:24, Maxime Ripard wrote:
> > On Sat, Feb 05, 2022 at 07:53:53PM +0100, Paul Kocialkowski wrote:
> > > Introduce a bridge v4l2 subdev to prepare for separation between the
> > > processing part (bridge) and the dma engine, which is required to
> > > properly support ths isp workflow later on.
> > >
> > > Currently the bridge just manages fwnode mapping to media pads,
> > > using an async notifier (which was previously in the main code).
> > > The s_stream video op just forwards to the connected v4l2 subdev
> > > (sensor or MIPI CSI-2 bridge).
> > >
> > > The video capture device is now registered after the bridge and
> > > attaches to it with a media link.
> > >
> > > Signed-off-by: Paul Kocialkowski <[email protected]>
> >
> > There's a bunch of checkpatch --strict warnings that need to be fixed
>
> Yes so it turns out these are adaptations to the existing video code
> which has these warnings already merged. They are cleaned up later on
> in a dedicated commit, but since it's not the topic of this change
> (which is a logic change) I kept the code as it is.
>
> What do you think?
+ async_subdev = v4l2_async_nf_add_fwnode_remote(notifier, handle,
+ struct v4l2_async_subdev);
CHECK: Alignment should match open parenthesis
This one at least is introduced by your patch
Maxime
Hi,
On Wed 09 Feb 22, 10:24, Maxime Ripard wrote:
> On Sat, Feb 05, 2022 at 07:53:53PM +0100, Paul Kocialkowski wrote:
> > Introduce a bridge v4l2 subdev to prepare for separation between the
> > processing part (bridge) and the dma engine, which is required to
> > properly support ths isp workflow later on.
> >
> > Currently the bridge just manages fwnode mapping to media pads,
> > using an async notifier (which was previously in the main code).
> > The s_stream video op just forwards to the connected v4l2 subdev
> > (sensor or MIPI CSI-2 bridge).
> >
> > The video capture device is now registered after the bridge and
> > attaches to it with a media link.
> >
> > Signed-off-by: Paul Kocialkowski <[email protected]>
>
> There's a bunch of checkpatch --strict warnings that need to be fixed
Yes so it turns out these are adaptations to the existing video code
which has these warnings already merged. They are cleaned up later on
in a dedicated commit, but since it's not the topic of this change
(which is a logic change) I kept the code as it is.
What do you think?
Cheers,
Paul
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul,
On Sat, Feb 05, 2022 at 07:53:53PM +0100, Paul Kocialkowski wrote:
> Introduce a bridge v4l2 subdev to prepare for separation between the
> processing part (bridge) and the dma engine, which is required to
> properly support ths isp workflow later on.
>
> Currently the bridge just manages fwnode mapping to media pads,
> using an async notifier (which was previously in the main code).
> The s_stream video op just forwards to the connected v4l2 subdev
> (sensor or MIPI CSI-2 bridge).
>
> The video capture device is now registered after the bridge and
> attaches to it with a media link.
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
> ---
> .../media/platform/sunxi/sun6i-csi/Makefile | 2 +-
> .../platform/sunxi/sun6i-csi/sun6i_csi.c | 156 +-----
> .../platform/sunxi/sun6i-csi/sun6i_csi.h | 12 +-
> .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 473 ++++++++++++++++++
> .../sunxi/sun6i-csi/sun6i_csi_bridge.h | 44 ++
> .../platform/sunxi/sun6i-csi/sun6i_video.c | 19 +
> 6 files changed, 571 insertions(+), 135 deletions(-)
> create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
> create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
>
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile
> index e7e315347804..7a699580a641 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/Makefile
> +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile
> @@ -1,4 +1,4 @@
> # SPDX-License-Identifier: GPL-2.0-only
> -sun6i-csi-y += sun6i_video.o sun6i_csi.o
> +sun6i-csi-y += sun6i_video.o sun6i_csi.o sun6i_csi_bridge.o
>
> obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> index c8fe31cc38b5..a1847ae3e88e 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> @@ -34,16 +34,17 @@
> bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev,
> u32 pixformat, u32 mbus_code)
> {
> - struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
> + struct v4l2_fwnode_endpoint *endpoint =
> + &csi_dev->bridge.source->endpoint;
>
> /*
> * Some video receivers have the ability to be compatible with
> * 8bit and 16bit bus width.
> * Identify the media bus format from device tree.
> */
> - if ((v4l2->v4l2_ep.bus_type == V4L2_MBUS_PARALLEL
> - || v4l2->v4l2_ep.bus_type == V4L2_MBUS_BT656)
> - && v4l2->v4l2_ep.bus.parallel.bus_width == 16) {
> + if ((endpoint->bus_type == V4L2_MBUS_PARALLEL
> + || endpoint->bus_type == V4L2_MBUS_BT656)
> + && endpoint->bus.parallel.bus_width == 16) {
> switch (pixformat) {
> case V4L2_PIX_FMT_NV12_16L16:
> case V4L2_PIX_FMT_NV12:
> @@ -328,7 +329,8 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev,
>
> static void sun6i_csi_setup_bus(struct sun6i_csi_device *csi_dev)
> {
> - struct v4l2_fwnode_endpoint *endpoint = &csi_dev->v4l2.v4l2_ep;
> + struct v4l2_fwnode_endpoint *endpoint =
> + &csi_dev->bridge.source->endpoint;
> struct sun6i_csi_config *config = &csi_dev->config;
> unsigned char bus_width;
> u32 flags;
> @@ -583,95 +585,11 @@ static const struct media_device_ops sun6i_csi_media_ops = {
>
> /* V4L2 */
>
> -static int sun6i_csi_link_entity(struct sun6i_csi_device *csi_dev,
> - struct media_entity *entity,
> - struct fwnode_handle *fwnode)
> +int sun6i_csi_v4l2_complete(struct sun6i_csi_device *csi_dev)
> {
> - struct media_entity *sink;
> - struct media_pad *sink_pad;
> - int src_pad_index;
> - int ret;
> -
> - ret = media_entity_get_fwnode_pad(entity, fwnode, MEDIA_PAD_FL_SOURCE);
> - if (ret < 0) {
> - dev_err(csi_dev->dev,
> - "%s: no source pad in external entity %s\n", __func__,
> - entity->name);
> - return -EINVAL;
> - }
> -
> - src_pad_index = ret;
> -
> - sink = &csi_dev->video.video_dev.entity;
> - sink_pad = &csi_dev->video.pad;
> -
> - dev_dbg(csi_dev->dev, "creating %s:%u -> %s:%u link\n",
> - entity->name, src_pad_index, sink->name, sink_pad->index);
> - ret = media_create_pad_link(entity, src_pad_index, sink,
> - sink_pad->index,
> - MEDIA_LNK_FL_ENABLED |
> - MEDIA_LNK_FL_IMMUTABLE);
> - if (ret < 0) {
> - dev_err(csi_dev->dev, "failed to create %s:%u -> %s:%u link\n",
> - entity->name, src_pad_index,
> - sink->name, sink_pad->index);
> - return ret;
> - }
> -
> - return 0;
> -}
> -
> -static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier)
> -{
> - struct sun6i_csi_device *csi_dev =
> - container_of(notifier, struct sun6i_csi_device,
> - v4l2.notifier);
> - struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
> - struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
> - struct v4l2_subdev *sd;
> - int ret;
> -
> - dev_dbg(csi_dev->dev, "notify complete, all subdevs registered\n");
> -
> - sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list);
> - if (!sd)
> - return -EINVAL;
> -
> - ret = sun6i_csi_link_entity(csi_dev, &sd->entity, sd->fwnode);
> - if (ret < 0)
> - return ret;
> -
> - ret = v4l2_device_register_subdev_nodes(v4l2_dev);
> - if (ret < 0)
> - return ret;
> -
> - return 0;
> -}
> -
> -static const struct v4l2_async_notifier_operations sun6i_csi_async_ops = {
> - .complete = sun6i_subdev_notify_complete,
> -};
> -
> -static int sun6i_csi_fwnode_parse(struct device *dev,
> - struct v4l2_fwnode_endpoint *vep,
> - struct v4l2_async_subdev *asd)
> -{
> - struct sun6i_csi_device *csi_dev = dev_get_drvdata(dev);
> + struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
>
> - if (vep->base.port || vep->base.id) {
> - dev_warn(dev, "Only support a single port with one endpoint\n");
> - return -ENOTCONN;
> - }
> -
> - switch (vep->bus_type) {
> - case V4L2_MBUS_PARALLEL:
> - case V4L2_MBUS_BT656:
> - csi_dev->v4l2.v4l2_ep = *vep;
> - return 0;
> - default:
> - dev_err(dev, "Unsupported media bus type\n");
> - return -ENOTCONN;
> - }
> + return v4l2_device_register_subdev_nodes(v4l2_dev);
> }
>
> static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
> @@ -679,7 +597,6 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
> struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
> struct media_device *media_dev = &v4l2->media_dev;
> struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev;
> - struct v4l2_async_notifier *notifier = &v4l2->notifier;
> struct device *dev = csi_dev->dev;
> int ret;
>
> @@ -720,42 +637,8 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev)
> goto error_v4l2_ctrl;
> }
>
> - /* Video */
> -
> - ret = sun6i_video_setup(csi_dev);
> - if (ret)
> - goto error_v4l2_device;
> -
> - /* V4L2 Async */
> -
> - v4l2_async_nf_init(notifier);
> - notifier->ops = &sun6i_csi_async_ops;
> -
> - ret = v4l2_async_nf_parse_fwnode_endpoints(dev, notifier,
> - sizeof(struct
> - v4l2_async_subdev),
> - sun6i_csi_fwnode_parse);
> - if (ret)
> - goto error_video;
> -
> - ret = v4l2_async_nf_register(v4l2_dev, notifier);
> - if (ret) {
> - dev_err(dev, "failed to register v4l2 async notifier: %d\n",
> - ret);
> - goto error_v4l2_async_notifier;
> - }
> -
> return 0;
>
> -error_v4l2_async_notifier:
> - v4l2_async_nf_cleanup(notifier);
> -
> -error_video:
> - sun6i_video_cleanup(csi_dev);
> -
> -error_v4l2_device:
> - v4l2_device_unregister(&v4l2->v4l2_dev);
> -
> error_v4l2_ctrl:
> v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
>
> @@ -771,9 +654,6 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi_device *csi_dev)
> struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2;
>
> media_device_unregister(&v4l2->media_dev);
> - v4l2_async_nf_unregister(&v4l2->notifier);
> - v4l2_async_nf_cleanup(&v4l2->notifier);
> - sun6i_video_cleanup(csi_dev);
> v4l2_device_unregister(&v4l2->v4l2_dev);
> v4l2_ctrl_handler_free(&v4l2->ctrl_handler);
> media_device_cleanup(&v4l2->media_dev);
> @@ -995,8 +875,22 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)
> if (ret)
> goto error_resources;
>
> + ret = sun6i_csi_bridge_setup(csi_dev);
> + if (ret)
> + goto error_v4l2;
> +
> + ret = sun6i_video_setup(csi_dev);
> + if (ret)
> + goto error_bridge;
> +
> return 0;
>
> +error_bridge:
> + sun6i_csi_bridge_cleanup(csi_dev);
> +
> +error_v4l2:
> + sun6i_csi_v4l2_cleanup(csi_dev);
> +
> error_resources:
> sun6i_csi_resources_cleanup(csi_dev);
>
> @@ -1007,6 +901,8 @@ static int sun6i_csi_remove(struct platform_device *pdev)
> {
> struct sun6i_csi_device *csi_dev = platform_get_drvdata(pdev);
>
> + sun6i_video_cleanup(csi_dev);
> + sun6i_csi_bridge_cleanup(csi_dev);
> sun6i_csi_v4l2_cleanup(csi_dev);
> sun6i_csi_resources_cleanup(csi_dev);
>
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> index 4dd83e57bafa..576c7f10289e 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h
> @@ -13,11 +13,16 @@
> #include <media/v4l2-fwnode.h>
> #include <media/videobuf2-v4l2.h>
>
> +#include "sun6i_csi_bridge.h"
> #include "sun6i_video.h"
>
> #define SUN6I_CSI_NAME "sun6i-csi"
> #define SUN6I_CSI_DESCRIPTION "Allwinner A31 CSI Device"
>
> +enum sun6i_csi_port {
> + SUN6I_CSI_PORT_PARALLEL = 0,
> +};
> +
> struct sun6i_csi_buffer {
> struct vb2_v4l2_buffer v4l2_buffer;
> struct list_head list;
> @@ -46,10 +51,6 @@ struct sun6i_csi_v4l2 {
> struct v4l2_device v4l2_dev;
> struct v4l2_ctrl_handler ctrl_handler;
> struct media_device media_dev;
> -
> - struct v4l2_async_notifier notifier;
> - /* video port settings */
> - struct v4l2_fwnode_endpoint v4l2_ep;
> };
>
> struct sun6i_csi_device {
> @@ -57,6 +58,7 @@ struct sun6i_csi_device {
>
> struct sun6i_csi_config config;
> struct sun6i_csi_v4l2 v4l2;
> + struct sun6i_csi_bridge bridge;
> struct sun6i_video video;
>
> struct regmap *regmap;
> @@ -156,4 +158,6 @@ static inline int sun6i_csi_get_bpp(unsigned int pixformat)
> return 0;
> }
>
> +int sun6i_csi_v4l2_complete(struct sun6i_csi_device *csi_dev);
> +
> #endif /* __SUN6I_CSI_H__ */
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
> new file mode 100644
> index 000000000000..74706d883359
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
> @@ -0,0 +1,473 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright 2021-2022 Bootlin
> + * Author: Paul Kocialkowski <[email protected]>
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#include "sun6i_csi.h"
> +#include "sun6i_csi_bridge.h"
> +
> +/* Format */
> +
> +static const u32 sun6i_csi_bridge_mbus_codes[] = {
> + /* Bayer */
> + MEDIA_BUS_FMT_SBGGR8_1X8,
> + MEDIA_BUS_FMT_SGBRG8_1X8,
> + MEDIA_BUS_FMT_SGRBG8_1X8,
> + MEDIA_BUS_FMT_SRGGB8_1X8,
> + MEDIA_BUS_FMT_SBGGR10_1X10,
> + MEDIA_BUS_FMT_SGBRG10_1X10,
> + MEDIA_BUS_FMT_SGRBG10_1X10,
> + MEDIA_BUS_FMT_SRGGB10_1X10,
> + MEDIA_BUS_FMT_SBGGR12_1X12,
> + MEDIA_BUS_FMT_SGBRG12_1X12,
> + MEDIA_BUS_FMT_SGRBG12_1X12,
> + MEDIA_BUS_FMT_SRGGB12_1X12,
> + /* RGB */
> + MEDIA_BUS_FMT_RGB565_2X8_LE,
> + MEDIA_BUS_FMT_RGB565_2X8_BE,
> + /* YUV422 */
> + MEDIA_BUS_FMT_YUYV8_2X8,
> + MEDIA_BUS_FMT_UYVY8_2X8,
> + MEDIA_BUS_FMT_YVYU8_2X8,
> + MEDIA_BUS_FMT_UYVY8_2X8,
> + MEDIA_BUS_FMT_VYUY8_2X8,
> + MEDIA_BUS_FMT_YUYV8_1X16,
> + MEDIA_BUS_FMT_UYVY8_1X16,
> + MEDIA_BUS_FMT_YVYU8_1X16,
> + MEDIA_BUS_FMT_UYVY8_1X16,
> + MEDIA_BUS_FMT_VYUY8_1X16,
> + /* Compressed */
> + MEDIA_BUS_FMT_JPEG_1X8,
> +};
> +
> +static bool sun6i_csi_bridge_mbus_code_check(u32 mbus_code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(sun6i_csi_bridge_mbus_codes); i++)
> + if (sun6i_csi_bridge_mbus_codes[i] == mbus_code)
> + return true;
> +
> + return false;
> +}
> +
> +/* V4L2 Subdev */
> +
> +static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on)
> +{
> + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> + struct v4l2_subdev *source_subdev;
> + /* Initialize to 0 to use both in disable label (ret != 0) and off. */
> + int ret = 0;
> +
> + /* Source */
> +
> + if (!csi_dev->bridge.source)
> + return -ENODEV;
> +
> + source_subdev = csi_dev->bridge.source->subdev;
> +
> + if (!on) {
> + v4l2_subdev_call(source_subdev, video, s_stream, 0);
> + goto disable;
> + }
> +
> + ret = v4l2_subdev_call(source_subdev, video, s_stream, 1);
> + if (ret && ret != -ENOIOCTLCMD)
> + goto disable;
> +
> + return 0;
> +
> +disable:
> + csi_dev->bridge.source = NULL;
> +
> + return ret;
> +}
> +
> +static const struct v4l2_subdev_video_ops sun6i_csi_bridge_video_ops = {
> + .s_stream = sun6i_csi_bridge_s_stream,
> +};
> +
> +static void
> +sun6i_csi_bridge_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format)
> +{
> + if (!sun6i_csi_bridge_mbus_code_check(mbus_format->code))
> + mbus_format->code = sun6i_csi_bridge_mbus_codes[0];
> +
> + mbus_format->field = V4L2_FIELD_NONE;
> + mbus_format->colorspace = V4L2_COLORSPACE_RAW;
> + mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT;
> + mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +}
> +
> +static int sun6i_csi_bridge_init_cfg(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state)
> +{
> + unsigned int pad = SUN6I_CSI_BRIDGE_PAD_SINK;
> + struct v4l2_mbus_framefmt *mbus_format =
> + v4l2_subdev_get_try_format(subdev, state, pad);
> +
> + mbus_format->code = sun6i_csi_bridge_mbus_codes[0];
> + mbus_format->width = 1280;
> + mbus_format->height = 720;
> +
> + sun6i_csi_bridge_mbus_format_prepare(mbus_format);
> +
> + return 0;
> +}
> +
> +static int
> +sun6i_csi_bridge_enum_mbus_code(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_mbus_code_enum *code_enum)
> +{
> + if (code_enum->index >= ARRAY_SIZE(sun6i_csi_bridge_mbus_codes))
> + return -EINVAL;
> +
> + code_enum->code = sun6i_csi_bridge_mbus_codes[code_enum->index];
> +
> + return 0;
> +}
> +
> +static int sun6i_csi_bridge_get_fmt(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> + struct v4l2_mbus_framefmt *mbus_format = &format->format;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + *mbus_format = *v4l2_subdev_get_try_format(subdev, state,
> + format->pad);
> + else
> + *mbus_format = csi_dev->bridge.mbus_format;
> +
> + return 0;
> +}
> +
> +static int sun6i_csi_bridge_set_fmt(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> + struct v4l2_mbus_framefmt *mbus_format = &format->format;
> +
> + sun6i_csi_bridge_mbus_format_prepare(mbus_format);
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + *v4l2_subdev_get_try_format(subdev, state, format->pad) =
> + *mbus_format;
> + else
> + csi_dev->bridge.mbus_format = *mbus_format;
Note that the driver is responsible for serialising access to its data,
i.e. you have to acquire the mutex here.
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops sun6i_csi_bridge_pad_ops = {
> + .init_cfg = sun6i_csi_bridge_init_cfg,
> + .enum_mbus_code = sun6i_csi_bridge_enum_mbus_code,
> + .get_fmt = sun6i_csi_bridge_get_fmt,
> + .set_fmt = sun6i_csi_bridge_set_fmt,
> +};
> +
> +const struct v4l2_subdev_ops sun6i_csi_bridge_subdev_ops = {
> + .video = &sun6i_csi_bridge_video_ops,
> + .pad = &sun6i_csi_bridge_pad_ops,
> +};
> +
> +/* Media Entity */
> +
> +static int sun6i_csi_bridge_link_validate(struct media_link *link)
> +{
> + struct v4l2_subdev *subdev =
> + media_entity_to_v4l2_subdev(link->sink->entity);
> + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> + struct device *dev = csi_dev->dev;
> + struct v4l2_subdev *source_subdev =
> + media_entity_to_v4l2_subdev(link->source->entity);
> + int ret;
> +
> + /* Only support one enabled source at a time. */
> + if (bridge->source) {
> + dev_err(dev, "multiple sources are connected to the bridge\n");
> + return -EBUSY;
> + }
> +
> + ret = v4l2_subdev_link_validate(link);
> + if (ret)
> + return ret;
> +
> + if (source_subdev == bridge->source_parallel.subdev)
> + bridge->source = &bridge->source_parallel;
> + else
Useless use of else.
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static const struct media_entity_operations sun6i_csi_bridge_entity_ops = {
> + .link_validate = sun6i_csi_bridge_link_validate,
> +};
> +
> +/* V4L2 Async */
> +
> +static int sun6i_csi_bridge_link(struct sun6i_csi_device *csi_dev,
> + int sink_pad_index,
> + struct v4l2_subdev *remote_subdev,
> + bool enabled)
> +{
> + struct device *dev = csi_dev->dev;
> + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
> + struct media_entity *sink_entity = &subdev->entity;
> + struct media_entity *source_entity = &remote_subdev->entity;
> + int source_pad_index;
> + int ret;
> +
> + /* Get the first remote source pad. */
> + ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (ret < 0) {
> + dev_err(dev, "missing source pad in external entity %s\n",
> + source_entity->name);
> + return -EINVAL;
> + }
> +
> + source_pad_index = ret;
> +
> + dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
> + source_pad_index, sink_entity->name, sink_pad_index);
> +
> + ret = media_create_pad_link(source_entity, source_pad_index,
> + sink_entity, sink_pad_index,
> + enabled ? MEDIA_LNK_FL_ENABLED : 0);
> + if (ret < 0) {
> + dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
> + source_entity->name, source_pad_index,
> + sink_entity->name, sink_pad_index);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *remote_subdev,
> + struct v4l2_async_subdev *async_subdev)
> +{
> + struct sun6i_csi_device *csi_dev =
> + container_of(notifier, struct sun6i_csi_device,
> + bridge.notifier);
> + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> + struct sun6i_csi_bridge_source *source = NULL;
> + struct fwnode_handle *fwnode = dev_fwnode(csi_dev->dev);
> + struct fwnode_handle *handle = NULL;
> + bool enabled;
> + int ret;
> +
> + while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
I'd instead store the information you need here in struct sun6i_csi_bridge.
You could remove the loop here.
> + struct fwnode_endpoint endpoint = { 0 };
> + struct fwnode_handle *remote_fwnode;
> +
> + remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
> + if (!remote_fwnode)
> + continue;
> +
> + if (remote_fwnode != remote_subdev->fwnode)
> + goto next;
> +
> + ret = fwnode_graph_parse_endpoint(handle, &endpoint);
> + if (ret < 0)
> + goto next;
> +
> + switch (endpoint.port) {
> + case SUN6I_CSI_PORT_PARALLEL:
> + source = &bridge->source_parallel;
> + enabled = true;
> + break;
> + default:
> + break;
> + }
> +
> +next:
> + fwnode_handle_put(remote_fwnode);
> + }
> +
> + if (!source)
> + return -EINVAL;
> +
> + source->subdev = remote_subdev;
> +
> + return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK,
> + remote_subdev, enabled);
> +}
> +
> +static int
> +sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier)
> +{
> + struct sun6i_csi_device *csi_dev =
> + container_of(notifier, struct sun6i_csi_device,
> + bridge.notifier);
> +
> + return sun6i_csi_v4l2_complete(csi_dev);
You could call v4l2_device_register_subdev_nodes() here.
> +}
> +
> +static const struct v4l2_async_notifier_operations
> +sun6i_csi_bridge_notifier_ops = {
> + .bound = sun6i_csi_bridge_notifier_bound,
> + .complete = sun6i_csi_bridge_notifier_complete,
> +};
> +
> +/* Bridge */
> +
> +static int sun6i_csi_bridge_source_setup(struct sun6i_csi_device *csi_dev,
> + struct sun6i_csi_bridge_source *source,
> + u32 port,
> + enum v4l2_mbus_type *bus_types)
> +{
> + struct device *dev = csi_dev->dev;
> + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
> + struct v4l2_fwnode_endpoint *endpoint = &source->endpoint;
> + struct v4l2_async_subdev *async_subdev;
> + struct fwnode_handle *handle;
> + int ret;
> +
> + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0);
> + if (!handle)
> + return -ENODEV;
> +
> + ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
> + if (ret)
> + goto complete;
> +
> + if (bus_types) {
> + bool valid = false;
> + unsigned int i;
> +
> + for (i = 0; bus_types[i] != V4L2_MBUS_INVALID; i++) {
> + if (endpoint->bus_type == bus_types[i]) {
> + valid = true;
> + break;
> + }
> + }
> +
> + if (!valid) {
> + dev_err(dev, "unsupported bus type for port %d\n",
> + port);
> + ret = -EINVAL;
> + goto complete;
> + }
> + }
> +
> + async_subdev = v4l2_async_nf_add_fwnode_remote(notifier, handle,
> + struct v4l2_async_subdev);
> + if (IS_ERR(async_subdev)) {
> + ret = PTR_ERR(async_subdev);
> + goto complete;
> + }
> +
> + source->expected = true;
> +
> +complete:
> + fwnode_handle_put(handle);
> +
> + return ret;
> +}
> +
> +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev)
> +{
> + struct device *dev = csi_dev->dev;
> + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> + struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
> + struct v4l2_subdev *subdev = &bridge->subdev;
> + struct v4l2_async_notifier *notifier = &bridge->notifier;
> + struct media_pad *pads = bridge->pads;
> + enum v4l2_mbus_type parallel_mbus_types[] = {
> + V4L2_MBUS_PARALLEL,
> + V4L2_MBUS_BT656,
> + V4L2_MBUS_INVALID
> + };
> + int ret;
> +
> + /* V4L2 Subdev */
> +
> + v4l2_subdev_init(subdev, &sun6i_csi_bridge_subdev_ops);
> + strscpy(subdev->name, SUN6I_CSI_BRIDGE_NAME, sizeof(subdev->name));
> + subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + subdev->owner = THIS_MODULE;
> + subdev->dev = dev;
> +
> + v4l2_set_subdevdata(subdev, csi_dev);
> +
> + /* Media Entity */
> +
> + subdev->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> + subdev->entity.ops = &sun6i_csi_bridge_entity_ops;
> +
> + /* Media Pads */
> +
> + pads[SUN6I_CSI_BRIDGE_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + pads[SUN6I_CSI_BRIDGE_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE |
> + MEDIA_PAD_FL_MUST_CONNECT;
> +
> + ret = media_entity_pads_init(&subdev->entity,
> + SUN6I_CSI_BRIDGE_PAD_COUNT, pads);
> + if (ret < 0)
> + return ret;
> +
> + /* V4L2 Subdev */
> +
> + ret = v4l2_device_register_subdev(v4l2_dev, subdev);
> + if (ret) {
> + dev_err(dev, "failed to register v4l2 subdev: %d\n", ret);
> + goto error_media_entity;
> + }
> +
> + /* V4L2 Async */
> +
> + v4l2_async_nf_init(notifier);
> + notifier->ops = &sun6i_csi_bridge_notifier_ops;
> +
> + sun6i_csi_bridge_source_setup(csi_dev, &bridge->source_parallel,
> + SUN6I_CSI_PORT_PARALLEL,
> + parallel_mbus_types);
> +
> + ret = v4l2_async_nf_register(v4l2_dev, notifier);
> + if (ret) {
> + dev_err(dev, "failed to register v4l2 async notifier: %d\n",
> + ret);
> + goto error_v4l2_async_notifier;
> + }
> +
> + return 0;
> +
> +error_v4l2_async_notifier:
> + v4l2_async_nf_cleanup(notifier);
> +
> + v4l2_device_unregister_subdev(subdev);
> +
> +error_media_entity:
> + media_entity_cleanup(&subdev->entity);
> +
> + return ret;
> +}
> +
> +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev)
> +{
> + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
> + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
> +
> + v4l2_async_nf_unregister(notifier);
> + v4l2_async_nf_cleanup(notifier);
> +
> + v4l2_device_unregister_subdev(subdev);
> +
> + media_entity_cleanup(&subdev->entity);
> +}
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
> new file mode 100644
> index 000000000000..2ee7878102b6
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
> @@ -0,0 +1,44 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright 2021 Bootlin
2022?
> + * Author: Paul Kocialkowski <[email protected]>
> + */
> +
> +#ifndef _SUN6I_CSI_BRIDGE_H_
> +#define _SUN6I_CSI_BRIDGE_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#define SUN6I_CSI_BRIDGE_NAME "sun6i-csi-bridge"
> +
> +enum sun6i_csi_bridge_pad {
> + SUN6I_CSI_BRIDGE_PAD_SINK = 0,
> + SUN6I_CSI_BRIDGE_PAD_SOURCE = 1,
> + SUN6I_CSI_BRIDGE_PAD_COUNT = 2,
> +};
> +
> +struct sun6i_csi_device;
> +
> +struct sun6i_csi_bridge_source {
> + struct v4l2_subdev *subdev;
> + struct v4l2_fwnode_endpoint endpoint;
> + bool expected;
> +};
> +
> +struct sun6i_csi_bridge {
> + struct v4l2_subdev subdev;
> + struct v4l2_async_notifier notifier;
> + struct media_pad pads[2];
> + struct v4l2_mbus_framefmt mbus_format;
> +
> + struct sun6i_csi_bridge_source source_parallel;
> + struct sun6i_csi_bridge_source *source;
> +};
> +
> +/* Bridge */
> +
> +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev);
> +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev);
> +
> +#endif
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> index d32ff6b81f8a..fa5bf3697ace 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> @@ -632,6 +632,7 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
> {
> struct sun6i_video *video = &csi_dev->video;
> struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
> + struct v4l2_subdev *bridge_subdev = &csi_dev->bridge.subdev;
> struct video_device *video_dev = &video->video_dev;
> struct vb2_queue *queue = &video->queue;
> struct media_pad *pad = &video->pad;
> @@ -715,8 +716,26 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
> v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
> video_device_node_name(video_dev));
>
> + /* Media Pad Link */
> +
> + ret = media_create_pad_link(&bridge_subdev->entity,
> + SUN6I_CSI_BRIDGE_PAD_SOURCE,
> + &video_dev->entity, 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret < 0) {
> + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
> + bridge_subdev->entity.name,
> + SUN6I_CSI_BRIDGE_PAD_SOURCE,
> + video_dev->entity.name, 0);
> + goto error_video_device;
> + }
> +
> return 0;
>
> +error_video_device:
> + vb2_video_unregister_device(video_dev);
> +
> error_media_entity:
> media_entity_cleanup(&video_dev->entity);
>
--
Sakari Ailus
Hi Paul,
Thanks for the patchbomb.
On Sat, Feb 05, 2022 at 07:53:44PM +0100, Paul Kocialkowski wrote:
> In some situations the default rate of the module clock is not the
> required one for operation (for example when reconfiguring the clock
> tree to use a different parent). As a result, always set the correct
> rate for the clock (and take care of cleanup).
>
> Signed-off-by: Paul Kocialkowski <[email protected]>
> ---
> .../platform/sunxi/sun6i-csi/sun6i_csi.c | 54 ++++++++++++++-----
> 1 file changed, 41 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> index 8155e9560164..2355088fdc37 100644
> --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> @@ -856,28 +849,53 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev,
> return PTR_ERR(csi_dev->clk_ram);
> }
>
> + if (of_device_is_compatible(dev->of_node, "allwinner,sun50i-a64-csi"))
> + clk_mod_rate = 300000000;
> + else
> + clk_mod_rate = 297000000;
This would be nice to put in OF match data.
Of course the driver did this already before the patch. The approach still
scales badly.
--
Regards,
Sakari Ailus
Hi Laurent,
On Mon 07 Feb 22, 18:16, Laurent Pinchart wrote:
> Hi Paul,
>
> Thank you for the patch.
>
> On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> > Some Allwinner platforms come with an Image Signal Processor, which
> > supports various features in order to enhance and transform data
> > received by image sensors into good-looking pictures. In most cases,
> > the data is raw bayer, which gets internally converted to RGB and
> > finally YUV, which is what the hardware produces.
> >
> > This driver supports ISPs that are similar to the A31 ISP, which was
> > the first standalone ISP found in Allwinner platforms. Simpler ISP
> > blocks were found in the A10 and A20, where they are tied to a CSI
> > controller. Newer generations of Allwinner SoCs (starting with the
> > H6, H616, etc) come with a new camera subsystem and revised ISP.
> > Even though these previous and next-generation ISPs are somewhat
> > similar to the A31 ISP, they have enough significant differences to
> > be out of the scope of this driver.
> >
> > While the ISP supports many features, including 3A and many
> > enhancement blocks, this implementation is limited to the following:
> > - V3s (V3/S3) platform support;
> > - Bayer media bus formats as input;
>
> Greyscale formats would also be nice to have, if the hardware can
> support that (it mostly just requires the ability to disable the CFA
> interpolation).
As far as I know there's no support for grayscale, only bayer formats
and YUV.
> > - Semi-planar YUV (NV12/NV21) as output;
>
> Packed YUV would also be useful if the hardware supports it.
Same here, it only supports planar and semi-planar YUV as output.
> > - Debayering with per-component gain and offset configuration;
> > - 2D noise filtering with configurable coefficients.
> >
> > Since many features are missing from the associated uAPI, the driver
> > is aimed to integrate staging until all features are properly
> > described.
> >
> > On the technical side, it uses the v4l2 and media controller APIs,
> > with a video node for capture, a processor subdev and a video node
> > for parameters submission. A specific uAPI structure and associated
> > v4l2 meta format are used to configure parameters of the supported
> > modules.
> >
> > One particular thing about the hardware is that configuration for
> > module registers needs to be stored in a DMA buffer and gets copied
> > to actual registers by the hardware at the next vsync, when instructed
> > by a flag. This is handled by the "state" mechanism in the driver.
> >
> > Signed-off-by: Paul Kocialkowski <[email protected]>
> > ---
> > drivers/staging/media/sunxi/Kconfig | 1 +
> > drivers/staging/media/sunxi/Makefile | 1 +
> > drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> > .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> > .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> > .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> > .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> > .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> > .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> > .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> > .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> > .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> > .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> > .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
>
> Could you add a TODO file to list the issues that need to be fixed for
> the driver to move out of staging ? I'll already propose one entry:
>
> - Add support in libcamera
Maybe it would be good to narrow down what level of support you have in mind
here. Just adding basic support fort the pipeline is probably doable, but
developing complex 3A algorithms would require very significant effort and
it would be a shame that this prevents the driver from leaving staging.
I think another obvious task would be to have a complete uAPI that reflects
all modules that are part of the ISP.
What do you think?
Paul
> This isn't required to merge the driver in staging as long as ABI
> compatibility doesn't need to be preserved until the driver is moved out
> of staging.
>
> > 14 files changed, 3109 insertions(+)
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
>
> [snip]
>
> --
> Regards,
>
> Laurent Pinchart
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Sakari,
On Mon 14 Feb 22, 18:31, Sakari Ailus wrote:
> Hi Paul,
>
> Thanks for the patchbomb.
I'll split it in the next revision.
> On Sat, Feb 05, 2022 at 07:53:44PM +0100, Paul Kocialkowski wrote:
> > In some situations the default rate of the module clock is not the
> > required one for operation (for example when reconfiguring the clock
> > tree to use a different parent). As a result, always set the correct
> > rate for the clock (and take care of cleanup).
> >
> > Signed-off-by: Paul Kocialkowski <[email protected]>
> > ---
> > .../platform/sunxi/sun6i-csi/sun6i_csi.c | 54 ++++++++++++++-----
> > 1 file changed, 41 insertions(+), 13 deletions(-)
> >
> > diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> > index 8155e9560164..2355088fdc37 100644
> > --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> > +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> > @@ -856,28 +849,53 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev,
> > return PTR_ERR(csi_dev->clk_ram);
> > }
> >
> > + if (of_device_is_compatible(dev->of_node, "allwinner,sun50i-a64-csi"))
> > + clk_mod_rate = 300000000;
> > + else
> > + clk_mod_rate = 297000000;
>
> This would be nice to put in OF match data.
>
> Of course the driver did this already before the patch. The approach still
> scales badly.
Agreed, that could be another follow-up patch in the sun6i-csi rework series.
Paul
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul,
On Wed, Mar 02, 2022 at 02:23:37PM +0100, Paul Kocialkowski wrote:
> On Wed 02 Mar 22, 10:51, Laurent Pinchart wrote:
> > On Tue, Mar 01, 2022 at 04:58:51PM +0100, Paul Kocialkowski wrote:
> > > On Mon 07 Feb 22, 18:16, Laurent Pinchart wrote:
> > > > On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> > > > > Some Allwinner platforms come with an Image Signal Processor, which
> > > > > supports various features in order to enhance and transform data
> > > > > received by image sensors into good-looking pictures. In most cases,
> > > > > the data is raw bayer, which gets internally converted to RGB and
> > > > > finally YUV, which is what the hardware produces.
> > > > >
> > > > > This driver supports ISPs that are similar to the A31 ISP, which was
> > > > > the first standalone ISP found in Allwinner platforms. Simpler ISP
> > > > > blocks were found in the A10 and A20, where they are tied to a CSI
> > > > > controller. Newer generations of Allwinner SoCs (starting with the
> > > > > H6, H616, etc) come with a new camera subsystem and revised ISP.
> > > > > Even though these previous and next-generation ISPs are somewhat
> > > > > similar to the A31 ISP, they have enough significant differences to
> > > > > be out of the scope of this driver.
> > > > >
> > > > > While the ISP supports many features, including 3A and many
> > > > > enhancement blocks, this implementation is limited to the following:
> > > > > - V3s (V3/S3) platform support;
> > > > > - Bayer media bus formats as input;
> > > >
> > > > Greyscale formats would also be nice to have, if the hardware can
> > > > support that (it mostly just requires the ability to disable the CFA
> > > > interpolation).
> > >
> > > As far as I know there's no support for grayscale, only bayer formats
> > > and YUV.
> > >
> > > > > - Semi-planar YUV (NV12/NV21) as output;
> > > >
> > > > Packed YUV would also be useful if the hardware supports it.
> > >
> > > Same here, it only supports planar and semi-planar YUV as output.
> > >
> > > > > - Debayering with per-component gain and offset configuration;
> > > > > - 2D noise filtering with configurable coefficients.
> > > > >
> > > > > Since many features are missing from the associated uAPI, the driver
> > > > > is aimed to integrate staging until all features are properly
> > > > > described.
> > > > >
> > > > > On the technical side, it uses the v4l2 and media controller APIs,
> > > > > with a video node for capture, a processor subdev and a video node
> > > > > for parameters submission. A specific uAPI structure and associated
> > > > > v4l2 meta format are used to configure parameters of the supported
> > > > > modules.
> > > > >
> > > > > One particular thing about the hardware is that configuration for
> > > > > module registers needs to be stored in a DMA buffer and gets copied
> > > > > to actual registers by the hardware at the next vsync, when instructed
> > > > > by a flag. This is handled by the "state" mechanism in the driver.
> > > > >
> > > > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > > > ---
> > > > > drivers/staging/media/sunxi/Kconfig | 1 +
> > > > > drivers/staging/media/sunxi/Makefile | 1 +
> > > > > drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> > > > > .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> > > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> > > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> > > > > .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> > > > > .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
> > > >
> > > > Could you add a TODO file to list the issues that need to be fixed for
> > > > the driver to move out of staging ? I'll already propose one entry:
> > > >
> > > > - Add support in libcamera
> > >
> > > Maybe it would be good to narrow down what level of support you have in mind
> > > here. Just adding basic support fort the pipeline is probably doable, but
> > > developing complex 3A algorithms would require very significant effort and
> > > it would be a shame that this prevents the driver from leaving staging.
> > >
> > > I think another obvious task would be to have a complete uAPI that reflects
> > > all modules that are part of the ISP.
> > >
> > > What do you think?
> >
> > The reason why I'd like to see libcamera support for the ISP driver is
> > to ensure that the kernel API is adequate for real use cases. The API
> > can be split in three parts:
> >
> > - Pipeline configuration (this includes the media controller topology,
> > link setting, subdev pad format/selection rectangle configuration,
> > ...)
> > - ISP parameters
> > - ISP statistics
> >
> > A pipeline handler implementation will cover the first parts. The second
> > and third parts need to be tested too, but we don't need to implement
> > every single feature. A very simple algorithm that demonstrates
> > statistics can be captured and ISP parameters can be set should be
> > enough to test and exercise the API in real scenarios. We're working on
> > making basic AE and AWB algorithm implementations generic (or at least
> > creating generic building blocks that can easily be assembled to create
> > those algorithms, as the ISP statistics and parameters are specific to
> > the ISP and thus require some ISP-specific code), so that should become
> > a fairly easy task soon. I expect most of the work to go in the pipeline
> > handler.
> >
> > Does this sound fair to you ?
>
> Yes I understand that these aspects need to be tested too, but I feel like
> having an implementation with a feedback look (even rudimentary and using
> some generic helpers) would be quite a stretch.
Don't be scared, it's not that difficult :-) A very basic feedback loop
is easier to implement than the code that configures the pipeline and
tracks buffers.
> It would probably be sufficient to have some demo code that can receive stats
> and set parameters, but without necessarily any connection between the two.
> As such it's also my feeling that a standalone demo program could be easier
> to manage for that purpose than libcamera support.
>
> So how about making the requirement that a (free software) userspace
> implementration must demonstrate ability to read relevant statistics
> from the ISP and (independently) control parameters that affect the output?
It's easy to create a test tool to demonstrate that the driver works
fine, when the purpose of the test tool is to exercise the driver API in
the way intended by the driver author. What such a test tool fails to
demonstrate is the fitness of the API for real uses cases. This is why
KMS requires userspace APIs to be exercised in a real display stack
(X.org, Weston, Android hwcomposer, ...) and not just in a test tool.
We're following the same rationale here.
> > > > This isn't required to merge the driver in staging as long as ABI
> > > > compatibility doesn't need to be preserved until the driver is moved out
> > > > of staging.
> > > >
> > > > > 14 files changed, 3109 insertions(+)
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> > > >
> > > > [snip]
--
Regards,
Laurent Pinchart
Hi Laurent,
On Wed 02 Mar 22, 15:33, Laurent Pinchart wrote:
> Hi Paul,
>
> On Wed, Mar 02, 2022 at 02:23:37PM +0100, Paul Kocialkowski wrote:
> > On Wed 02 Mar 22, 10:51, Laurent Pinchart wrote:
> > > On Tue, Mar 01, 2022 at 04:58:51PM +0100, Paul Kocialkowski wrote:
> > > > On Mon 07 Feb 22, 18:16, Laurent Pinchart wrote:
> > > > > On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> > > > > > Some Allwinner platforms come with an Image Signal Processor, which
> > > > > > supports various features in order to enhance and transform data
> > > > > > received by image sensors into good-looking pictures. In most cases,
> > > > > > the data is raw bayer, which gets internally converted to RGB and
> > > > > > finally YUV, which is what the hardware produces.
> > > > > >
> > > > > > This driver supports ISPs that are similar to the A31 ISP, which was
> > > > > > the first standalone ISP found in Allwinner platforms. Simpler ISP
> > > > > > blocks were found in the A10 and A20, where they are tied to a CSI
> > > > > > controller. Newer generations of Allwinner SoCs (starting with the
> > > > > > H6, H616, etc) come with a new camera subsystem and revised ISP.
> > > > > > Even though these previous and next-generation ISPs are somewhat
> > > > > > similar to the A31 ISP, they have enough significant differences to
> > > > > > be out of the scope of this driver.
> > > > > >
> > > > > > While the ISP supports many features, including 3A and many
> > > > > > enhancement blocks, this implementation is limited to the following:
> > > > > > - V3s (V3/S3) platform support;
> > > > > > - Bayer media bus formats as input;
> > > > >
> > > > > Greyscale formats would also be nice to have, if the hardware can
> > > > > support that (it mostly just requires the ability to disable the CFA
> > > > > interpolation).
> > > >
> > > > As far as I know there's no support for grayscale, only bayer formats
> > > > and YUV.
> > > >
> > > > > > - Semi-planar YUV (NV12/NV21) as output;
> > > > >
> > > > > Packed YUV would also be useful if the hardware supports it.
> > > >
> > > > Same here, it only supports planar and semi-planar YUV as output.
> > > >
> > > > > > - Debayering with per-component gain and offset configuration;
> > > > > > - 2D noise filtering with configurable coefficients.
> > > > > >
> > > > > > Since many features are missing from the associated uAPI, the driver
> > > > > > is aimed to integrate staging until all features are properly
> > > > > > described.
> > > > > >
> > > > > > On the technical side, it uses the v4l2 and media controller APIs,
> > > > > > with a video node for capture, a processor subdev and a video node
> > > > > > for parameters submission. A specific uAPI structure and associated
> > > > > > v4l2 meta format are used to configure parameters of the supported
> > > > > > modules.
> > > > > >
> > > > > > One particular thing about the hardware is that configuration for
> > > > > > module registers needs to be stored in a DMA buffer and gets copied
> > > > > > to actual registers by the hardware at the next vsync, when instructed
> > > > > > by a flag. This is handled by the "state" mechanism in the driver.
> > > > > >
> > > > > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > > > > ---
> > > > > > drivers/staging/media/sunxi/Kconfig | 1 +
> > > > > > drivers/staging/media/sunxi/Makefile | 1 +
> > > > > > drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> > > > > > .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> > > > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> > > > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> > > > > > .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> > > > > > .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
> > > > >
> > > > > Could you add a TODO file to list the issues that need to be fixed for
> > > > > the driver to move out of staging ? I'll already propose one entry:
> > > > >
> > > > > - Add support in libcamera
> > > >
> > > > Maybe it would be good to narrow down what level of support you have in mind
> > > > here. Just adding basic support fort the pipeline is probably doable, but
> > > > developing complex 3A algorithms would require very significant effort and
> > > > it would be a shame that this prevents the driver from leaving staging.
> > > >
> > > > I think another obvious task would be to have a complete uAPI that reflects
> > > > all modules that are part of the ISP.
> > > >
> > > > What do you think?
> > >
> > > The reason why I'd like to see libcamera support for the ISP driver is
> > > to ensure that the kernel API is adequate for real use cases. The API
> > > can be split in three parts:
> > >
> > > - Pipeline configuration (this includes the media controller topology,
> > > link setting, subdev pad format/selection rectangle configuration,
> > > ...)
> > > - ISP parameters
> > > - ISP statistics
> > >
> > > A pipeline handler implementation will cover the first parts. The second
> > > and third parts need to be tested too, but we don't need to implement
> > > every single feature. A very simple algorithm that demonstrates
> > > statistics can be captured and ISP parameters can be set should be
> > > enough to test and exercise the API in real scenarios. We're working on
> > > making basic AE and AWB algorithm implementations generic (or at least
> > > creating generic building blocks that can easily be assembled to create
> > > those algorithms, as the ISP statistics and parameters are specific to
> > > the ISP and thus require some ISP-specific code), so that should become
> > > a fairly easy task soon. I expect most of the work to go in the pipeline
> > > handler.
> > >
> > > Does this sound fair to you ?
> >
> > Yes I understand that these aspects need to be tested too, but I feel like
> > having an implementation with a feedback look (even rudimentary and using
> > some generic helpers) would be quite a stretch.
>
> Don't be scared, it's not that difficult :-) A very basic feedback loop
> is easier to implement than the code that configures the pipeline and
> tracks buffers.
Thanks for the reassurance :) In any case I think there's work to be done
to figure the precise meaning of the feedback data. I remember seeing headers
that describe the data layout but the meaning of individual fields seems
very vague. But in any case we need proper understanding of that since
that's what the driver will be returning.
> > It would probably be sufficient to have some demo code that can receive stats
> > and set parameters, but without necessarily any connection between the two.
> > As such it's also my feeling that a standalone demo program could be easier
> > to manage for that purpose than libcamera support.
> >
> > So how about making the requirement that a (free software) userspace
> > implementration must demonstrate ability to read relevant statistics
> > from the ISP and (independently) control parameters that affect the output?
>
> It's easy to create a test tool to demonstrate that the driver works
> fine, when the purpose of the test tool is to exercise the driver API in
> the way intended by the driver author. What such a test tool fails to
> demonstrate is the fitness of the API for real uses cases. This is why
> KMS requires userspace APIs to be exercised in a real display stack
> (X.org, Weston, Android hwcomposer, ...) and not just in a test tool.
> We're following the same rationale here.
Okay I understand your point here. If the expectation is to fully validate
proper use-cases and not just parts of the API individually, I don't object.
It's a high standard but it might prevent issues down the road.
Cheers,
Paul
> > > > > This isn't required to merge the driver in staging as long as ABI
> > > > > compatibility doesn't need to be preserved until the driver is moved out
> > > > > of staging.
> > > > >
> > > > > > 14 files changed, 3109 insertions(+)
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> > > > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> > > > >
> > > > > [snip]
>
> --
> Regards,
>
> Laurent Pinchart
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul,
On Tue, Mar 01, 2022 at 04:58:51PM +0100, Paul Kocialkowski wrote:
> On Mon 07 Feb 22, 18:16, Laurent Pinchart wrote:
> > On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> > > Some Allwinner platforms come with an Image Signal Processor, which
> > > supports various features in order to enhance and transform data
> > > received by image sensors into good-looking pictures. In most cases,
> > > the data is raw bayer, which gets internally converted to RGB and
> > > finally YUV, which is what the hardware produces.
> > >
> > > This driver supports ISPs that are similar to the A31 ISP, which was
> > > the first standalone ISP found in Allwinner platforms. Simpler ISP
> > > blocks were found in the A10 and A20, where they are tied to a CSI
> > > controller. Newer generations of Allwinner SoCs (starting with the
> > > H6, H616, etc) come with a new camera subsystem and revised ISP.
> > > Even though these previous and next-generation ISPs are somewhat
> > > similar to the A31 ISP, they have enough significant differences to
> > > be out of the scope of this driver.
> > >
> > > While the ISP supports many features, including 3A and many
> > > enhancement blocks, this implementation is limited to the following:
> > > - V3s (V3/S3) platform support;
> > > - Bayer media bus formats as input;
> >
> > Greyscale formats would also be nice to have, if the hardware can
> > support that (it mostly just requires the ability to disable the CFA
> > interpolation).
>
> As far as I know there's no support for grayscale, only bayer formats
> and YUV.
>
> > > - Semi-planar YUV (NV12/NV21) as output;
> >
> > Packed YUV would also be useful if the hardware supports it.
>
> Same here, it only supports planar and semi-planar YUV as output.
>
> > > - Debayering with per-component gain and offset configuration;
> > > - 2D noise filtering with configurable coefficients.
> > >
> > > Since many features are missing from the associated uAPI, the driver
> > > is aimed to integrate staging until all features are properly
> > > described.
> > >
> > > On the technical side, it uses the v4l2 and media controller APIs,
> > > with a video node for capture, a processor subdev and a video node
> > > for parameters submission. A specific uAPI structure and associated
> > > v4l2 meta format are used to configure parameters of the supported
> > > modules.
> > >
> > > One particular thing about the hardware is that configuration for
> > > module registers needs to be stored in a DMA buffer and gets copied
> > > to actual registers by the hardware at the next vsync, when instructed
> > > by a flag. This is handled by the "state" mechanism in the driver.
> > >
> > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > ---
> > > drivers/staging/media/sunxi/Kconfig | 1 +
> > > drivers/staging/media/sunxi/Makefile | 1 +
> > > drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> > > .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> > > .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> > > .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
> >
> > Could you add a TODO file to list the issues that need to be fixed for
> > the driver to move out of staging ? I'll already propose one entry:
> >
> > - Add support in libcamera
>
> Maybe it would be good to narrow down what level of support you have in mind
> here. Just adding basic support fort the pipeline is probably doable, but
> developing complex 3A algorithms would require very significant effort and
> it would be a shame that this prevents the driver from leaving staging.
>
> I think another obvious task would be to have a complete uAPI that reflects
> all modules that are part of the ISP.
>
> What do you think?
The reason why I'd like to see libcamera support for the ISP driver is
to ensure that the kernel API is adequate for real use cases. The API
can be split in three parts:
- Pipeline configuration (this includes the media controller topology,
link setting, subdev pad format/selection rectangle configuration,
...)
- ISP parameters
- ISP statistics
A pipeline handler implementation will cover the first parts. The second
and third parts need to be tested too, but we don't need to implement
every single feature. A very simple algorithm that demonstrates
statistics can be captured and ISP parameters can be set should be
enough to test and exercise the API in real scenarios. We're working on
making basic AE and AWB algorithm implementations generic (or at least
creating generic building blocks that can easily be assembled to create
those algorithms, as the ISP statistics and parameters are specific to
the ISP and thus require some ISP-specific code), so that should become
a fairly easy task soon. I expect most of the work to go in the pipeline
handler.
Does this sound fair to you ?
> > This isn't required to merge the driver in staging as long as ABI
> > compatibility doesn't need to be preserved until the driver is moved out
> > of staging.
> >
> > > 14 files changed, 3109 insertions(+)
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> >
> > [snip]
--
Regards,
Laurent Pinchart
Hi Sakari,
On Mon 14 Feb 22, 20:12, Sakari Ailus wrote:
> Hi Paul,
>
> On Sat, Feb 05, 2022 at 07:53:53PM +0100, Paul Kocialkowski wrote:
> > Introduce a bridge v4l2 subdev to prepare for separation between the
> > processing part (bridge) and the dma engine, which is required to
> > properly support ths isp workflow later on.
> >
> > Currently the bridge just manages fwnode mapping to media pads,
> > using an async notifier (which was previously in the main code).
> > The s_stream video op just forwards to the connected v4l2 subdev
> > (sensor or MIPI CSI-2 bridge).
> >
> > The video capture device is now registered after the bridge and
> > attaches to it with a media link.
> >
> > Signed-off-by: Paul Kocialkowski <[email protected]>
> > ---
> > .../media/platform/sunxi/sun6i-csi/Makefile | 2 +-
> > .../platform/sunxi/sun6i-csi/sun6i_csi.c | 156 +-----
> > .../platform/sunxi/sun6i-csi/sun6i_csi.h | 12 +-
> > .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 473 ++++++++++++++++++
> > .../sunxi/sun6i-csi/sun6i_csi_bridge.h | 44 ++
> > .../platform/sunxi/sun6i-csi/sun6i_video.c | 19 +
> > 6 files changed, 571 insertions(+), 135 deletions(-)
> > create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
> > create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
[...]
> > +static int sun6i_csi_bridge_set_fmt(struct v4l2_subdev *subdev,
> > + struct v4l2_subdev_state *state,
> > + struct v4l2_subdev_format *format)
> > +{
> > + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> > + struct v4l2_mbus_framefmt *mbus_format = &format->format;
> > +
> > + sun6i_csi_bridge_mbus_format_prepare(mbus_format);
> > +
> > + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> > + *v4l2_subdev_get_try_format(subdev, state, format->pad) =
> > + *mbus_format;
> > + else
> > + csi_dev->bridge.mbus_format = *mbus_format;
>
> Note that the driver is responsible for serialising access to its data,
> i.e. you have to acquire the mutex here.
Thanks, will take care of that next time.
> > +
> > + return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_pad_ops sun6i_csi_bridge_pad_ops = {
> > + .init_cfg = sun6i_csi_bridge_init_cfg,
> > + .enum_mbus_code = sun6i_csi_bridge_enum_mbus_code,
> > + .get_fmt = sun6i_csi_bridge_get_fmt,
> > + .set_fmt = sun6i_csi_bridge_set_fmt,
> > +};
> > +
> > +const struct v4l2_subdev_ops sun6i_csi_bridge_subdev_ops = {
> > + .video = &sun6i_csi_bridge_video_ops,
> > + .pad = &sun6i_csi_bridge_pad_ops,
> > +};
> > +
> > +/* Media Entity */
> > +
> > +static int sun6i_csi_bridge_link_validate(struct media_link *link)
> > +{
> > + struct v4l2_subdev *subdev =
> > + media_entity_to_v4l2_subdev(link->sink->entity);
> > + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev);
> > + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> > + struct device *dev = csi_dev->dev;
> > + struct v4l2_subdev *source_subdev =
> > + media_entity_to_v4l2_subdev(link->source->entity);
> > + int ret;
> > +
> > + /* Only support one enabled source at a time. */
> > + if (bridge->source) {
> > + dev_err(dev, "multiple sources are connected to the bridge\n");
> > + return -EBUSY;
> > + }
> > +
> > + ret = v4l2_subdev_link_validate(link);
> > + if (ret)
> > + return ret;
> > +
> > + if (source_subdev == bridge->source_parallel.subdev)
> > + bridge->source = &bridge->source_parallel;
> > + else
>
> Useless use of else.
Fair enough.
> > + return -EINVAL;
> > +
> > + return 0;
> > +}
> > +
> > +static const struct media_entity_operations sun6i_csi_bridge_entity_ops = {
> > + .link_validate = sun6i_csi_bridge_link_validate,
> > +};
> > +
> > +/* V4L2 Async */
> > +
> > +static int sun6i_csi_bridge_link(struct sun6i_csi_device *csi_dev,
> > + int sink_pad_index,
> > + struct v4l2_subdev *remote_subdev,
> > + bool enabled)
> > +{
> > + struct device *dev = csi_dev->dev;
> > + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
> > + struct media_entity *sink_entity = &subdev->entity;
> > + struct media_entity *source_entity = &remote_subdev->entity;
> > + int source_pad_index;
> > + int ret;
> > +
> > + /* Get the first remote source pad. */
> > + ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode,
> > + MEDIA_PAD_FL_SOURCE);
> > + if (ret < 0) {
> > + dev_err(dev, "missing source pad in external entity %s\n",
> > + source_entity->name);
> > + return -EINVAL;
> > + }
> > +
> > + source_pad_index = ret;
> > +
> > + dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name,
> > + source_pad_index, sink_entity->name, sink_pad_index);
> > +
> > + ret = media_create_pad_link(source_entity, source_pad_index,
> > + sink_entity, sink_pad_index,
> > + enabled ? MEDIA_LNK_FL_ENABLED : 0);
> > + if (ret < 0) {
> > + dev_err(dev, "failed to create %s:%u -> %s:%u link\n",
> > + source_entity->name, source_pad_index,
> > + sink_entity->name, sink_pad_index);
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > +sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,
> > + struct v4l2_subdev *remote_subdev,
> > + struct v4l2_async_subdev *async_subdev)
> > +{
> > + struct sun6i_csi_device *csi_dev =
> > + container_of(notifier, struct sun6i_csi_device,
> > + bridge.notifier);
> > + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> > + struct sun6i_csi_bridge_source *source = NULL;
> > + struct fwnode_handle *fwnode = dev_fwnode(csi_dev->dev);
> > + struct fwnode_handle *handle = NULL;
> > + bool enabled;
> > + int ret;
> > +
> > + while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
>
> I'd instead store the information you need here in struct sun6i_csi_bridge.
> You could remove the loop here.
Is there a different method for matching a remote subdev to a local port?
The rationale here is that I need the handle for fwnode_graph_parse_endpoint
but cannot get that handle from the remote subdev's fwnode pointer directly.
> > + struct fwnode_endpoint endpoint = { 0 };
> > + struct fwnode_handle *remote_fwnode;
> > +
> > + remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
> > + if (!remote_fwnode)
> > + continue;
> > +
> > + if (remote_fwnode != remote_subdev->fwnode)
> > + goto next;
> > +
> > + ret = fwnode_graph_parse_endpoint(handle, &endpoint);
> > + if (ret < 0)
> > + goto next;
> > +
> > + switch (endpoint.port) {
> > + case SUN6I_CSI_PORT_PARALLEL:
> > + source = &bridge->source_parallel;
> > + enabled = true;
> > + break;
> > + default:
> > + break;
> > + }
> > +
> > +next:
> > + fwnode_handle_put(remote_fwnode);
> > + }
> > +
> > + if (!source)
> > + return -EINVAL;
> > +
> > + source->subdev = remote_subdev;
> > +
> > + return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK,
> > + remote_subdev, enabled);
> > +}
> > +
> > +static int
> > +sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier)
> > +{
> > + struct sun6i_csi_device *csi_dev =
> > + container_of(notifier, struct sun6i_csi_device,
> > + bridge.notifier);
> > +
> > + return sun6i_csi_v4l2_complete(csi_dev);
>
> You could call v4l2_device_register_subdev_nodes() here.
That's definitely what sun6i_csi_v4l2_complete does (the diff is probably not
very clear). Note that the wrapper is extended later on to register the capture
video device for the no-isp path.
Maybe the capture registration could be kept in sun6i_csi_probe for the non-isp
path and then the wrapper wouldn't be needed. I don't mind either way.
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations
> > +sun6i_csi_bridge_notifier_ops = {
> > + .bound = sun6i_csi_bridge_notifier_bound,
> > + .complete = sun6i_csi_bridge_notifier_complete,
> > +};
> > +
> > +/* Bridge */
> > +
> > +static int sun6i_csi_bridge_source_setup(struct sun6i_csi_device *csi_dev,
> > + struct sun6i_csi_bridge_source *source,
> > + u32 port,
> > + enum v4l2_mbus_type *bus_types)
> > +{
> > + struct device *dev = csi_dev->dev;
> > + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
> > + struct v4l2_fwnode_endpoint *endpoint = &source->endpoint;
> > + struct v4l2_async_subdev *async_subdev;
> > + struct fwnode_handle *handle;
> > + int ret;
> > +
> > + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0);
> > + if (!handle)
> > + return -ENODEV;
> > +
> > + ret = v4l2_fwnode_endpoint_parse(handle, endpoint);
> > + if (ret)
> > + goto complete;
> > +
> > + if (bus_types) {
> > + bool valid = false;
> > + unsigned int i;
> > +
> > + for (i = 0; bus_types[i] != V4L2_MBUS_INVALID; i++) {
> > + if (endpoint->bus_type == bus_types[i]) {
> > + valid = true;
> > + break;
> > + }
> > + }
> > +
> > + if (!valid) {
> > + dev_err(dev, "unsupported bus type for port %d\n",
> > + port);
> > + ret = -EINVAL;
> > + goto complete;
> > + }
> > + }
> > +
> > + async_subdev = v4l2_async_nf_add_fwnode_remote(notifier, handle,
> > + struct v4l2_async_subdev);
> > + if (IS_ERR(async_subdev)) {
> > + ret = PTR_ERR(async_subdev);
> > + goto complete;
> > + }
> > +
> > + source->expected = true;
> > +
> > +complete:
> > + fwnode_handle_put(handle);
> > +
> > + return ret;
> > +}
> > +
> > +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev)
> > +{
> > + struct device *dev = csi_dev->dev;
> > + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> > + struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
> > + struct v4l2_subdev *subdev = &bridge->subdev;
> > + struct v4l2_async_notifier *notifier = &bridge->notifier;
> > + struct media_pad *pads = bridge->pads;
> > + enum v4l2_mbus_type parallel_mbus_types[] = {
> > + V4L2_MBUS_PARALLEL,
> > + V4L2_MBUS_BT656,
> > + V4L2_MBUS_INVALID
> > + };
> > + int ret;
> > +
> > + /* V4L2 Subdev */
> > +
> > + v4l2_subdev_init(subdev, &sun6i_csi_bridge_subdev_ops);
> > + strscpy(subdev->name, SUN6I_CSI_BRIDGE_NAME, sizeof(subdev->name));
> > + subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> > + subdev->owner = THIS_MODULE;
> > + subdev->dev = dev;
> > +
> > + v4l2_set_subdevdata(subdev, csi_dev);
> > +
> > + /* Media Entity */
> > +
> > + subdev->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > + subdev->entity.ops = &sun6i_csi_bridge_entity_ops;
> > +
> > + /* Media Pads */
> > +
> > + pads[SUN6I_CSI_BRIDGE_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > + pads[SUN6I_CSI_BRIDGE_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE |
> > + MEDIA_PAD_FL_MUST_CONNECT;
> > +
> > + ret = media_entity_pads_init(&subdev->entity,
> > + SUN6I_CSI_BRIDGE_PAD_COUNT, pads);
> > + if (ret < 0)
> > + return ret;
> > +
> > + /* V4L2 Subdev */
> > +
> > + ret = v4l2_device_register_subdev(v4l2_dev, subdev);
> > + if (ret) {
> > + dev_err(dev, "failed to register v4l2 subdev: %d\n", ret);
> > + goto error_media_entity;
> > + }
> > +
> > + /* V4L2 Async */
> > +
> > + v4l2_async_nf_init(notifier);
> > + notifier->ops = &sun6i_csi_bridge_notifier_ops;
> > +
> > + sun6i_csi_bridge_source_setup(csi_dev, &bridge->source_parallel,
> > + SUN6I_CSI_PORT_PARALLEL,
> > + parallel_mbus_types);
> > +
> > + ret = v4l2_async_nf_register(v4l2_dev, notifier);
> > + if (ret) {
> > + dev_err(dev, "failed to register v4l2 async notifier: %d\n",
> > + ret);
> > + goto error_v4l2_async_notifier;
> > + }
> > +
> > + return 0;
> > +
> > +error_v4l2_async_notifier:
> > + v4l2_async_nf_cleanup(notifier);
> > +
> > + v4l2_device_unregister_subdev(subdev);
> > +
> > +error_media_entity:
> > + media_entity_cleanup(&subdev->entity);
> > +
> > + return ret;
> > +}
> > +
> > +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev)
> > +{
> > + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev;
> > + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier;
> > +
> > + v4l2_async_nf_unregister(notifier);
> > + v4l2_async_nf_cleanup(notifier);
> > +
> > + v4l2_device_unregister_subdev(subdev);
> > +
> > + media_entity_cleanup(&subdev->entity);
> > +}
> > diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
> > new file mode 100644
> > index 000000000000..2ee7878102b6
> > --- /dev/null
> > +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h
> > @@ -0,0 +1,44 @@
> > +/* SPDX-License-Identifier: GPL-2.0+ */
> > +/*
> > + * Copyright 2021 Bootlin
>
> 2022?
Right, thanks!
> > + * Author: Paul Kocialkowski <[email protected]>
> > + */
> > +
> > +#ifndef _SUN6I_CSI_BRIDGE_H_
> > +#define _SUN6I_CSI_BRIDGE_H_
> > +
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-fwnode.h>
> > +
> > +#define SUN6I_CSI_BRIDGE_NAME "sun6i-csi-bridge"
> > +
> > +enum sun6i_csi_bridge_pad {
> > + SUN6I_CSI_BRIDGE_PAD_SINK = 0,
> > + SUN6I_CSI_BRIDGE_PAD_SOURCE = 1,
> > + SUN6I_CSI_BRIDGE_PAD_COUNT = 2,
> > +};
> > +
> > +struct sun6i_csi_device;
> > +
> > +struct sun6i_csi_bridge_source {
> > + struct v4l2_subdev *subdev;
> > + struct v4l2_fwnode_endpoint endpoint;
> > + bool expected;
> > +};
> > +
> > +struct sun6i_csi_bridge {
> > + struct v4l2_subdev subdev;
> > + struct v4l2_async_notifier notifier;
> > + struct media_pad pads[2];
> > + struct v4l2_mbus_framefmt mbus_format;
> > +
> > + struct sun6i_csi_bridge_source source_parallel;
> > + struct sun6i_csi_bridge_source *source;
> > +};
> > +
> > +/* Bridge */
> > +
> > +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev);
> > +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev);
> > +
> > +#endif
> > diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> > index d32ff6b81f8a..fa5bf3697ace 100644
> > --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> > +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
> > @@ -632,6 +632,7 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
> > {
> > struct sun6i_video *video = &csi_dev->video;
> > struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev;
> > + struct v4l2_subdev *bridge_subdev = &csi_dev->bridge.subdev;
> > struct video_device *video_dev = &video->video_dev;
> > struct vb2_queue *queue = &video->queue;
> > struct media_pad *pad = &video->pad;
> > @@ -715,8 +716,26 @@ int sun6i_video_setup(struct sun6i_csi_device *csi_dev)
> > v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
> > video_device_node_name(video_dev));
> >
> > + /* Media Pad Link */
> > +
> > + ret = media_create_pad_link(&bridge_subdev->entity,
> > + SUN6I_CSI_BRIDGE_PAD_SOURCE,
> > + &video_dev->entity, 0,
> > + MEDIA_LNK_FL_ENABLED |
> > + MEDIA_LNK_FL_IMMUTABLE);
> > + if (ret < 0) {
> > + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n",
> > + bridge_subdev->entity.name,
> > + SUN6I_CSI_BRIDGE_PAD_SOURCE,
> > + video_dev->entity.name, 0);
> > + goto error_video_device;
> > + }
> > +
> > return 0;
> >
> > +error_video_device:
> > + vb2_video_unregister_device(video_dev);
> > +
> > error_media_entity:
> > media_entity_cleanup(&video_dev->entity);
> >
>
> --
> Sakari Ailus
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Laurent,
On Wed 02 Mar 22, 10:51, Laurent Pinchart wrote:
> Hi Paul,
>
> On Tue, Mar 01, 2022 at 04:58:51PM +0100, Paul Kocialkowski wrote:
> > On Mon 07 Feb 22, 18:16, Laurent Pinchart wrote:
> > > On Sat, Feb 05, 2022 at 07:54:26PM +0100, Paul Kocialkowski wrote:
> > > > Some Allwinner platforms come with an Image Signal Processor, which
> > > > supports various features in order to enhance and transform data
> > > > received by image sensors into good-looking pictures. In most cases,
> > > > the data is raw bayer, which gets internally converted to RGB and
> > > > finally YUV, which is what the hardware produces.
> > > >
> > > > This driver supports ISPs that are similar to the A31 ISP, which was
> > > > the first standalone ISP found in Allwinner platforms. Simpler ISP
> > > > blocks were found in the A10 and A20, where they are tied to a CSI
> > > > controller. Newer generations of Allwinner SoCs (starting with the
> > > > H6, H616, etc) come with a new camera subsystem and revised ISP.
> > > > Even though these previous and next-generation ISPs are somewhat
> > > > similar to the A31 ISP, they have enough significant differences to
> > > > be out of the scope of this driver.
> > > >
> > > > While the ISP supports many features, including 3A and many
> > > > enhancement blocks, this implementation is limited to the following:
> > > > - V3s (V3/S3) platform support;
> > > > - Bayer media bus formats as input;
> > >
> > > Greyscale formats would also be nice to have, if the hardware can
> > > support that (it mostly just requires the ability to disable the CFA
> > > interpolation).
> >
> > As far as I know there's no support for grayscale, only bayer formats
> > and YUV.
> >
> > > > - Semi-planar YUV (NV12/NV21) as output;
> > >
> > > Packed YUV would also be useful if the hardware supports it.
> >
> > Same here, it only supports planar and semi-planar YUV as output.
> >
> > > > - Debayering with per-component gain and offset configuration;
> > > > - 2D noise filtering with configurable coefficients.
> > > >
> > > > Since many features are missing from the associated uAPI, the driver
> > > > is aimed to integrate staging until all features are properly
> > > > described.
> > > >
> > > > On the technical side, it uses the v4l2 and media controller APIs,
> > > > with a video node for capture, a processor subdev and a video node
> > > > for parameters submission. A specific uAPI structure and associated
> > > > v4l2 meta format are used to configure parameters of the supported
> > > > modules.
> > > >
> > > > One particular thing about the hardware is that configuration for
> > > > module registers needs to be stored in a DMA buffer and gets copied
> > > > to actual registers by the hardware at the next vsync, when instructed
> > > > by a flag. This is handled by the "state" mechanism in the driver.
> > > >
> > > > Signed-off-by: Paul Kocialkowski <[email protected]>
> > > > ---
> > > > drivers/staging/media/sunxi/Kconfig | 1 +
> > > > drivers/staging/media/sunxi/Makefile | 1 +
> > > > drivers/staging/media/sunxi/sun6i-isp/Kconfig | 13 +
> > > > .../staging/media/sunxi/sun6i-isp/Makefile | 4 +
> > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 572 +++++++++++++
> > > > .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 86 ++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 751 ++++++++++++++++++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 573 +++++++++++++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 599 ++++++++++++++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 61 ++
> > > > .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++
> > > > .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 +
> > >
> > > Could you add a TODO file to list the issues that need to be fixed for
> > > the driver to move out of staging ? I'll already propose one entry:
> > >
> > > - Add support in libcamera
> >
> > Maybe it would be good to narrow down what level of support you have in mind
> > here. Just adding basic support fort the pipeline is probably doable, but
> > developing complex 3A algorithms would require very significant effort and
> > it would be a shame that this prevents the driver from leaving staging.
> >
> > I think another obvious task would be to have a complete uAPI that reflects
> > all modules that are part of the ISP.
> >
> > What do you think?
>
> The reason why I'd like to see libcamera support for the ISP driver is
> to ensure that the kernel API is adequate for real use cases. The API
> can be split in three parts:
>
> - Pipeline configuration (this includes the media controller topology,
> link setting, subdev pad format/selection rectangle configuration,
> ...)
> - ISP parameters
> - ISP statistics
>
> A pipeline handler implementation will cover the first parts. The second
> and third parts need to be tested too, but we don't need to implement
> every single feature. A very simple algorithm that demonstrates
> statistics can be captured and ISP parameters can be set should be
> enough to test and exercise the API in real scenarios. We're working on
> making basic AE and AWB algorithm implementations generic (or at least
> creating generic building blocks that can easily be assembled to create
> those algorithms, as the ISP statistics and parameters are specific to
> the ISP and thus require some ISP-specific code), so that should become
> a fairly easy task soon. I expect most of the work to go in the pipeline
> handler.
>
> Does this sound fair to you ?
Yes I understand that these aspects need to be tested too, but I feel like
having an implementation with a feedback look (even rudimentary and using
some generic helpers) would be quite a stretch.
It would probably be sufficient to have some demo code that can receive stats
and set parameters, but without necessarily any connection between the two.
As such it's also my feeling that a standalone demo program could be easier
to manage for that purpose than libcamera support.
So how about making the requirement that a (free software) userspace
implementration must demonstrate ability to read relevant statistics
from the ISP and (independently) control parameters that affect the output?
Paul
> > > This isn't required to merge the driver in staging as long as ABI
> > > compatibility doesn't need to be preserved until the driver is moved out
> > > of staging.
> > >
> > > > 14 files changed, 3109 insertions(+)
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h
> > > > create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
> > >
> > > [snip]
>
> --
> Regards,
>
> Laurent Pinchart
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
Hi Paul,
On Wed, Mar 02, 2022 at 03:59:50PM +0100, Paul Kocialkowski wrote:
> > > +static int
> > > +sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,
> > > + struct v4l2_subdev *remote_subdev,
> > > + struct v4l2_async_subdev *async_subdev)
> > > +{
> > > + struct sun6i_csi_device *csi_dev =
> > > + container_of(notifier, struct sun6i_csi_device,
> > > + bridge.notifier);
> > > + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> > > + struct sun6i_csi_bridge_source *source = NULL;
> > > + struct fwnode_handle *fwnode = dev_fwnode(csi_dev->dev);
> > > + struct fwnode_handle *handle = NULL;
> > > + bool enabled;
> > > + int ret;
> > > +
> > > + while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
> >
> > I'd instead store the information you need here in struct sun6i_csi_bridge.
> > You could remove the loop here.
>
> Is there a different method for matching a remote subdev to a local port?
> The rationale here is that I need the handle for fwnode_graph_parse_endpoint
> but cannot get that handle from the remote subdev's fwnode pointer directly.
You generally shouldn't try to match fwnodes here as the V4L2 async
framework has already done that job. This information can be found behind
the async_subdev pointer.
See e.g. drivers/media/pci/intel/ipu3/ipu3-cio2-main.c for an example.
>
> > > + struct fwnode_endpoint endpoint = { 0 };
> > > + struct fwnode_handle *remote_fwnode;
> > > +
> > > + remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
> > > + if (!remote_fwnode)
> > > + continue;
> > > +
> > > + if (remote_fwnode != remote_subdev->fwnode)
> > > + goto next;
> > > +
> > > + ret = fwnode_graph_parse_endpoint(handle, &endpoint);
> > > + if (ret < 0)
> > > + goto next;
> > > +
> > > + switch (endpoint.port) {
> > > + case SUN6I_CSI_PORT_PARALLEL:
> > > + source = &bridge->source_parallel;
> > > + enabled = true;
> > > + break;
> > > + default:
> > > + break;
> > > + }
> > > +
> > > +next:
> > > + fwnode_handle_put(remote_fwnode);
> > > + }
> > > +
> > > + if (!source)
> > > + return -EINVAL;
> > > +
> > > + source->subdev = remote_subdev;
> > > +
> > > + return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK,
> > > + remote_subdev, enabled);
> > > +}
> > > +
> > > +static int
> > > +sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier)
> > > +{
> > > + struct sun6i_csi_device *csi_dev =
> > > + container_of(notifier, struct sun6i_csi_device,
> > > + bridge.notifier);
> > > +
> > > + return sun6i_csi_v4l2_complete(csi_dev);
> >
> > You could call v4l2_device_register_subdev_nodes() here.
>
> That's definitely what sun6i_csi_v4l2_complete does (the diff is probably not
> very clear). Note that the wrapper is extended later on to register the capture
> video device for the no-isp path.
I could be missing something... Do you need to call
sun6i_csi_v4l2_complete() in multiple places or not? If not, then I think
it'd be probably better to just move the code here.
>
> Maybe the capture registration could be kept in sun6i_csi_probe for the non-isp
> path and then the wrapper wouldn't be needed. I don't mind either way.
--
Kind regards,
Sakari Ailus
Hi Sakari,
On Fri 04 Mar 22, 00:43, Sakari Ailus wrote:
> Hi Paul,
>
> On Wed, Mar 02, 2022 at 03:59:50PM +0100, Paul Kocialkowski wrote:
> > > > +static int
> > > > +sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,
> > > > + struct v4l2_subdev *remote_subdev,
> > > > + struct v4l2_async_subdev *async_subdev)
> > > > +{
> > > > + struct sun6i_csi_device *csi_dev =
> > > > + container_of(notifier, struct sun6i_csi_device,
> > > > + bridge.notifier);
> > > > + struct sun6i_csi_bridge *bridge = &csi_dev->bridge;
> > > > + struct sun6i_csi_bridge_source *source = NULL;
> > > > + struct fwnode_handle *fwnode = dev_fwnode(csi_dev->dev);
> > > > + struct fwnode_handle *handle = NULL;
> > > > + bool enabled;
> > > > + int ret;
> > > > +
> > > > + while ((handle = fwnode_graph_get_next_endpoint(fwnode, handle))) {
> > >
> > > I'd instead store the information you need here in struct sun6i_csi_bridge.
> > > You could remove the loop here.
> >
> > Is there a different method for matching a remote subdev to a local port?
> > The rationale here is that I need the handle for fwnode_graph_parse_endpoint
> > but cannot get that handle from the remote subdev's fwnode pointer directly.
>
> You generally shouldn't try to match fwnodes here as the V4L2 async
> framework has already done that job. This information can be found behind
> the async_subdev pointer.
>
> See e.g. drivers/media/pci/intel/ipu3/ipu3-cio2-main.c for an example.
Thanks for the feedback, I'll look into that.
> >
> > > > + struct fwnode_endpoint endpoint = { 0 };
> > > > + struct fwnode_handle *remote_fwnode;
> > > > +
> > > > + remote_fwnode = fwnode_graph_get_remote_port_parent(handle);
> > > > + if (!remote_fwnode)
> > > > + continue;
> > > > +
> > > > + if (remote_fwnode != remote_subdev->fwnode)
> > > > + goto next;
> > > > +
> > > > + ret = fwnode_graph_parse_endpoint(handle, &endpoint);
> > > > + if (ret < 0)
> > > > + goto next;
> > > > +
> > > > + switch (endpoint.port) {
> > > > + case SUN6I_CSI_PORT_PARALLEL:
> > > > + source = &bridge->source_parallel;
> > > > + enabled = true;
> > > > + break;
> > > > + default:
> > > > + break;
> > > > + }
> > > > +
> > > > +next:
> > > > + fwnode_handle_put(remote_fwnode);
> > > > + }
> > > > +
> > > > + if (!source)
> > > > + return -EINVAL;
> > > > +
> > > > + source->subdev = remote_subdev;
> > > > +
> > > > + return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK,
> > > > + remote_subdev, enabled);
> > > > +}
> > > > +
> > > > +static int
> > > > +sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier)
> > > > +{
> > > > + struct sun6i_csi_device *csi_dev =
> > > > + container_of(notifier, struct sun6i_csi_device,
> > > > + bridge.notifier);
> > > > +
> > > > + return sun6i_csi_v4l2_complete(csi_dev);
> > >
> > > You could call v4l2_device_register_subdev_nodes() here.
> >
> > That's definitely what sun6i_csi_v4l2_complete does (the diff is probably not
> > very clear). Note that the wrapper is extended later on to register the capture
> > video device for the no-isp path.
>
> I could be missing something... Do you need to call
> sun6i_csi_v4l2_complete() in multiple places or not? If not, then I think
> it'd be probably better to just move the code here.
No this is only called here so I guess we can avoid it entirely.
Thanks,
Paul
> >
> > Maybe the capture registration could be kept in sun6i_csi_probe for the non-isp
> > path and then the wrapper wouldn't be needed. I don't mind either way.
>
> --
> Kind regards,
>
> Sakari Ailus
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com