This series adds the support of the Mediatek ISP3.0 found on some
Mediatek SoCs such as the mt8365. The driver is divided into 2 parts:
* SENINF: the sensor interface
* CAMSV: this driver provides a path to bypass the SoC ISP so that image
data coming from the SENINF can go directly into memory without any
image processing. This allows the use of an external ISP or camera
sensor directly.
The SENINF driver is based on previous work done by Louis Kuo available
as an RFC here: https://lore.kernel.org/all/[email protected]/
This series depends on the following series for the phy [1]
Changes in v4:
- fix suspend/resume deadlock
- fix various locking issues reported by Laurent Pinchart on v3
- run LOCKDEP
- add missing include reported by kernel-test-robot for non mediatek arch and COMPILE_TEST=y
- use atomic poll inside mtk_camsv30_setup
- drop second lane support as it was not used
- remove useless members in structs
- fix media entity initialization
- initialize correct pad for camsv video device
- add isp support in mt8365.dtsi
- rebase on 6.7
Changes in v3:
- fix a lot of formatting issues/coding style issues found in camsv/seninf reported by Angelo on v2
- fix camsv/seninf binding file error reported by Rob
Changes in v2:
- renamed clock `cam_seninf` to `camsys`
- renamed clock `top_mux_seninf` to `top_mux`
- moved phy properties from port nodes to top level
- remove patternProperties
- specify power management dependency in the cover letter description to fix
missing include in dt-binding example
- change '$ref' properties on some endpoint nodes from
'$ref: video-interfaces.yaml#' to '$ref: /schemas/graph.yaml#/$defs/endpoint-base'
where applicable
Best
Julien Stephan
[1] : https://lore.kernel.org/all/[email protected]/
Louis Kuo (2):
dt-bindings: media: add mediatek ISP3.0 sensor interface
media: platform: mediatek: isp_30: add mediatek ISP3.0 sensor
interface
Phi-bang Nguyen (2):
dt-bindings: media: add mediatek ISP3.0 camsv
media: platform: mediatek: isp_30: add mediatek ISP3.0 camsv
.../bindings/media/mediatek,mt8365-camsv.yaml | 109 ++
.../media/mediatek,mt8365-seninf.yaml | 259 +++
MAINTAINERS | 10 +
arch/arm64/boot/dts/mediatek/mt8365.dtsi | 128 ++
drivers/media/platform/mediatek/Kconfig | 1 +
drivers/media/platform/mediatek/Makefile | 1 +
drivers/media/platform/mediatek/isp/Kconfig | 2 +
drivers/media/platform/mediatek/isp/Makefile | 3 +
.../platform/mediatek/isp/isp_30/Kconfig | 35 +
.../platform/mediatek/isp/isp_30/Makefile | 4 +
.../mediatek/isp/isp_30/camsv/Makefile | 7 +
.../mediatek/isp/isp_30/camsv/mtk_camsv.c | 328 ++++
.../mediatek/isp/isp_30/camsv/mtk_camsv.h | 199 +++
.../isp/isp_30/camsv/mtk_camsv30_hw.c | 427 +++++
.../isp/isp_30/camsv/mtk_camsv30_regs.h | 60 +
.../isp/isp_30/camsv/mtk_camsv_video.c | 774 +++++++++
.../mediatek/isp/isp_30/seninf/Makefile | 5 +
.../mediatek/isp/isp_30/seninf/mtk_seninf.c | 1488 +++++++++++++++++
.../isp/isp_30/seninf/mtk_seninf_reg.h | 112 ++
19 files changed, 3952 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8365-camsv.yaml
create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8365-seninf.yaml
create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
create mode 100644 drivers/media/platform/mediatek/isp/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/Kconfig
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv30_hw.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv30_regs.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv_video.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/mtk_seninf.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/mtk_seninf_reg.h
--
2.43.0
Add base support for cameras for mt8365 platforms. This requires nodes
for the sensor interface, camsv, and CSI receivers.
Signed-off-by: Julien Stephan <[email protected]>
---
arch/arm64/boot/dts/mediatek/mt8365.dtsi | 128 +++++++++++++++++++++++
1 file changed, 128 insertions(+)
diff --git a/arch/arm64/boot/dts/mediatek/mt8365.dtsi b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
index 24581f7410aa..9059b2f83b83 100644
--- a/arch/arm64/boot/dts/mediatek/mt8365.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
@@ -10,6 +10,7 @@
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/phy/phy.h>
#include <dt-bindings/power/mediatek,mt8365-power.h>
+#include <dt-bindings/memory/mediatek,mt8365-larb-port.h>
/ {
compatible = "mediatek,mt8365";
@@ -703,6 +704,133 @@ ethernet: ethernet@112a0000 {
status = "disabled";
};
+ camsv1: camsv@15050000 {
+ compatible = "mediatek,mt8365-camsv";
+ reg = <0 0x15050000 0 0x0040>,
+ <0 0x15050208 0 0x0020>,
+ <0 0x15050400 0 0x0100>;
+ interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&camsys CLK_CAM>,
+ <&camsys CLK_CAMTG>,
+ <&camsys CLK_CAMSV0>;
+ clock-names = "cam", "camtg", "camsv";
+ iommus = <&iommu M4U_PORT_CAM_IMGO>;
+ mediatek,larb = <&larb2>;
+ power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ camsv1_endpoint: endpoint {
+ remote-endpoint =
+ <&seninf_camsv1_endpoint>;
+ };
+ };
+ };
+ };
+
+ camsv2: camsv@15050800 {
+ compatible = "mediatek,mt8365-camsv";
+ reg = <0 0x15050800 0 0x0040>,
+ <0 0x15050228 0 0x0020>,
+ <0 0x15050C00 0 0x0100>;
+ interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&camsys CLK_CAM>,
+ <&camsys CLK_CAMTG>,
+ <&camsys CLK_CAMSV1>;
+ clock-names = "cam", "camtg", "camsv";
+ iommus = <&iommu M4U_PORT_CAM_IMGO>;
+
+ mediatek,larb = <&larb2>;
+ power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ camsv2_endpoint: endpoint {
+ remote-endpoint =
+ <&seninf_camsv2_endpoint>;
+ };
+ };
+ };
+ };
+
+ seninf: seninf@15040000 {
+ compatible = "mediatek,mt8365-seninf";
+ reg = <0 0x15040000 0 0x6000>;
+ interrupts = <GIC_SPI 210 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&camsys CLK_CAM_SENIF>,
+ <&topckgen CLK_TOP_SENIF_SEL>;
+ clock-names = "camsys", "top_mux";
+
+ power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
+
+ phys = <&mipi_csi0 PHY_TYPE_DPHY>, <&mipi_csi1>;
+ phy-names = "csi0", "csi1";
+
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ };
+
+ port@1 {
+ reg = <1>;
+ };
+
+ port@2 {
+ reg = <2>;
+ };
+
+ port@3 {
+ reg = <3>;
+ };
+
+ port@4 {
+ reg = <4>;
+ seninf_camsv1_endpoint: endpoint {
+ remote-endpoint =
+ <&camsv1_endpoint>;
+ };
+ };
+
+ port@5 {
+ reg = <5>;
+ seninf_camsv2_endpoint: endpoint {
+ remote-endpoint =
+ <&camsv2_endpoint>;
+ };
+ };
+ };
+ };
+
+ mipi_csi0: mipi-csi0@11c10000 {
+ compatible = "mediatek,mt8365-csi-rx";
+ reg = <0 0x11C10000 0 0x2000>;
+ status = "disabled";
+ num-lanes = <4>;
+ #phy-cells = <1>;
+ };
+
+ mipi_csi1: mipi-csi1@11c12000 {
+ compatible = "mediatek,mt8365-csi-rx";
+ reg = <0 0x11C12000 0 0x2000>;
+ phy-type = <PHY_TYPE_DPHY>;
+ status = "disabled";
+ num-lanes = <4>;
+ #phy-cells = <0>;
+ };
+
u3phy: t-phy@11cc0000 {
compatible = "mediatek,mt8365-tphy", "mediatek,generic-tphy-v2";
#address-cells = <1>;
--
2.43.0
Hi Julien,
Thank you for the patches.
On Wed, Jan 10, 2024 at 03:14:37PM +0100, Julien Stephan wrote:
> This series adds the support of the Mediatek ISP3.0 found on some
> Mediatek SoCs such as the mt8365. The driver is divided into 2 parts:
>
> * SENINF: the sensor interface
> * CAMSV: this driver provides a path to bypass the SoC ISP so that image
> data coming from the SENINF can go directly into memory without any
> image processing. This allows the use of an external ISP or camera
> sensor directly.
I'm afraid this doesn't compile based on the latest media tree's master
branch, scheduled for v6.8-rc1. I've pushed the following fixes to [1]:
0d2e75fb8909 media: mediatek: isp_30: Update to new subdev state API
05dc498f151a media: mediatek: isp_30: Update to new subdev state API
You can squash those in v5 when you will rebase on v6.8-rc1.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=mtk/v6.8/pumpkin/camera
> The SENINF driver is based on previous work done by Louis Kuo available
> as an RFC here: https://lore.kernel.org/all/[email protected]/
>
> This series depends on the following series for the phy [1]
>
> Changes in v4:
> - fix suspend/resume deadlock
> - fix various locking issues reported by Laurent Pinchart on v3
> - run LOCKDEP
> - add missing include reported by kernel-test-robot for non mediatek arch and COMPILE_TEST=y
> - use atomic poll inside mtk_camsv30_setup
> - drop second lane support as it was not used
> - remove useless members in structs
> - fix media entity initialization
> - initialize correct pad for camsv video device
> - add isp support in mt8365.dtsi
> - rebase on 6.7
>
> Changes in v3:
> - fix a lot of formatting issues/coding style issues found in camsv/seninf reported by Angelo on v2
> - fix camsv/seninf binding file error reported by Rob
>
> Changes in v2:
> - renamed clock `cam_seninf` to `camsys`
> - renamed clock `top_mux_seninf` to `top_mux`
> - moved phy properties from port nodes to top level
> - remove patternProperties
> - specify power management dependency in the cover letter description to fix
> missing include in dt-binding example
> - change '$ref' properties on some endpoint nodes from
> '$ref: video-interfaces.yaml#' to '$ref: /schemas/graph.yaml#/$defs/endpoint-base'
> where applicable
>
> Best
> Julien Stephan
>
> [1] : https://lore.kernel.org/all/[email protected]/
>
>
>
> Louis Kuo (2):
> dt-bindings: media: add mediatek ISP3.0 sensor interface
> media: platform: mediatek: isp_30: add mediatek ISP3.0 sensor
> interface
>
> Phi-bang Nguyen (2):
> dt-bindings: media: add mediatek ISP3.0 camsv
> media: platform: mediatek: isp_30: add mediatek ISP3.0 camsv
>
> .../bindings/media/mediatek,mt8365-camsv.yaml | 109 ++
> .../media/mediatek,mt8365-seninf.yaml | 259 +++
> MAINTAINERS | 10 +
> arch/arm64/boot/dts/mediatek/mt8365.dtsi | 128 ++
> drivers/media/platform/mediatek/Kconfig | 1 +
> drivers/media/platform/mediatek/Makefile | 1 +
> drivers/media/platform/mediatek/isp/Kconfig | 2 +
> drivers/media/platform/mediatek/isp/Makefile | 3 +
> .../platform/mediatek/isp/isp_30/Kconfig | 35 +
> .../platform/mediatek/isp/isp_30/Makefile | 4 +
> .../mediatek/isp/isp_30/camsv/Makefile | 7 +
> .../mediatek/isp/isp_30/camsv/mtk_camsv.c | 328 ++++
> .../mediatek/isp/isp_30/camsv/mtk_camsv.h | 199 +++
> .../isp/isp_30/camsv/mtk_camsv30_hw.c | 427 +++++
> .../isp/isp_30/camsv/mtk_camsv30_regs.h | 60 +
> .../isp/isp_30/camsv/mtk_camsv_video.c | 774 +++++++++
> .../mediatek/isp/isp_30/seninf/Makefile | 5 +
> .../mediatek/isp/isp_30/seninf/mtk_seninf.c | 1488 +++++++++++++++++
> .../isp/isp_30/seninf/mtk_seninf_reg.h | 112 ++
> 19 files changed, 3952 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8365-camsv.yaml
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8365-seninf.yaml
> create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
> create mode 100644 drivers/media/platform/mediatek/isp/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/Kconfig
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv30_hw.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv30_regs.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/camsv/mtk_camsv_video.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/mtk_seninf.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_30/seninf/mtk_seninf_reg.h
--
Regards,
Laurent Pinchart
Hi Julien,
Thank you for the patch.
On Wed, Jan 10, 2024 at 03:14:42PM +0100, Julien Stephan wrote:
> Add base support for cameras for mt8365 platforms. This requires nodes
> for the sensor interface, camsv, and CSI receivers.
>
> Signed-off-by: Julien Stephan <[email protected]>
> ---
> arch/arm64/boot/dts/mediatek/mt8365.dtsi | 128 +++++++++++++++++++++++
> 1 file changed, 128 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/mediatek/mt8365.dtsi b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
> index 24581f7410aa..9059b2f83b83 100644
> --- a/arch/arm64/boot/dts/mediatek/mt8365.dtsi
> +++ b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
> @@ -10,6 +10,7 @@
> #include <dt-bindings/interrupt-controller/irq.h>
> #include <dt-bindings/phy/phy.h>
> #include <dt-bindings/power/mediatek,mt8365-power.h>
> +#include <dt-bindings/memory/mediatek,mt8365-larb-port.h>
>
> / {
> compatible = "mediatek,mt8365";
> @@ -703,6 +704,133 @@ ethernet: ethernet@112a0000 {
> status = "disabled";
> };
>
> + camsv1: camsv@15050000 {
> + compatible = "mediatek,mt8365-camsv";
> + reg = <0 0x15050000 0 0x0040>,
> + <0 0x15050208 0 0x0020>,
> + <0 0x15050400 0 0x0100>;
> + interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_LOW>;
> + clocks = <&camsys CLK_CAM>,
> + <&camsys CLK_CAMTG>,
> + <&camsys CLK_CAMSV0>;
> + clock-names = "cam", "camtg", "camsv";
> + iommus = <&iommu M4U_PORT_CAM_IMGO>;
> + mediatek,larb = <&larb2>;
> + power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
> + status = "disabled";
> +
> + ports {
> + #address-cells = <1>;
> + #size-cells = <0>;
> + port@0 {
> + reg = <0>;
> + camsv1_endpoint: endpoint {
> + remote-endpoint =
> + <&seninf_camsv1_endpoint>;
I think you can keep this on a single line. Same below.
> + };
> + };
> + };
> + };
> +
> + camsv2: camsv@15050800 {
> + compatible = "mediatek,mt8365-camsv";
> + reg = <0 0x15050800 0 0x0040>,
> + <0 0x15050228 0 0x0020>,
> + <0 0x15050C00 0 0x0100>;
Lower-case hex constants.
> + interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>;
> + clocks = <&camsys CLK_CAM>,
> + <&camsys CLK_CAMTG>,
> + <&camsys CLK_CAMSV1>;
> + clock-names = "cam", "camtg", "camsv";
> + iommus = <&iommu M4U_PORT_CAM_IMGO>;
> +
Nitpicking, camsv doesn't have a blank line here. I'm fine either way,
but please do the same for both nodes.
> + mediatek,larb = <&larb2>;
> + power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
> + status = "disabled";
> +
> + ports {
> + #address-cells = <1>;
> + #size-cells = <0>;
> + port@0 {
> + reg = <0>;
> + camsv2_endpoint: endpoint {
> + remote-endpoint =
> + <&seninf_camsv2_endpoint>;
> + };
> + };
> + };
> + };
> +
> + seninf: seninf@15040000 {
This should go before the two camsv instances to keep nodes sorted by
address. The camsv1, camsv2 and seninf nodes need to be moved further
down for the same reason.
> + compatible = "mediatek,mt8365-seninf";
> + reg = <0 0x15040000 0 0x6000>;
> + interrupts = <GIC_SPI 210 IRQ_TYPE_LEVEL_LOW>;
> + clocks = <&camsys CLK_CAM_SENIF>,
> + <&topckgen CLK_TOP_SENIF_SEL>;
> + clock-names = "camsys", "top_mux";
> +
> + power-domains = <&spm MT8365_POWER_DOMAIN_CAM>;
> +
> + phys = <&mipi_csi0 PHY_TYPE_DPHY>, <&mipi_csi1>;
> + phy-names = "csi0", "csi1";
> +
> + status = "disabled";
> +
> + ports {
> + #address-cells = <1>;
> + #size-cells = <0>;
> +
> + port@0 {
> + reg = <0>;
> + };
> +
> + port@1 {
> + reg = <1>;
> + };
> +
> + port@2 {
> + reg = <2>;
> + };
> +
> + port@3 {
> + reg = <3>;
> + };
> +
> + port@4 {
> + reg = <4>;
> + seninf_camsv1_endpoint: endpoint {
> + remote-endpoint =
> + <&camsv1_endpoint>;
> + };
> + };
> +
> + port@5 {
> + reg = <5>;
> + seninf_camsv2_endpoint: endpoint {
> + remote-endpoint =
> + <&camsv2_endpoint>;
> + };
> + };
> + };
> + };
> +
> + mipi_csi0: mipi-csi0@11c10000 {
> + compatible = "mediatek,mt8365-csi-rx";
> + reg = <0 0x11C10000 0 0x2000>;
Lower-case hex constants.
> + status = "disabled";
> + num-lanes = <4>;
> + #phy-cells = <1>;
> + };
> +
> + mipi_csi1: mipi-csi1@11c12000 {
> + compatible = "mediatek,mt8365-csi-rx";
> + reg = <0 0x11C12000 0 0x2000>;
Lower-case hex constants.
Reviewed-by: Laurent Pinchart <[email protected]>
> + phy-type = <PHY_TYPE_DPHY>;
> + status = "disabled";
> + num-lanes = <4>;
> + #phy-cells = <0>;
> + };
> +
> u3phy: t-phy@11cc0000 {
> compatible = "mediatek,mt8365-tphy", "mediatek,generic-tphy-v2";
> #address-cells = <1>;
--
Regards,
Laurent Pinchart