2022-09-11 20:05:06

by Mikhail Rudenko

[permalink] [raw]
Subject: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hello,

this series implements support for Omnivision OV4689 image
sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
megapixel image sensor. Ihis chip supports high frame rate speeds up
to 90 fps at 2688x1520 resolution. It is programmable through an I2C
interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
connection.

The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.

While porting the driver, I stumbled upon two issues:

(1) In the original driver, horizontal total size (HTS) was set to a
value (2584) lower then the frame width (2688), resulting in negative
hblank. In this driver, I increased HTS to 2688, but fps dropped from
29.88 to 28.73. What is the preferred way to handle this?

(2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
gain is not linear across that range. Instead, it is piecewise linear
(and discontinuous). 0x0-0xff register values result in 0x-2x gain,
0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
with more linear segments in between. Rockchip's camera engine code
chooses one of the above segments depenging on the desired gain
value. The question is, how should we proceed keeping in mind
libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
libcamera will do the mapping, or the driver will do the mapping
itself and expose some logical gain units not tied to the actual gain
register value? Meanwhile, this driver conservatively exposes only
0x0-0xf8 gain register range.

[1] https://github.com/rockchip-linux/kernel/blob/develop-4.19/drivers/media/i2c/ov4689.c

changes in v2:
- bindings: reword descriptions
- bindings: move clock description to clocks property
- bindings: add data-lanes and link-frequencies properties to port
- driver: validate media bus configuration when probing

Mikhail Rudenko (2):
media: dt-bindings: media: i2c: document OV4689 DT bindings
media: i2c: add support for ov4689

.../bindings/media/i2c/ovti,ov4689.yaml | 141 +++
MAINTAINERS | 8 +
drivers/media/i2c/Kconfig | 14 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov4689.c | 951 ++++++++++++++++++
5 files changed, 1115 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
create mode 100644 drivers/media/i2c/ov4689.c

--
2.37.3


2022-09-11 20:05:05

by Mikhail Rudenko

[permalink] [raw]
Subject: [PATCH v2 2/2] media: i2c: add support for ov4689

Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
bus for data.

This driver supports following features:
- manual exposure and analog gain control support
- test pattern support
- media controller support
- runtime PM support
- support following resolutions:
+ 2688x1520 at 30 fps

The driver provides all mandatory V4L2 controls for compatibility with
libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
implements 4 lane mode only at this moment.

Signed-off-by: Mikhail Rudenko <[email protected]>
---
MAINTAINERS | 1 +
drivers/media/i2c/Kconfig | 14 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
4 files changed, 967 insertions(+)
create mode 100644 drivers/media/i2c/ov4689.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 63c4844f26e6..1857f3864e1b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14529,6 +14529,7 @@ L: [email protected]
S: Maintained
T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
+F: drivers/media/i2c/ov5647.c

OMNIVISION OV5640 SENSOR DRIVER
M: Steve Longerbeam <[email protected]>
diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index fae2baabb773..4993e1ae2ea8 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -429,6 +429,20 @@ config VIDEO_OV2740
To compile this driver as a module, choose M here: the
module will be called ov2740.

+config VIDEO_OV4689
+ tristate "OmniVision OV4689 sensor support"
+ depends on OF
+ depends on GPIOLIB && VIDEO_DEV && I2C
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select V4L2_FWNODE
+ help
+ This is a Video4Linux2 sensor-level driver for the OmniVision
+ OV4689 camera.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ov4689.
+
config VIDEO_OV5640
tristate "OmniVision OV5640 sensor support"
depends on OF
diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
index 3e1696963e7f..7446c0a1eed0 100644
--- a/drivers/media/i2c/Makefile
+++ b/drivers/media/i2c/Makefile
@@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
+obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
new file mode 100644
index 000000000000..9f05e812acf8
--- /dev/null
+++ b/drivers/media/i2c/ov4689.c
@@ -0,0 +1,951 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ov4689 driver
+ *
+ * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
+ */
+
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/regulator/consumer.h>
+#include <media/media-entity.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-fwnode.h>
+
+#define CHIP_ID 0x004688
+#define OV4689_REG_CHIP_ID 0x300a
+
+#define OV4689_XVCLK_FREQ 24000000
+
+#define OV4689_REG_CTRL_MODE 0x0100
+#define OV4689_MODE_SW_STANDBY 0x0
+#define OV4689_MODE_STREAMING BIT(0)
+
+#define OV4689_REG_EXPOSURE 0x3500
+#define OV4689_EXPOSURE_MIN 4
+#define OV4689_EXPOSURE_STEP 1
+#define OV4689_VTS_MAX 0x7fff
+
+#define OV4689_REG_GAIN_H 0x3508
+#define OV4689_REG_GAIN_L 0x3509
+#define OV4689_GAIN_H_MASK 0x07
+#define OV4689_GAIN_H_SHIFT 8
+#define OV4689_GAIN_L_MASK 0xff
+#define OV4689_GAIN_MIN 0x10
+#define OV4689_GAIN_MAX 0xf8
+#define OV4689_GAIN_STEP 1
+#define OV4689_GAIN_DEFAULT 0x10
+
+#define OV4689_REG_TEST_PATTERN 0x5040
+#define OV4689_TEST_PATTERN_ENABLE 0x80
+#define OV4689_TEST_PATTERN_DISABLE 0x0
+
+#define OV4689_REG_VTS 0x380e
+
+#define REG_NULL 0xFFFF
+
+#define OV4689_REG_VALUE_08BIT 1
+#define OV4689_REG_VALUE_16BIT 2
+#define OV4689_REG_VALUE_24BIT 3
+
+#define OV4689_LANES 4
+#define OV4689_BITS_PER_SAMPLE 10
+
+static const char *const ov4689_supply_names[] = {
+ "avdd", /* Analog power */
+ "dovdd", /* Digital I/O power */
+ "dvdd", /* Digital core power */
+};
+
+#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
+
+struct regval {
+ u16 addr;
+ u8 val;
+};
+
+struct ov4689_mode {
+ u32 width;
+ u32 height;
+ u32 max_fps;
+ u32 hts_def;
+ u32 vts_def;
+ u32 exp_def;
+ const struct regval *reg_list;
+};
+
+struct ov4689 {
+ struct i2c_client *client;
+ struct clk *xvclk;
+ struct gpio_desc *reset_gpio;
+ struct gpio_desc *pwdn_gpio;
+ struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
+
+ struct v4l2_subdev subdev;
+ struct media_pad pad;
+
+ struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
+ bool streaming;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct v4l2_ctrl *exposure;
+ struct v4l2_ctrl *anal_gain;
+ struct v4l2_ctrl *digi_gain;
+ struct v4l2_ctrl *hblank;
+ struct v4l2_ctrl *vblank;
+ struct v4l2_ctrl *test_pattern;
+
+ const struct ov4689_mode *cur_mode;
+};
+
+#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
+
+/*
+ * Xclk 24Mhz
+ */
+static const struct regval ov4689_global_regs[] = {
+ { REG_NULL, 0x00 },
+};
+
+/*
+ * Xclk 24Mhz
+ * max_framerate 30fps
+ * mipi_datarate per lane 1008Mbps
+ */
+static const struct regval ov4689_2688x1520_regs[] = {
+ {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
+ {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
+ {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
+ {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
+ {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
+ {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
+ {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
+ {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
+ {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
+ {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
+ {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
+ {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
+ {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
+ {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
+ {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
+ {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
+ {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
+ {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
+ {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
+ {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
+ {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
+ {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
+ {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
+ {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
+ {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
+ {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
+ {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
+ {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
+ {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
+ {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
+ {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
+ {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
+ {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
+ {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
+ {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
+ {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
+ {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
+ {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
+ {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
+ {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
+ {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
+ {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
+ {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
+ {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
+ {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
+ {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
+ {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
+ {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
+ {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
+ {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
+ {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
+ {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
+ {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
+ {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
+ {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
+ {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
+ {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
+ {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
+ {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
+ {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
+ {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
+ {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
+ {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
+ {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
+ {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
+ {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
+ {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
+ {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
+ {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
+ {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
+ {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
+ {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
+ {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
+ {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
+ {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
+ {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
+ {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
+ {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
+ {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
+ {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
+ {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
+ {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
+ {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
+ {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
+ {REG_NULL, 0x00},
+};
+
+static const struct ov4689_mode supported_modes[] = {
+ {
+ .width = 2688,
+ .height = 1520,
+ .max_fps = 30,
+ .exp_def = 0x0600,
+ .hts_def = 0x0a80,
+ .vts_def = 0x0612,
+ .reg_list = ov4689_2688x1520_regs,
+ },
+};
+
+#define OV4689_LINK_FREQ_500MHZ 500000000
+static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
+
+static const char *const ov4689_test_pattern_menu[] = {
+ "Disabled",
+ "Vertical Color Bar Type 1",
+ "Vertical Color Bar Type 2",
+ "Vertical Color Bar Type 3",
+ "Vertical Color Bar Type 4"
+};
+
+/* Write registers up to 4 at a time */
+static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
+ u32 val)
+{
+ u32 buf_i, val_i;
+ __be32 val_be;
+ u8 *val_p;
+ u8 buf[6];
+
+ if (len > 4)
+ return -EINVAL;
+
+ buf[0] = reg >> 8;
+ buf[1] = reg & 0xff;
+
+ val_be = cpu_to_be32(val);
+ val_p = (u8 *)&val_be;
+ buf_i = 2;
+ val_i = 4 - len;
+
+ while (val_i < 4)
+ buf[buf_i++] = val_p[val_i++];
+
+ if (i2c_master_send(client, buf, len + 2) != len + 2)
+ return -EIO;
+
+ return 0;
+}
+
+static int ov4689_write_array(struct i2c_client *client,
+ const struct regval *regs)
+{
+ int ret = 0;
+ u32 i;
+
+ for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
+ ret = ov4689_write_reg(client, regs[i].addr,
+ OV4689_REG_VALUE_08BIT, regs[i].val);
+
+ return ret;
+}
+
+/* Read registers up to 4 at a time */
+static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
+ u32 *val)
+{
+ __be16 reg_addr_be = cpu_to_be16(reg);
+ struct i2c_msg msgs[2];
+ __be32 data_be = 0;
+ u8 *data_be_p;
+ int ret;
+
+ if (len > 4 || !len)
+ return -EINVAL;
+
+ data_be_p = (u8 *)&data_be;
+ /* Write register address */
+ msgs[0].addr = client->addr;
+ msgs[0].flags = 0;
+ msgs[0].len = 2;
+ msgs[0].buf = (u8 *)&reg_addr_be;
+
+ /* Read data from register */
+ msgs[1].addr = client->addr;
+ msgs[1].flags = I2C_M_RD;
+ msgs[1].len = len;
+ msgs[1].buf = &data_be_p[4 - len];
+
+ ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
+ if (ret != ARRAY_SIZE(msgs))
+ return -EIO;
+
+ *val = be32_to_cpu(data_be);
+
+ return 0;
+}
+
+static void ov4689_fill_fmt(const struct ov4689_mode *mode,
+ struct v4l2_mbus_framefmt *fmt)
+{
+ fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
+ fmt->width = mode->width;
+ fmt->height = mode->height;
+ fmt->field = V4L2_FIELD_NONE;
+}
+
+static int ov4689_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
+ struct ov4689 *ov4689 = to_ov4689(sd);
+
+ /* only one mode supported for now */
+ ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
+
+ return 0;
+}
+
+static int ov4689_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
+ struct ov4689 *ov4689 = to_ov4689(sd);
+
+ /* only one mode supported for now */
+ ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
+
+ return 0;
+}
+
+static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->index != 0)
+ return -EINVAL;
+ code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
+
+ return 0;
+}
+
+static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->index >= ARRAY_SIZE(supported_modes))
+ return -EINVAL;
+
+ if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
+ return -EINVAL;
+
+ fse->min_width = supported_modes[fse->index].width;
+ fse->max_width = supported_modes[fse->index].width;
+ fse->max_height = supported_modes[fse->index].height;
+ fse->min_height = supported_modes[fse->index].height;
+
+ return 0;
+}
+
+static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
+{
+ u32 val;
+
+ if (pattern)
+ val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
+ else
+ val = OV4689_TEST_PATTERN_DISABLE;
+
+ return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
+ OV4689_REG_VALUE_08BIT, val);
+}
+
+static int ov4689_get_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
+ return -EINVAL;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ sel->r.top = 0;
+ sel->r.left = 0;
+ sel->r.width = 2720;
+ sel->r.height = 1536;
+ return 0;
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ sel->r.top = 8;
+ sel->r.left = 16;
+ sel->r.width = 2688;
+ sel->r.height = 1520;
+ return 0;
+ }
+ return -EINVAL;
+}
+
+static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
+{
+ struct ov4689 *ov4689 = to_ov4689(sd);
+ struct i2c_client *client = ov4689->client;
+ int ret = 0;
+
+ mutex_lock(&ov4689->mutex);
+
+ on = !!on;
+ if (on == ov4689->streaming)
+ goto unlock_and_return;
+
+ if (on) {
+ ret = pm_runtime_resume_and_get(&client->dev);
+ if (ret < 0)
+ goto unlock_and_return;
+
+ ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
+ if (ret) {
+ pm_runtime_put(&client->dev);
+ goto unlock_and_return;
+ }
+
+ ret = ov4689_write_array(ov4689->client,
+ ov4689->cur_mode->reg_list);
+ if (ret) {
+ pm_runtime_put(&client->dev);
+ goto unlock_and_return;
+ }
+
+ ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
+ OV4689_REG_VALUE_08BIT,
+ OV4689_MODE_STREAMING);
+ if (ret) {
+ pm_runtime_put(&client->dev);
+ goto unlock_and_return;
+ }
+ } else {
+ ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
+ OV4689_REG_VALUE_08BIT,
+ OV4689_MODE_SW_STANDBY);
+ pm_runtime_put(&client->dev);
+ }
+
+ ov4689->streaming = on;
+
+unlock_and_return:
+ mutex_unlock(&ov4689->mutex);
+
+ return ret;
+}
+
+/* Calculate the delay in us by clock rate and clock cycles */
+static inline u32 ov4689_cal_delay(u32 cycles)
+{
+ return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
+}
+
+static int __ov4689_power_on(struct ov4689 *ov4689)
+{
+ struct device *dev = &ov4689->client->dev;
+ u32 delay_us;
+ int ret;
+
+ ret = clk_prepare_enable(ov4689->xvclk);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable xvclk\n");
+ return ret;
+ }
+
+ gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
+
+ ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable regulators\n");
+ goto disable_clk;
+ }
+
+ gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
+ usleep_range(500, 1000);
+ gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
+
+ /* 8192 cycles prior to first SCCB transaction */
+ delay_us = ov4689_cal_delay(8192);
+ usleep_range(delay_us, delay_us * 2);
+
+ return 0;
+
+disable_clk:
+ clk_disable_unprepare(ov4689->xvclk);
+
+ return ret;
+}
+
+static void __ov4689_power_off(struct ov4689 *ov4689)
+{
+ gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
+ clk_disable_unprepare(ov4689->xvclk);
+ gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
+ regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
+}
+
+static int __maybe_unused ov4689_runtime_resume(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct ov4689 *ov4689 = to_ov4689(sd);
+
+ return __ov4689_power_on(ov4689);
+}
+
+static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct ov4689 *ov4689 = to_ov4689(sd);
+
+ __ov4689_power_off(ov4689);
+
+ return 0;
+}
+
+#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
+static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+ struct ov4689 *ov4689 = to_ov4689(sd);
+ struct v4l2_mbus_framefmt *try_fmt;
+
+ mutex_lock(&ov4689->mutex);
+
+ try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
+ /* Initialize try_fmt */
+ ov4689_fill_fmt(&supported_modes[0], try_fmt);
+
+ mutex_unlock(&ov4689->mutex);
+
+ return 0;
+}
+#endif
+
+static const struct dev_pm_ops ov4689_pm_ops = {
+ SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
+};
+
+#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
+static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
+ .open = ov4689_open,
+};
+#endif
+
+static const struct v4l2_subdev_video_ops ov4689_video_ops = {
+ .s_stream = ov4689_s_stream,
+};
+
+static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
+ .enum_mbus_code = ov4689_enum_mbus_code,
+ .enum_frame_size = ov4689_enum_frame_sizes,
+ .get_fmt = ov4689_get_fmt,
+ .set_fmt = ov4689_set_fmt,
+ .get_selection = ov4689_get_selection,
+};
+
+static const struct v4l2_subdev_ops ov4689_subdev_ops = {
+ .video = &ov4689_video_ops,
+ .pad = &ov4689_pad_ops,
+};
+
+static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct ov4689 *ov4689 =
+ container_of(ctrl->handler, struct ov4689, ctrl_handler);
+ struct i2c_client *client = ov4689->client;
+ s64 max_expo;
+ int ret;
+
+ /* Propagate change of current control to all related controls */
+ switch (ctrl->id) {
+ case V4L2_CID_VBLANK:
+ /* Update max exposure while meeting expected vblanking */
+ max_expo = ov4689->cur_mode->height + ctrl->val - 4;
+ __v4l2_ctrl_modify_range(ov4689->exposure,
+ ov4689->exposure->minimum, max_expo,
+ ov4689->exposure->step,
+ ov4689->exposure->default_value);
+ break;
+ }
+
+ if (!pm_runtime_get_if_in_use(&client->dev))
+ return 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_EXPOSURE:
+ /* 4 least significant bits of expsoure are fractional part */
+ ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
+ OV4689_REG_VALUE_24BIT, ctrl->val << 4);
+ break;
+ case V4L2_CID_ANALOGUE_GAIN:
+ ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
+ OV4689_REG_VALUE_08BIT,
+ (ctrl->val >> OV4689_GAIN_H_SHIFT) &
+ OV4689_GAIN_H_MASK);
+ ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
+ OV4689_REG_VALUE_08BIT,
+ ctrl->val & OV4689_GAIN_L_MASK);
+ break;
+ case V4L2_CID_VBLANK:
+ ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
+ OV4689_REG_VALUE_16BIT,
+ ctrl->val + ov4689->cur_mode->height);
+ break;
+ case V4L2_CID_TEST_PATTERN:
+ ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
+ break;
+ default:
+ dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
+ __func__, ctrl->id, ctrl->val);
+ ret = -EINVAL;
+ break;
+ }
+
+ pm_runtime_put(&client->dev);
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
+ .s_ctrl = ov4689_set_ctrl,
+};
+
+static int ov4689_initialize_controls(struct ov4689 *ov4689)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
+ struct v4l2_fwnode_device_properties props;
+ struct v4l2_ctrl_handler *handler;
+ const struct ov4689_mode *mode;
+ s64 exposure_max, vblank_def;
+ struct v4l2_ctrl *ctrl;
+ u32 h_blank, pixel_rate;
+ int ret;
+
+ handler = &ov4689->ctrl_handler;
+ mode = ov4689->cur_mode;
+ ret = v4l2_ctrl_handler_init(handler, 10);
+ if (ret)
+ return ret;
+ handler->lock = &ov4689->mutex;
+
+ ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
+ link_freq_menu_items);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
+ OV4689_BITS_PER_SAMPLE;
+ v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
+ pixel_rate);
+
+ h_blank = mode->hts_def - mode->width;
+ ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
+ h_blank, h_blank, 1, h_blank);
+ if (ov4689->hblank)
+ ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ vblank_def = mode->vts_def - mode->height;
+ ov4689->vblank =
+ v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
+ vblank_def, OV4689_VTS_MAX - mode->height, 1,
+ vblank_def);
+
+ exposure_max = mode->vts_def - 4;
+ ov4689->exposure =
+ v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
+ OV4689_EXPOSURE_MIN, exposure_max,
+ OV4689_EXPOSURE_STEP, mode->exp_def);
+
+ ov4689->anal_gain =
+ v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
+ V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
+ OV4689_GAIN_MAX, OV4689_GAIN_STEP,
+ OV4689_GAIN_DEFAULT);
+
+ ov4689->test_pattern =
+ v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
+ V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
+ 0, 0, ov4689_test_pattern_menu);
+
+ if (handler->error) {
+ ret = handler->error;
+ dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
+ ret);
+ goto err_free_handler;
+ }
+
+ ret = v4l2_fwnode_device_parse(&client->dev, &props);
+ if (ret)
+ goto err_free_handler;
+
+ ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
+ &props);
+ if (ret)
+ goto err_free_handler;
+
+ ov4689->subdev.ctrl_handler = handler;
+
+ return 0;
+
+err_free_handler:
+ v4l2_ctrl_handler_free(handler);
+
+ return ret;
+}
+
+static int ov4689_check_sensor_id(struct ov4689 *ov4689,
+ struct i2c_client *client)
+{
+ struct device *dev = &ov4689->client->dev;
+ u32 id = 0;
+ int ret;
+
+ ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
+ OV4689_REG_VALUE_16BIT, &id);
+ if (id != CHIP_ID) {
+ dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
+ return -ENODEV;
+ }
+
+ dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
+
+ return 0;
+}
+
+static int ov4689_configure_regulators(struct ov4689 *ov4689)
+{
+ unsigned int i;
+
+ for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
+ ov4689->supplies[i].supply = ov4689_supply_names[i];
+
+ return devm_regulator_bulk_get(&ov4689->client->dev,
+ OV4689_NUM_SUPPLIES, ov4689->supplies);
+}
+
+static int ov4689_check_hwcfg(struct device *dev)
+{
+ struct fwnode_handle *fwnode = dev_fwnode(dev);
+ struct v4l2_fwnode_endpoint bus_cfg = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY,
+ };
+ struct fwnode_handle *endpoint;
+ unsigned int i;
+ int ret;
+
+ endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
+ if (!endpoint)
+ return -EPROBE_DEFER;
+
+ ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
+ fwnode_handle_put(endpoint);
+ if (ret)
+ return ret;
+
+ if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
+ dev_err(dev, "only a 4-lane CSI2 config is supported");
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+ if (!bus_cfg.nr_of_link_frequencies) {
+ dev_err(dev, "no link frequencies defined\n");
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+ for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
+ if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
+ break;
+
+ if (i == bus_cfg.nr_of_link_frequencies) {
+ dev_err(dev, "supported link freq %ull not found\n",
+ OV4689_LINK_FREQ_500MHZ);
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+out_free_bus_cfg:
+ v4l2_fwnode_endpoint_free(&bus_cfg);
+
+ return ret;
+}
+
+static int ov4689_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &client->dev;
+ struct v4l2_subdev *sd;
+ struct ov4689 *ov4689;
+ int ret;
+
+ ret = ov4689_check_hwcfg(dev);
+ if (ret)
+ return ret;
+
+ ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
+ if (!ov4689)
+ return -ENOMEM;
+
+ ov4689->client = client;
+ ov4689->cur_mode = &supported_modes[0];
+
+ ov4689->xvclk = devm_clk_get(dev, "xvclk");
+ if (IS_ERR(ov4689->xvclk)) {
+ dev_err(dev, "Failed to get xvclk\n");
+ return -EINVAL;
+ }
+
+ ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
+ return ret;
+ }
+ if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
+ dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
+
+ ov4689->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+ if (IS_ERR(ov4689->reset_gpio)) {
+ dev_err(dev, "Failed to get reset-gpios\n");
+ return -EINVAL;
+ }
+
+ ov4689->pwdn_gpio = devm_gpiod_get(dev, "pwdn", GPIOD_OUT_LOW);
+ if (IS_ERR(ov4689->pwdn_gpio)) {
+ dev_err(dev, "Failed to get pwdn-gpios\n");
+ return -EINVAL;
+ }
+
+ ret = ov4689_configure_regulators(ov4689);
+ if (ret) {
+ dev_err(dev, "Failed to get power regulators\n");
+ return ret;
+ }
+
+ mutex_init(&ov4689->mutex);
+
+ sd = &ov4689->subdev;
+ v4l2_i2c_subdev_init(sd, client, &ov4689_subdev_ops);
+ ret = ov4689_initialize_controls(ov4689);
+ if (ret)
+ goto err_destroy_mutex;
+
+ ret = __ov4689_power_on(ov4689);
+ if (ret)
+ goto err_free_handler;
+
+ ret = ov4689_check_sensor_id(ov4689, client);
+ if (ret)
+ goto err_power_off;
+
+#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
+ sd->internal_ops = &ov4689_internal_ops;
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+#endif
+#if defined(CONFIG_MEDIA_CONTROLLER)
+ ov4689->pad.flags = MEDIA_PAD_FL_SOURCE;
+ sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
+ ret = media_entity_pads_init(&sd->entity, 1, &ov4689->pad);
+ if (ret < 0)
+ goto err_power_off;
+#endif
+
+ ret = v4l2_async_register_subdev_sensor(sd);
+ if (ret) {
+ dev_err(dev, "v4l2 async register subdev failed\n");
+ goto err_clean_entity;
+ }
+
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+ pm_runtime_idle(dev);
+
+ return 0;
+
+err_clean_entity:
+#if defined(CONFIG_MEDIA_CONTROLLER)
+ media_entity_cleanup(&sd->entity);
+#endif
+err_power_off:
+ __ov4689_power_off(ov4689);
+err_free_handler:
+ v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
+err_destroy_mutex:
+ mutex_destroy(&ov4689->mutex);
+
+ return ret;
+}
+
+static int ov4689_remove(struct i2c_client *client)
+{
+ struct v4l2_subdev *sd = i2c_get_clientdata(client);
+ struct ov4689 *ov4689 = to_ov4689(sd);
+
+ v4l2_async_unregister_subdev(sd);
+#if defined(CONFIG_MEDIA_CONTROLLER)
+ media_entity_cleanup(&sd->entity);
+#endif
+ v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
+ mutex_destroy(&ov4689->mutex);
+
+ pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev))
+ __ov4689_power_off(ov4689);
+ pm_runtime_set_suspended(&client->dev);
+
+ return 0;
+}
+
+static const struct i2c_device_id ov4689_id[] = {
+ { "ov4689", 0 },
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, ov4689_id);
+
+static const struct of_device_id ov4689_of_match[] = {
+ { .compatible = "ovti,ov4689" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, ov4689_of_match);
+
+static struct i2c_driver ov4689_i2c_driver = {
+ .driver = {
+ .name = "ov4689",
+ .pm = &ov4689_pm_ops,
+ .of_match_table = of_match_ptr(ov4689_of_match),
+ },
+ .probe = ov4689_probe,
+ .remove = ov4689_remove,
+ .id_table = ov4689_id,
+};
+
+module_i2c_driver(ov4689_i2c_driver);
+
+MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
+MODULE_LICENSE("GPL");
--
2.37.3

2022-09-11 23:25:15

by kernel test robot

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,

I love your patch! Perhaps something to improve:

[auto build test WARNING on media-tree/master]
[also build test WARNING on robh/for-next krzk-dt/for-next linus/master v6.0-rc5 next-20220909]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url: https://github.com/intel-lab-lkp/linux/commits/Mikhail-Rudenko/Add-Omnivision-OV4689-image-sensor-driver/20220912-040337
base: git://linuxtv.org/media_tree.git master
config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20220912/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-5) 11.3.0
reproduce (this is a W=1 build):
# https://github.com/intel-lab-lkp/linux/commit/7c4d2965802d2be20badfef953b1d6f0d13d718f
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Mikhail-Rudenko/Add-Omnivision-OV4689-image-sensor-driver/20220912-040337
git checkout 7c4d2965802d2be20badfef953b1d6f0d13d718f
# save the config file
mkdir build_dir && cp config build_dir/.config
make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash drivers/media/i2c/

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <[email protected]>

All warnings (new ones prefixed by >>):

>> drivers/media/i2c/ov4689.c:112:28: warning: 'ov4689_global_regs' defined but not used [-Wunused-const-variable=]
112 | static const struct regval ov4689_global_regs[] = {
| ^~~~~~~~~~~~~~~~~~


vim +/ov4689_global_regs +112 drivers/media/i2c/ov4689.c

108
109 /*
110 * Xclk 24Mhz
111 */
> 112 static const struct regval ov4689_global_regs[] = {
113 { REG_NULL, 0x00 },
114 };
115

--
0-DAY CI Kernel Test Service
https://01.org/lkp

2022-09-12 12:00:16

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

On 11/09/2022 22:01, Mikhail Rudenko wrote:
> +static const struct i2c_device_id ov4689_id[] = {
> + { "ov4689", 0 },
> + {},
> +};
> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
> +
> +static const struct of_device_id ov4689_of_match[] = {
> + { .compatible = "ovti,ov4689" },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
> +
> +static struct i2c_driver ov4689_i2c_driver = {
> + .driver = {
> + .name = "ov4689",
> + .pm = &ov4689_pm_ops,
> + .of_match_table = of_match_ptr(ov4689_of_match),

of_match_ptr is usually paired with maybe_unused, otherwise you will
have compile test warnings.

> + },
> + .probe = ov4689_probe,
> + .remove = ov4689_remove,
> + .id_table = ov4689_id,
> +};
> +
> +module_i2c_driver(ov4689_i2c_driver);
> +
> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
> +MODULE_LICENSE("GPL");


Best regards,
Krzysztof

2022-09-14 10:12:02

by Dave Stevenson

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail

On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
>
> Hello,
>
> this series implements support for Omnivision OV4689 image
> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> megapixel image sensor. Ihis chip supports high frame rate speeds up
> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> connection.
>
> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
>
> While porting the driver, I stumbled upon two issues:
>
> (1) In the original driver, horizontal total size (HTS) was set to a
> value (2584) lower then the frame width (2688), resulting in negative
> hblank. In this driver, I increased HTS to 2688, but fps dropped from
> 29.88 to 28.73. What is the preferred way to handle this?

This is one of the joys of sensors - they don't all work in the same way.

I don't have an official datasheet for OV4689 from Omnivision, but
found one on the internet [1]. That should allow you to reverse the
PLL configuration to confirm that the pixel rate is the value you've
computed based on link frequency (they aren't necessarily related). Do
the frame rate calculations work using width + HBLANK, height +
VBLANK, and pixel rate?
The datasheet claims the sensor supports 2688x1520 @ 90 fps, so
something doesn't hold true between 4 data lanes at 500MHz/1Gbit/s per
lane when your default hts/vts is 2688x1554 and it only gives
28.73fps.

I have seen modes in sensors where the HTS register is in units of 2
pixels, so what range of HTS (and VTS) values actually works on this
sensor? (I don't see it documented, but I'm not surprised).

[1] https://cdn.hackaday.io/files/19354828041536/OV4689-OmniVision.pdf

> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> gain is not linear across that range. Instead, it is piecewise linear
> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> with more linear segments in between. Rockchip's camera engine code
> chooses one of the above segments depenging on the desired gain
> value. The question is, how should we proceed keeping in mind
> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> libcamera will do the mapping, or the driver will do the mapping
> itself and expose some logical gain units not tied to the actual gain
> register value? Meanwhile, this driver conservatively exposes only
> 0x0-0xf8 gain register range.

The datasheet linked above says "for the gain formula, please contact
your local OmniVision FAE" :-(
I would assume that the range is from 1x rather than 0x - people
rarely want a totally black image that 0x would give. Or is it ranges
of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?

Other sensors expose the full range of the register via
V4L2_CID_ANALOGUE_GAIN, and require userspace (mainly libcamera now)
to know how to convert a gain into the register value. If the gain
range goes up to x16, then exposing that would be useful. I'd advocate
just exposing the full range of 0x000 - 0x7ff, as then you can have
the accuracy of 256 values between x1 to x2, but also the full range.

I might see if I can pick up one of these sensors and see if I can get
it running on a Raspberry Pi. Thanks for trying to upstream this -
it's nice to have such a range of sensor drivers to choose from.

Dave

> [1] https://github.com/rockchip-linux/kernel/blob/develop-4.19/drivers/media/i2c/ov4689.c
>
> changes in v2:
> - bindings: reword descriptions
> - bindings: move clock description to clocks property
> - bindings: add data-lanes and link-frequencies properties to port
> - driver: validate media bus configuration when probing
>
> Mikhail Rudenko (2):
> media: dt-bindings: media: i2c: document OV4689 DT bindings
> media: i2c: add support for ov4689
>
> .../bindings/media/i2c/ovti,ov4689.yaml | 141 +++
> MAINTAINERS | 8 +
> drivers/media/i2c/Kconfig | 14 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov4689.c | 951 ++++++++++++++++++
> 5 files changed, 1115 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
> create mode 100644 drivers/media/i2c/ov4689.c
>
> --
> 2.37.3

2022-09-14 16:08:44

by Tommaso Merciai

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,
I do a first round on reviewing your driver :)

On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
> bus for data.
>
> This driver supports following features:
> - manual exposure and analog gain control support
> - test pattern support
> - media controller support
> - runtime PM support
> - support following resolutions:
> + 2688x1520 at 30 fps
>
> The driver provides all mandatory V4L2 controls for compatibility with
> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
> implements 4 lane mode only at this moment.
>
> Signed-off-by: Mikhail Rudenko <[email protected]>
> ---
> MAINTAINERS | 1 +
> drivers/media/i2c/Kconfig | 14 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 967 insertions(+)
> create mode 100644 drivers/media/i2c/ov4689.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 63c4844f26e6..1857f3864e1b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -14529,6 +14529,7 @@ L: [email protected]
> S: Maintained
> T: git git://linuxtv.org/media_tree.git
> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
> +F: drivers/media/i2c/ov5647.c
>
> OMNIVISION OV5640 SENSOR DRIVER
> M: Steve Longerbeam <[email protected]>
> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
> index fae2baabb773..4993e1ae2ea8 100644
> --- a/drivers/media/i2c/Kconfig
> +++ b/drivers/media/i2c/Kconfig
> @@ -429,6 +429,20 @@ config VIDEO_OV2740
> To compile this driver as a module, choose M here: the
> module will be called ov2740.
>
> +config VIDEO_OV4689
> + tristate "OmniVision OV4689 sensor support"
> + depends on OF
> + depends on GPIOLIB && VIDEO_DEV && I2C
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + This is a Video4Linux2 sensor-level driver for the OmniVision
> + OV4689 camera.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called ov4689.
> +
> config VIDEO_OV5640
> tristate "OmniVision OV5640 sensor support"
> depends on OF
> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
> index 3e1696963e7f..7446c0a1eed0 100644
> --- a/drivers/media/i2c/Makefile
> +++ b/drivers/media/i2c/Makefile
> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
> new file mode 100644
> index 000000000000..9f05e812acf8
> --- /dev/null
> +++ b/drivers/media/i2c/ov4689.c
> @@ -0,0 +1,951 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ov4689 driver
> + *
> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/device.h>
> +#include <linux/delay.h>
> +#include <linux/gpio/consumer.h>
> +#include <linux/i2c.h>
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regulator/consumer.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#define CHIP_ID 0x004688
> +#define OV4689_REG_CHIP_ID 0x300a
> +
> +#define OV4689_XVCLK_FREQ 24000000
> +
> +#define OV4689_REG_CTRL_MODE 0x0100
> +#define OV4689_MODE_SW_STANDBY 0x0
> +#define OV4689_MODE_STREAMING BIT(0)
> +
> +#define OV4689_REG_EXPOSURE 0x3500
> +#define OV4689_EXPOSURE_MIN 4
> +#define OV4689_EXPOSURE_STEP 1
> +#define OV4689_VTS_MAX 0x7fff
> +
> +#define OV4689_REG_GAIN_H 0x3508
> +#define OV4689_REG_GAIN_L 0x3509
> +#define OV4689_GAIN_H_MASK 0x07
> +#define OV4689_GAIN_H_SHIFT 8
> +#define OV4689_GAIN_L_MASK 0xff
> +#define OV4689_GAIN_MIN 0x10
> +#define OV4689_GAIN_MAX 0xf8
> +#define OV4689_GAIN_STEP 1
> +#define OV4689_GAIN_DEFAULT 0x10
> +
> +#define OV4689_REG_TEST_PATTERN 0x5040
> +#define OV4689_TEST_PATTERN_ENABLE 0x80
> +#define OV4689_TEST_PATTERN_DISABLE 0x0
> +
> +#define OV4689_REG_VTS 0x380e
> +
> +#define REG_NULL 0xFFFF
> +
> +#define OV4689_REG_VALUE_08BIT 1
> +#define OV4689_REG_VALUE_16BIT 2
> +#define OV4689_REG_VALUE_24BIT 3
> +
> +#define OV4689_LANES 4
> +#define OV4689_BITS_PER_SAMPLE 10
> +
> +static const char *const ov4689_supply_names[] = {
> + "avdd", /* Analog power */
> + "dovdd", /* Digital I/O power */
> + "dvdd", /* Digital core power */
> +};
> +
> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
> +
> +struct regval {
> + u16 addr;
> + u8 val;
> +};

What about use ov4689_mode_id? I think could be usefull for the future:

+ enum ov4689_mode_id {
+ OV4689_MODE_2688_1520 = 0,
+ OV4689_NUM_MODES,
+ };


> +
> +struct ov4689_mode {
+ enum ov4689_mode_id id;
> + u32 width;
> + u32 height;
> + u32 max_fps;
> + u32 hts_def;
> + u32 vts_def;
> + u32 exp_def;
> + const struct regval *reg_list;
> +};
> +
> +struct ov4689 {
> + struct i2c_client *client;
> + struct clk *xvclk;
> + struct gpio_desc *reset_gpio;
> + struct gpio_desc *pwdn_gpio;
> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
> +
> + struct v4l2_subdev subdev;
> + struct media_pad pad;
> +
> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
> + bool streaming;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct v4l2_ctrl *exposure;
> + struct v4l2_ctrl *anal_gain;
> + struct v4l2_ctrl *digi_gain;
> + struct v4l2_ctrl *hblank;
> + struct v4l2_ctrl *vblank;
> + struct v4l2_ctrl *test_pattern;
> +
> + const struct ov4689_mode *cur_mode;
> +};
> +
> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
> +
> +/*
> + * Xclk 24Mhz
> + */
> +static const struct regval ov4689_global_regs[] = {
> + { REG_NULL, 0x00 },
> +};
> +
> +/*
> + * Xclk 24Mhz
> + * max_framerate 30fps
> + * mipi_datarate per lane 1008Mbps
> + */
> +static const struct regval ov4689_2688x1520_regs[] = {
> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
> + {REG_NULL, 0x00},
> +};
> +
> +static const struct ov4689_mode supported_modes[] = {
> + {
+ .id = OV4689_MODE_2688_1520,
> + .width = 2688,
> + .height = 1520,
> + .max_fps = 30,
> + .exp_def = 0x0600,
> + .hts_def = 0x0a80,
> + .vts_def = 0x0612,
> + .reg_list = ov4689_2688x1520_regs,
> + },
> +};
> +
> +#define OV4689_LINK_FREQ_500MHZ 500000000
> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
> +
> +static const char *const ov4689_test_pattern_menu[] = {
> + "Disabled",
> + "Vertical Color Bar Type 1",
> + "Vertical Color Bar Type 2",
> + "Vertical Color Bar Type 3",
> + "Vertical Color Bar Type 4"
> +};
> +
> +/* Write registers up to 4 at a time */
> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
> + u32 val)
> +{
> + u32 buf_i, val_i;
> + __be32 val_be;
> + u8 *val_p;
> + u8 buf[6];
> +
> + if (len > 4)
> + return -EINVAL;
> +
> + buf[0] = reg >> 8;
> + buf[1] = reg & 0xff;
> +
> + val_be = cpu_to_be32(val);
> + val_p = (u8 *)&val_be;
> + buf_i = 2;
> + val_i = 4 - len;
> +
> + while (val_i < 4)
> + buf[buf_i++] = val_p[val_i++];
> +
> + if (i2c_master_send(client, buf, len + 2) != len + 2)
> + return -EIO;
> +
> + return 0;
> +}
> +
> +static int ov4689_write_array(struct i2c_client *client,
> + const struct regval *regs)
> +{
> + int ret = 0;
> + u32 i;
> +
> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
> + ret = ov4689_write_reg(client, regs[i].addr,
> + OV4689_REG_VALUE_08BIT, regs[i].val);
> +
> + return ret;
> +}
> +
> +/* Read registers up to 4 at a time */
> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
> + u32 *val)
> +{
> + __be16 reg_addr_be = cpu_to_be16(reg);
> + struct i2c_msg msgs[2];
> + __be32 data_be = 0;
> + u8 *data_be_p;
> + int ret;
> +
> + if (len > 4 || !len)
> + return -EINVAL;
> +
> + data_be_p = (u8 *)&data_be;
> + /* Write register address */
> + msgs[0].addr = client->addr;
> + msgs[0].flags = 0;
> + msgs[0].len = 2;
> + msgs[0].buf = (u8 *)&reg_addr_be;
> +
> + /* Read data from register */
> + msgs[1].addr = client->addr;
> + msgs[1].flags = I2C_M_RD;
> + msgs[1].len = len;
> + msgs[1].buf = &data_be_p[4 - len];
> +
> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
> + if (ret != ARRAY_SIZE(msgs))
> + return -EIO;
> +
> + *val = be32_to_cpu(data_be);
> +
> + return 0;
> +}
> +
> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
> + struct v4l2_mbus_framefmt *fmt)
> +{
> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> + fmt->width = mode->width;
> + fmt->height = mode->height;
> + fmt->field = V4L2_FIELD_NONE;
> +}
> +
> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index != 0)
> + return -EINVAL;
> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + if (fse->index >= ARRAY_SIZE(supported_modes))
> + return -EINVAL;
> +
> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
> + return -EINVAL;
> +
> + fse->min_width = supported_modes[fse->index].width;
> + fse->max_width = supported_modes[fse->index].width;
> + fse->max_height = supported_modes[fse->index].height;
> + fse->min_height = supported_modes[fse->index].height;
> +
> + return 0;
> +}
> +
> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
> +{
> + u32 val;
> +
> + if (pattern)
> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
> + else
> + val = OV4689_TEST_PATTERN_DISABLE;
> +
> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
> + OV4689_REG_VALUE_08BIT, val);
> +}
> +
> +static int ov4689_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> + return -EINVAL;
> +
> + switch (sel->target) {
> + case V4L2_SEL_TGT_CROP_BOUNDS:
> + sel->r.top = 0;
> + sel->r.left = 0;
> + sel->r.width = 2720;
> + sel->r.height = 1536;
> + return 0;
> + case V4L2_SEL_TGT_CROP:
> + case V4L2_SEL_TGT_CROP_DEFAULT:
> + sel->r.top = 8;
> + sel->r.left = 16;
> + sel->r.width = 2688;
> + sel->r.height = 1520;
> + return 0;
> + }
> + return -EINVAL;
> +}
> +
> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct i2c_client *client = ov4689->client;
> + int ret = 0;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + on = !!on;
> + if (on == ov4689->streaming)
> + goto unlock_and_return;
> +
> + if (on) {
> + ret = pm_runtime_resume_and_get(&client->dev);
> + if (ret < 0)
> + goto unlock_and_return;
> +
> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_array(ov4689->client,
> + ov4689->cur_mode->reg_list);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_STREAMING);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> + } else {
> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_SW_STANDBY);
> + pm_runtime_put(&client->dev);
> + }
> +
> + ov4689->streaming = on;
> +
> +unlock_and_return:
> + mutex_unlock(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +/* Calculate the delay in us by clock rate and clock cycles */
> +static inline u32 ov4689_cal_delay(u32 cycles)
> +{
> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
> +}
> +
> +static int __ov4689_power_on(struct ov4689 *ov4689)

Just a doubt on this name function. Why __ ? Is this name reserved for?

> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 delay_us;
> + int ret;
> +
> + ret = clk_prepare_enable(ov4689->xvclk);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable xvclk\n");
> + return ret;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> +
> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable regulators\n");
> + goto disable_clk;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
> + usleep_range(500, 1000);
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
> +
> + /* 8192 cycles prior to first SCCB transaction */
> + delay_us = ov4689_cal_delay(8192);
> + usleep_range(delay_us, delay_us * 2);
> +
> + return 0;
> +
> +disable_clk:
> + clk_disable_unprepare(ov4689->xvclk);
> +
> + return ret;
> +}
> +
> +static void __ov4689_power_off(struct ov4689 *ov4689)
> +{
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
> + clk_disable_unprepare(ov4689->xvclk);
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}
> +
> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + return __ov4689_power_on(ov4689);
> +}
> +
> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + __ov4689_power_off(ov4689);
> +
> + return 0;
> +}
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct v4l2_mbus_framefmt *try_fmt;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
> + /* Initialize try_fmt */
> + ov4689_fill_fmt(&supported_modes[0], try_fmt);

In this way instead of use magic number we can use this:

ov4689_fill_fmt(&supported_modes[OV4689_MODE_2688_1520], try_fmt);

> +
> + mutex_unlock(&ov4689->mutex);
> +
> + return 0;
> +}
> +#endif
> +
> +static const struct dev_pm_ops ov4689_pm_ops = {
> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
> +};
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
> + .open = ov4689_open,
> +};
> +#endif
> +
> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
> + .s_stream = ov4689_s_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
> + .enum_mbus_code = ov4689_enum_mbus_code,
> + .enum_frame_size = ov4689_enum_frame_sizes,
> + .get_fmt = ov4689_get_fmt,
> + .set_fmt = ov4689_set_fmt,
> + .get_selection = ov4689_get_selection,
> +};
> +
> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
> + .video = &ov4689_video_ops,
> + .pad = &ov4689_pad_ops,
> +};
> +
> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct ov4689 *ov4689 =
> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
> + struct i2c_client *client = ov4689->client;
> + s64 max_expo;
> + int ret;
> +
> + /* Propagate change of current control to all related controls */
> + switch (ctrl->id) {
> + case V4L2_CID_VBLANK:
> + /* Update max exposure while meeting expected vblanking */
> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
> + __v4l2_ctrl_modify_range(ov4689->exposure,
> + ov4689->exposure->minimum, max_expo,
> + ov4689->exposure->step,
> + ov4689->exposure->default_value);
> + break;
> + }
> +
> + if (!pm_runtime_get_if_in_use(&client->dev))
> + return 0;
> +
> + switch (ctrl->id) {
> + case V4L2_CID_EXPOSURE:
> + /* 4 least significant bits of expsoure are fractional part */
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
> + break;
> + case V4L2_CID_ANALOGUE_GAIN:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
> + OV4689_REG_VALUE_08BIT,
> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
> + OV4689_GAIN_H_MASK);
> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
> + OV4689_REG_VALUE_08BIT,
> + ctrl->val & OV4689_GAIN_L_MASK);
> + break;
> + case V4L2_CID_VBLANK:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
> + OV4689_REG_VALUE_16BIT,
> + ctrl->val + ov4689->cur_mode->height);
> + break;
> + case V4L2_CID_TEST_PATTERN:
> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
> + break;
> + default:
> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
> + __func__, ctrl->id, ctrl->val);
> + ret = -EINVAL;
> + break;
> + }
> +
> + pm_runtime_put(&client->dev);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
> + .s_ctrl = ov4689_set_ctrl,
> +};
> +
> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
> +{
> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
> + struct v4l2_fwnode_device_properties props;
> + struct v4l2_ctrl_handler *handler;
> + const struct ov4689_mode *mode;
> + s64 exposure_max, vblank_def;
> + struct v4l2_ctrl *ctrl;
> + u32 h_blank, pixel_rate;
> + int ret;
> +
> + handler = &ov4689->ctrl_handler;
> + mode = ov4689->cur_mode;
> + ret = v4l2_ctrl_handler_init(handler, 10);
> + if (ret)
> + return ret;
> + handler->lock = &ov4689->mutex;
> +
> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
> + link_freq_menu_items);
> + if (ctrl)
> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
> + OV4689_BITS_PER_SAMPLE;
> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
> + pixel_rate);
> +
> + h_blank = mode->hts_def - mode->width;
> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
> + h_blank, h_blank, 1, h_blank);
> + if (ov4689->hblank)
> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + vblank_def = mode->vts_def - mode->height;
> + ov4689->vblank =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
> + vblank_def);
> +
> + exposure_max = mode->vts_def - 4;
> + ov4689->exposure =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
> + OV4689_EXPOSURE_MIN, exposure_max,
> + OV4689_EXPOSURE_STEP, mode->exp_def);
> +
> + ov4689->anal_gain =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
> + OV4689_GAIN_DEFAULT);
> +
> + ov4689->test_pattern =
> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
> + V4L2_CID_TEST_PATTERN,
> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
> + 0, 0, ov4689_test_pattern_menu);
> +
> + if (handler->error) {
> + ret = handler->error;
> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
> + ret);
> + goto err_free_handler;
> + }
> +
> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
> + &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ov4689->subdev.ctrl_handler = handler;
> +
> + return 0;
> +
> +err_free_handler:
> + v4l2_ctrl_handler_free(handler);
> +
> + return ret;
> +}
> +
> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
> + struct i2c_client *client)
> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 id = 0;
> + int ret;
> +
> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
> + OV4689_REG_VALUE_16BIT, &id);
> + if (id != CHIP_ID) {
> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
> + return -ENODEV;
> + }
> +
> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
> +
> + return 0;
> +}
> +
> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
> + ov4689->supplies[i].supply = ov4689_supply_names[i];
> +
> + return devm_regulator_bulk_get(&ov4689->client->dev,
> + OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}
> +
> +static int ov4689_check_hwcfg(struct device *dev)
> +{
> + struct fwnode_handle *fwnode = dev_fwnode(dev);
> + struct v4l2_fwnode_endpoint bus_cfg = {
> + .bus_type = V4L2_MBUS_CSI2_DPHY,
> + };
> + struct fwnode_handle *endpoint;
> + unsigned int i;
> + int ret;
> +
> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
> + if (!endpoint)
> + return -EPROBE_DEFER;
> +
> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
> + fwnode_handle_put(endpoint);
> + if (ret)
> + return ret;
> +
> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
> + dev_err(dev, "only a 4-lane CSI2 config is supported");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + if (!bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "no link frequencies defined\n");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
> + break;
> +
> + if (i == bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "supported link freq %ull not found\n",
> + OV4689_LINK_FREQ_500MHZ);
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> +out_free_bus_cfg:
> + v4l2_fwnode_endpoint_free(&bus_cfg);
> +
> + return ret;
> +}
> +
> +static int ov4689_probe(struct i2c_client *client,
> + const struct i2c_device_id *id)

We are sure that we need need i2c_device_id *id?

> +{
> + struct device *dev = &client->dev;
> + struct v4l2_subdev *sd;
> + struct ov4689 *ov4689;
> + int ret;
> +
> + ret = ov4689_check_hwcfg(dev);
> + if (ret)
> + return ret;
> +
> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
> + if (!ov4689)
> + return -ENOMEM;
> +
> + ov4689->client = client;
> + ov4689->cur_mode = &supported_modes[0];

Here aswell we can use:
ov4689->cur_mode = &supported_modes[OV4689_MODE_2688_1520];

> +
> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
> + if (IS_ERR(ov4689->xvclk)) {
> + dev_err(dev, "Failed to get xvclk\n");
> + return -EINVAL;
> + }

^ I think is better to use devm_clk_get_optional instead of clck_get.
clck_get can fail in CPU's that use ACPI

> +
> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> + if (ret < 0) {
> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> + return ret;
> + }
> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");


What do you think about?
Thanks.

Regards,
Tommaso

> +
> + ov4689->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->reset_gpio)) {
> + dev_err(dev, "Failed to get reset-gpios\n");
> + return -EINVAL;
> + }
> +
> + ov4689->pwdn_gpio = devm_gpiod_get(dev, "pwdn", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->pwdn_gpio)) {
> + dev_err(dev, "Failed to get pwdn-gpios\n");
> + return -EINVAL;
> + }
> +
> + ret = ov4689_configure_regulators(ov4689);
> + if (ret) {
> + dev_err(dev, "Failed to get power regulators\n");
> + return ret;
> + }
> +
> + mutex_init(&ov4689->mutex);
> +
> + sd = &ov4689->subdev;
> + v4l2_i2c_subdev_init(sd, client, &ov4689_subdev_ops);
> + ret = ov4689_initialize_controls(ov4689);
> + if (ret)
> + goto err_destroy_mutex;
> +
> + ret = __ov4689_power_on(ov4689);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = ov4689_check_sensor_id(ov4689, client);
> + if (ret)
> + goto err_power_off;
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> + sd->internal_ops = &ov4689_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> +#endif
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + ov4689->pad.flags = MEDIA_PAD_FL_SOURCE;
> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
> + ret = media_entity_pads_init(&sd->entity, 1, &ov4689->pad);
> + if (ret < 0)
> + goto err_power_off;
> +#endif
> +
> + ret = v4l2_async_register_subdev_sensor(sd);
> + if (ret) {
> + dev_err(dev, "v4l2 async register subdev failed\n");
> + goto err_clean_entity;
> + }
> +
> + pm_runtime_set_active(dev);
> + pm_runtime_enable(dev);
> + pm_runtime_idle(dev);
> +
> + return 0;
> +
> +err_clean_entity:
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + media_entity_cleanup(&sd->entity);
> +#endif
> +err_power_off:
> + __ov4689_power_off(ov4689);
> +err_free_handler:
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> +err_destroy_mutex:
> + mutex_destroy(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +static int ov4689_remove(struct i2c_client *client)
> +{
> + struct v4l2_subdev *sd = i2c_get_clientdata(client);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + v4l2_async_unregister_subdev(sd);
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + media_entity_cleanup(&sd->entity);
> +#endif
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> + mutex_destroy(&ov4689->mutex);
> +
> + pm_runtime_disable(&client->dev);
> + if (!pm_runtime_status_suspended(&client->dev))
> + __ov4689_power_off(ov4689);
> + pm_runtime_set_suspended(&client->dev);
> +
> + return 0;
> +}
> +
> +static const struct i2c_device_id ov4689_id[] = {
> + { "ov4689", 0 },
> + {},
> +};
> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
> +
> +static const struct of_device_id ov4689_of_match[] = {
> + { .compatible = "ovti,ov4689" },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
> +
> +static struct i2c_driver ov4689_i2c_driver = {
> + .driver = {
> + .name = "ov4689",
> + .pm = &ov4689_pm_ops,
> + .of_match_table = of_match_ptr(ov4689_of_match),
> + },
> + .probe = ov4689_probe,
> + .remove = ov4689_remove,
> + .id_table = ov4689_id,
> +};
> +
> +module_i2c_driver(ov4689_i2c_driver);
> +
> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
> +MODULE_LICENSE("GPL");
> --
> 2.37.3
>

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
http://www.amarulasolutions.com

2022-09-15 22:19:41

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689


Hi Tommaso,

On 2022-09-14 at 17:51 +02, Tommaso Merciai <[email protected]> wrote:
> Hi Mikhail,
> I do a first round on reviewing your driver :)
>
> On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
>> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
>> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
>> bus for data.
>>
>> This driver supports following features:
>> - manual exposure and analog gain control support
>> - test pattern support
>> - media controller support
>> - runtime PM support
>> - support following resolutions:
>> + 2688x1520 at 30 fps
>>
>> The driver provides all mandatory V4L2 controls for compatibility with
>> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
>> implements 4 lane mode only at this moment.
>>
>> Signed-off-by: Mikhail Rudenko <[email protected]>
>> ---
>> MAINTAINERS | 1 +
>> drivers/media/i2c/Kconfig | 14 +
>> drivers/media/i2c/Makefile | 1 +
>> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
>> 4 files changed, 967 insertions(+)
>> create mode 100644 drivers/media/i2c/ov4689.c
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 63c4844f26e6..1857f3864e1b 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -14529,6 +14529,7 @@ L: [email protected]
>> S: Maintained
>> T: git git://linuxtv.org/media_tree.git
>> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
>> +F: drivers/media/i2c/ov5647.c
>>
>> OMNIVISION OV5640 SENSOR DRIVER
>> M: Steve Longerbeam <[email protected]>
>> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
>> index fae2baabb773..4993e1ae2ea8 100644
>> --- a/drivers/media/i2c/Kconfig
>> +++ b/drivers/media/i2c/Kconfig
>> @@ -429,6 +429,20 @@ config VIDEO_OV2740
>> To compile this driver as a module, choose M here: the
>> module will be called ov2740.
>>
>> +config VIDEO_OV4689
>> + tristate "OmniVision OV4689 sensor support"
>> + depends on OF
>> + depends on GPIOLIB && VIDEO_DEV && I2C
>> + select MEDIA_CONTROLLER
>> + select VIDEO_V4L2_SUBDEV_API
>> + select V4L2_FWNODE
>> + help
>> + This is a Video4Linux2 sensor-level driver for the OmniVision
>> + OV4689 camera.
>> +
>> + To compile this driver as a module, choose M here: the
>> + module will be called ov4689.
>> +
>> config VIDEO_OV5640
>> tristate "OmniVision OV5640 sensor support"
>> depends on OF
>> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
>> index 3e1696963e7f..7446c0a1eed0 100644
>> --- a/drivers/media/i2c/Makefile
>> +++ b/drivers/media/i2c/Makefile
>> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
>> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
>> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
>> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
>> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
>> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
>> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
>> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
>> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
>> new file mode 100644
>> index 000000000000..9f05e812acf8
>> --- /dev/null
>> +++ b/drivers/media/i2c/ov4689.c
>> @@ -0,0 +1,951 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ov4689 driver
>> + *
>> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
>> + */
>> +
>> +#include <linux/clk.h>
>> +#include <linux/device.h>
>> +#include <linux/delay.h>
>> +#include <linux/gpio/consumer.h>
>> +#include <linux/i2c.h>
>> +#include <linux/module.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/regulator/consumer.h>
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-async.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/v4l2-fwnode.h>
>> +
>> +#define CHIP_ID 0x004688
>> +#define OV4689_REG_CHIP_ID 0x300a
>> +
>> +#define OV4689_XVCLK_FREQ 24000000
>> +
>> +#define OV4689_REG_CTRL_MODE 0x0100
>> +#define OV4689_MODE_SW_STANDBY 0x0
>> +#define OV4689_MODE_STREAMING BIT(0)
>> +
>> +#define OV4689_REG_EXPOSURE 0x3500
>> +#define OV4689_EXPOSURE_MIN 4
>> +#define OV4689_EXPOSURE_STEP 1
>> +#define OV4689_VTS_MAX 0x7fff
>> +
>> +#define OV4689_REG_GAIN_H 0x3508
>> +#define OV4689_REG_GAIN_L 0x3509
>> +#define OV4689_GAIN_H_MASK 0x07
>> +#define OV4689_GAIN_H_SHIFT 8
>> +#define OV4689_GAIN_L_MASK 0xff
>> +#define OV4689_GAIN_MIN 0x10
>> +#define OV4689_GAIN_MAX 0xf8
>> +#define OV4689_GAIN_STEP 1
>> +#define OV4689_GAIN_DEFAULT 0x10
>> +
>> +#define OV4689_REG_TEST_PATTERN 0x5040
>> +#define OV4689_TEST_PATTERN_ENABLE 0x80
>> +#define OV4689_TEST_PATTERN_DISABLE 0x0
>> +
>> +#define OV4689_REG_VTS 0x380e
>> +
>> +#define REG_NULL 0xFFFF
>> +
>> +#define OV4689_REG_VALUE_08BIT 1
>> +#define OV4689_REG_VALUE_16BIT 2
>> +#define OV4689_REG_VALUE_24BIT 3
>> +
>> +#define OV4689_LANES 4
>> +#define OV4689_BITS_PER_SAMPLE 10
>> +
>> +static const char *const ov4689_supply_names[] = {
>> + "avdd", /* Analog power */
>> + "dovdd", /* Digital I/O power */
>> + "dvdd", /* Digital core power */
>> +};
>> +
>> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
>> +
>> +struct regval {
>> + u16 addr;
>> + u8 val;
>> +};
>
> What about use ov4689_mode_id? I think could be usefull for the future:
>
> + enum ov4689_mode_id {
> + OV4689_MODE_2688_1520 = 0,
> + OV4689_NUM_MODES,
> + };

Looks like a good idea, will add in v3.

>> +
>> +struct ov4689_mode {
> + enum ov4689_mode_id id;

Same.

>> + u32 width;
>> + u32 height;
>> + u32 max_fps;
>> + u32 hts_def;
>> + u32 vts_def;
>> + u32 exp_def;
>> + const struct regval *reg_list;
>> +};
>> +
>> +struct ov4689 {
>> + struct i2c_client *client;
>> + struct clk *xvclk;
>> + struct gpio_desc *reset_gpio;
>> + struct gpio_desc *pwdn_gpio;
>> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
>> +
>> + struct v4l2_subdev subdev;
>> + struct media_pad pad;
>> +
>> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
>> + bool streaming;
>> + struct v4l2_ctrl_handler ctrl_handler;
>> + struct v4l2_ctrl *exposure;
>> + struct v4l2_ctrl *anal_gain;
>> + struct v4l2_ctrl *digi_gain;
>> + struct v4l2_ctrl *hblank;
>> + struct v4l2_ctrl *vblank;
>> + struct v4l2_ctrl *test_pattern;
>> +
>> + const struct ov4689_mode *cur_mode;
>> +};
>> +
>> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
>> +
>> +/*
>> + * Xclk 24Mhz
>> + */
>> +static const struct regval ov4689_global_regs[] = {
>> + { REG_NULL, 0x00 },
>> +};
>> +
>> +/*
>> + * Xclk 24Mhz
>> + * max_framerate 30fps
>> + * mipi_datarate per lane 1008Mbps
>> + */
>> +static const struct regval ov4689_2688x1520_regs[] = {
>> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
>> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
>> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
>> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
>> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
>> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
>> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
>> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
>> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
>> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
>> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
>> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
>> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
>> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
>> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
>> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
>> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
>> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
>> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
>> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
>> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
>> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
>> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
>> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
>> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
>> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
>> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
>> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
>> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
>> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
>> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
>> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
>> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
>> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
>> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
>> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
>> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
>> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
>> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
>> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
>> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
>> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
>> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
>> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
>> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
>> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
>> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
>> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
>> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
>> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
>> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
>> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
>> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
>> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
>> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
>> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
>> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
>> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
>> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
>> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
>> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
>> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
>> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
>> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
>> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
>> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
>> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
>> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
>> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
>> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
>> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
>> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
>> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
>> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
>> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
>> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
>> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
>> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
>> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
>> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
>> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
>> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
>> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
>> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
>> + {REG_NULL, 0x00},
>> +};
>> +
>> +static const struct ov4689_mode supported_modes[] = {
>> + {
> + .id = OV4689_MODE_2688_1520,

Same.

>> + .width = 2688,
>> + .height = 1520,
>> + .max_fps = 30,
>> + .exp_def = 0x0600,
>> + .hts_def = 0x0a80,
>> + .vts_def = 0x0612,
>> + .reg_list = ov4689_2688x1520_regs,
>> + },
>> +};
>> +
>> +#define OV4689_LINK_FREQ_500MHZ 500000000
>> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
>> +
>> +static const char *const ov4689_test_pattern_menu[] = {
>> + "Disabled",
>> + "Vertical Color Bar Type 1",
>> + "Vertical Color Bar Type 2",
>> + "Vertical Color Bar Type 3",
>> + "Vertical Color Bar Type 4"
>> +};
>> +
>> +/* Write registers up to 4 at a time */
>> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
>> + u32 val)
>> +{
>> + u32 buf_i, val_i;
>> + __be32 val_be;
>> + u8 *val_p;
>> + u8 buf[6];
>> +
>> + if (len > 4)
>> + return -EINVAL;
>> +
>> + buf[0] = reg >> 8;
>> + buf[1] = reg & 0xff;
>> +
>> + val_be = cpu_to_be32(val);
>> + val_p = (u8 *)&val_be;
>> + buf_i = 2;
>> + val_i = 4 - len;
>> +
>> + while (val_i < 4)
>> + buf[buf_i++] = val_p[val_i++];
>> +
>> + if (i2c_master_send(client, buf, len + 2) != len + 2)
>> + return -EIO;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_write_array(struct i2c_client *client,
>> + const struct regval *regs)
>> +{
>> + int ret = 0;
>> + u32 i;
>> +
>> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
>> + ret = ov4689_write_reg(client, regs[i].addr,
>> + OV4689_REG_VALUE_08BIT, regs[i].val);
>> +
>> + return ret;
>> +}
>> +
>> +/* Read registers up to 4 at a time */
>> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
>> + u32 *val)
>> +{
>> + __be16 reg_addr_be = cpu_to_be16(reg);
>> + struct i2c_msg msgs[2];
>> + __be32 data_be = 0;
>> + u8 *data_be_p;
>> + int ret;
>> +
>> + if (len > 4 || !len)
>> + return -EINVAL;
>> +
>> + data_be_p = (u8 *)&data_be;
>> + /* Write register address */
>> + msgs[0].addr = client->addr;
>> + msgs[0].flags = 0;
>> + msgs[0].len = 2;
>> + msgs[0].buf = (u8 *)&reg_addr_be;
>> +
>> + /* Read data from register */
>> + msgs[1].addr = client->addr;
>> + msgs[1].flags = I2C_M_RD;
>> + msgs[1].len = len;
>> + msgs[1].buf = &data_be_p[4 - len];
>> +
>> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
>> + if (ret != ARRAY_SIZE(msgs))
>> + return -EIO;
>> +
>> + *val = be32_to_cpu(data_be);
>> +
>> + return 0;
>> +}
>> +
>> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
>> + struct v4l2_mbus_framefmt *fmt)
>> +{
>> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
>> + fmt->width = mode->width;
>> + fmt->height = mode->height;
>> + fmt->field = V4L2_FIELD_NONE;
>> +}
>> +
>> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + /* only one mode supported for now */
>> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + /* only one mode supported for now */
>> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + if (code->index != 0)
>> + return -EINVAL;
>> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + if (fse->index >= ARRAY_SIZE(supported_modes))
>> + return -EINVAL;
>> +
>> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
>> + return -EINVAL;
>> +
>> + fse->min_width = supported_modes[fse->index].width;
>> + fse->max_width = supported_modes[fse->index].width;
>> + fse->max_height = supported_modes[fse->index].height;
>> + fse->min_height = supported_modes[fse->index].height;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
>> +{
>> + u32 val;
>> +
>> + if (pattern)
>> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
>> + else
>> + val = OV4689_TEST_PATTERN_DISABLE;
>> +
>> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
>> + OV4689_REG_VALUE_08BIT, val);
>> +}
>> +
>> +static int ov4689_get_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
>> + return -EINVAL;
>> +
>> + switch (sel->target) {
>> + case V4L2_SEL_TGT_CROP_BOUNDS:
>> + sel->r.top = 0;
>> + sel->r.left = 0;
>> + sel->r.width = 2720;
>> + sel->r.height = 1536;
>> + return 0;
>> + case V4L2_SEL_TGT_CROP:
>> + case V4L2_SEL_TGT_CROP_DEFAULT:
>> + sel->r.top = 8;
>> + sel->r.left = 16;
>> + sel->r.width = 2688;
>> + sel->r.height = 1520;
>> + return 0;
>> + }
>> + return -EINVAL;
>> +}
>> +
>> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
>> +{
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> + struct i2c_client *client = ov4689->client;
>> + int ret = 0;
>> +
>> + mutex_lock(&ov4689->mutex);
>> +
>> + on = !!on;
>> + if (on == ov4689->streaming)
>> + goto unlock_and_return;
>> +
>> + if (on) {
>> + ret = pm_runtime_resume_and_get(&client->dev);
>> + if (ret < 0)
>> + goto unlock_and_return;
>> +
>> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> +
>> + ret = ov4689_write_array(ov4689->client,
>> + ov4689->cur_mode->reg_list);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> +
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
>> + OV4689_REG_VALUE_08BIT,
>> + OV4689_MODE_STREAMING);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> + } else {
>> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
>> + OV4689_REG_VALUE_08BIT,
>> + OV4689_MODE_SW_STANDBY);
>> + pm_runtime_put(&client->dev);
>> + }
>> +
>> + ov4689->streaming = on;
>> +
>> +unlock_and_return:
>> + mutex_unlock(&ov4689->mutex);
>> +
>> + return ret;
>> +}
>> +
>> +/* Calculate the delay in us by clock rate and clock cycles */
>> +static inline u32 ov4689_cal_delay(u32 cycles)
>> +{
>> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
>> +}
>> +
>> +static int __ov4689_power_on(struct ov4689 *ov4689)
>
> Just a doubt on this name function. Why __ ? Is this name reserved for?

Just a leftover from BSP driver, will clean this up in v3.

>> +{
>> + struct device *dev = &ov4689->client->dev;
>> + u32 delay_us;
>> + int ret;
>> +
>> + ret = clk_prepare_enable(ov4689->xvclk);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to enable xvclk\n");
>> + return ret;
>> + }
>> +
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
>> +
>> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to enable regulators\n");
>> + goto disable_clk;
>> + }
>> +
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
>> + usleep_range(500, 1000);
>> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
>> +
>> + /* 8192 cycles prior to first SCCB transaction */
>> + delay_us = ov4689_cal_delay(8192);
>> + usleep_range(delay_us, delay_us * 2);
>> +
>> + return 0;
>> +
>> +disable_clk:
>> + clk_disable_unprepare(ov4689->xvclk);
>> +
>> + return ret;
>> +}
>> +
>> +static void __ov4689_power_off(struct ov4689 *ov4689)
>> +{
>> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
>> + clk_disable_unprepare(ov4689->xvclk);
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
>> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
>> +}
>> +
>> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
>> +{
>> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + return __ov4689_power_on(ov4689);
>> +}
>> +
>> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
>> +{
>> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + __ov4689_power_off(ov4689);
>> +
>> + return 0;
>> +}
>> +
>> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
>> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
>> +{
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> + struct v4l2_mbus_framefmt *try_fmt;
>> +
>> + mutex_lock(&ov4689->mutex);
>> +
>> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
>> + /* Initialize try_fmt */
>> + ov4689_fill_fmt(&supported_modes[0], try_fmt);
>
> In this way instead of use magic number we can use this:
>
> ov4689_fill_fmt(&supported_modes[OV4689_MODE_2688_1520], try_fmt);

Ack, will fix in v3.

>> +
>> + mutex_unlock(&ov4689->mutex);
>> +
>> + return 0;
>> +}
>> +#endif
>> +
>> +static const struct dev_pm_ops ov4689_pm_ops = {
>> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
>> +};
>> +
>> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
>> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
>> + .open = ov4689_open,
>> +};
>> +#endif
>> +
>> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
>> + .s_stream = ov4689_s_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
>> + .enum_mbus_code = ov4689_enum_mbus_code,
>> + .enum_frame_size = ov4689_enum_frame_sizes,
>> + .get_fmt = ov4689_get_fmt,
>> + .set_fmt = ov4689_set_fmt,
>> + .get_selection = ov4689_get_selection,
>> +};
>> +
>> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
>> + .video = &ov4689_video_ops,
>> + .pad = &ov4689_pad_ops,
>> +};
>> +
>> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
>> +{
>> + struct ov4689 *ov4689 =
>> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
>> + struct i2c_client *client = ov4689->client;
>> + s64 max_expo;
>> + int ret;
>> +
>> + /* Propagate change of current control to all related controls */
>> + switch (ctrl->id) {
>> + case V4L2_CID_VBLANK:
>> + /* Update max exposure while meeting expected vblanking */
>> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
>> + __v4l2_ctrl_modify_range(ov4689->exposure,
>> + ov4689->exposure->minimum, max_expo,
>> + ov4689->exposure->step,
>> + ov4689->exposure->default_value);
>> + break;
>> + }
>> +
>> + if (!pm_runtime_get_if_in_use(&client->dev))
>> + return 0;
>> +
>> + switch (ctrl->id) {
>> + case V4L2_CID_EXPOSURE:
>> + /* 4 least significant bits of expsoure are fractional part */
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
>> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
>> + break;
>> + case V4L2_CID_ANALOGUE_GAIN:
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
>> + OV4689_REG_VALUE_08BIT,
>> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
>> + OV4689_GAIN_H_MASK);
>> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
>> + OV4689_REG_VALUE_08BIT,
>> + ctrl->val & OV4689_GAIN_L_MASK);
>> + break;
>> + case V4L2_CID_VBLANK:
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
>> + OV4689_REG_VALUE_16BIT,
>> + ctrl->val + ov4689->cur_mode->height);
>> + break;
>> + case V4L2_CID_TEST_PATTERN:
>> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
>> + break;
>> + default:
>> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
>> + __func__, ctrl->id, ctrl->val);
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + pm_runtime_put(&client->dev);
>> +
>> + return ret;
>> +}
>> +
>> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
>> + .s_ctrl = ov4689_set_ctrl,
>> +};
>> +
>> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
>> +{
>> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
>> + struct v4l2_fwnode_device_properties props;
>> + struct v4l2_ctrl_handler *handler;
>> + const struct ov4689_mode *mode;
>> + s64 exposure_max, vblank_def;
>> + struct v4l2_ctrl *ctrl;
>> + u32 h_blank, pixel_rate;
>> + int ret;
>> +
>> + handler = &ov4689->ctrl_handler;
>> + mode = ov4689->cur_mode;
>> + ret = v4l2_ctrl_handler_init(handler, 10);
>> + if (ret)
>> + return ret;
>> + handler->lock = &ov4689->mutex;
>> +
>> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
>> + link_freq_menu_items);
>> + if (ctrl)
>> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
>> + OV4689_BITS_PER_SAMPLE;
>> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
>> + pixel_rate);
>> +
>> + h_blank = mode->hts_def - mode->width;
>> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
>> + h_blank, h_blank, 1, h_blank);
>> + if (ov4689->hblank)
>> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + vblank_def = mode->vts_def - mode->height;
>> + ov4689->vblank =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
>> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
>> + vblank_def);
>> +
>> + exposure_max = mode->vts_def - 4;
>> + ov4689->exposure =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
>> + OV4689_EXPOSURE_MIN, exposure_max,
>> + OV4689_EXPOSURE_STEP, mode->exp_def);
>> +
>> + ov4689->anal_gain =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
>> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
>> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
>> + OV4689_GAIN_DEFAULT);
>> +
>> + ov4689->test_pattern =
>> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
>> + V4L2_CID_TEST_PATTERN,
>> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
>> + 0, 0, ov4689_test_pattern_menu);
>> +
>> + if (handler->error) {
>> + ret = handler->error;
>> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
>> + ret);
>> + goto err_free_handler;
>> + }
>> +
>> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
>> + if (ret)
>> + goto err_free_handler;
>> +
>> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
>> + &props);
>> + if (ret)
>> + goto err_free_handler;
>> +
>> + ov4689->subdev.ctrl_handler = handler;
>> +
>> + return 0;
>> +
>> +err_free_handler:
>> + v4l2_ctrl_handler_free(handler);
>> +
>> + return ret;
>> +}
>> +
>> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
>> + struct i2c_client *client)
>> +{
>> + struct device *dev = &ov4689->client->dev;
>> + u32 id = 0;
>> + int ret;
>> +
>> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
>> + OV4689_REG_VALUE_16BIT, &id);
>> + if (id != CHIP_ID) {
>> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
>> + return -ENODEV;
>> + }
>> +
>> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
>> + ov4689->supplies[i].supply = ov4689_supply_names[i];
>> +
>> + return devm_regulator_bulk_get(&ov4689->client->dev,
>> + OV4689_NUM_SUPPLIES, ov4689->supplies);
>> +}
>> +
>> +static int ov4689_check_hwcfg(struct device *dev)
>> +{
>> + struct fwnode_handle *fwnode = dev_fwnode(dev);
>> + struct v4l2_fwnode_endpoint bus_cfg = {
>> + .bus_type = V4L2_MBUS_CSI2_DPHY,
>> + };
>> + struct fwnode_handle *endpoint;
>> + unsigned int i;
>> + int ret;
>> +
>> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
>> + if (!endpoint)
>> + return -EPROBE_DEFER;
>> +
>> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
>> + fwnode_handle_put(endpoint);
>> + if (ret)
>> + return ret;
>> +
>> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
>> + dev_err(dev, "only a 4-lane CSI2 config is supported");
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> + if (!bus_cfg.nr_of_link_frequencies) {
>> + dev_err(dev, "no link frequencies defined\n");
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
>> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
>> + break;
>> +
>> + if (i == bus_cfg.nr_of_link_frequencies) {
>> + dev_err(dev, "supported link freq %ull not found\n",
>> + OV4689_LINK_FREQ_500MHZ);
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> +out_free_bus_cfg:
>> + v4l2_fwnode_endpoint_free(&bus_cfg);
>> +
>> + return ret;
>> +}
>> +
>> +static int ov4689_probe(struct i2c_client *client,
>> + const struct i2c_device_id *id)
>
> We are sure that we need need i2c_device_id *id?

I see, will convert to single-argument .probe_new in v3.

>> +{
>> + struct device *dev = &client->dev;
>> + struct v4l2_subdev *sd;
>> + struct ov4689 *ov4689;
>> + int ret;
>> +
>> + ret = ov4689_check_hwcfg(dev);
>> + if (ret)
>> + return ret;
>> +
>> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
>> + if (!ov4689)
>> + return -ENOMEM;
>> +
>> + ov4689->client = client;
>> + ov4689->cur_mode = &supported_modes[0];
>
> Here aswell we can use:
> ov4689->cur_mode = &supported_modes[OV4689_MODE_2688_1520]

Ack.

>> +
>> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
>> + if (IS_ERR(ov4689->xvclk)) {
>> + dev_err(dev, "Failed to get xvclk\n");
>> + return -EINVAL;
>> + }
>
> ^ I think is better to use devm_clk_get_optional instead of clck_get.
> clck_get can fail in CPU's that use ACPI
>
>> +
>> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
>> + return ret;
>> + }
>> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
>> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
>
>
> What do you think about?
> Thanks.

Unfortunately, I have no experience with ACPI-based devices. :(

Do you mean that in the case of an ACPI device and devm_clk_get_optional
returning NULL we should assume that the clock is already enabled and
will stay enabled during sensor operation? How should we distinguish it
from the case of an OF-based system and clock just missing from device
tree?

As a note to myself:

-static const struct ov4689_mode supported_modes[]
+static const struct ov4689_mode supported_modes[OV4689_NUM_MODES]

Overall, thanks for the review!

> Regards,
> Tommaso
>
--
Best regards,
Mikhail Rudenko

2022-09-15 22:24:52

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver


Hi Dave,

On 2022-09-14 at 10:58 +01, Dave Stevenson <[email protected]> wrote:
> Hi Mikhail
>
> On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
>>
>> Hello,
>>
>> this series implements support for Omnivision OV4689 image
>> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
>> megapixel image sensor. Ihis chip supports high frame rate speeds up
>> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
>> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
>> connection.
>>
>> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
>> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
>> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
>>
>> While porting the driver, I stumbled upon two issues:
>>
>> (1) In the original driver, horizontal total size (HTS) was set to a
>> value (2584) lower then the frame width (2688), resulting in negative
>> hblank. In this driver, I increased HTS to 2688, but fps dropped from
>> 29.88 to 28.73. What is the preferred way to handle this?
>
> This is one of the joys of sensors - they don't all work in the same way.
>
> I don't have an official datasheet for OV4689 from Omnivision, but
> found one on the internet [1]. That should allow you to reverse the
> PLL configuration to confirm that the pixel rate is the value you've
> computed based on link frequency (they aren't necessarily related). Do
> the frame rate calculations work using width + HBLANK, height +
> VBLANK, and pixel rate?
> The datasheet claims the sensor supports 2688x1520 @ 90 fps, so
> something doesn't hold true between 4 data lanes at 500MHz/1Gbit/s per
> lane when your default hts/vts is 2688x1554 and it only gives
> 28.73fps.

Seems like those 90 fps is about CSI throughput, not actual sensor
performance. I've checked the datasheet and the register values, and it
seems like the pixel clock is 126 Mhz in this configuration (the maximum
is 150 MHz according to the datasheet). This corresponds to a
theoretical fps of 30.16 at hts=2688 and vts=1554. At the same time the
observed fps is 28.73. I'm not sure where those 1.43 frames are lost,
hope to do more experimentation with VTS and HTS over the weekend.

> I have seen modes in sensors where the HTS register is in units of 2
> pixels, so what range of HTS (and VTS) values actually works on this
> sensor? (I don't see it documented, but I'm not surprised).
>
> [1] https://cdn.hackaday.io/files/19354828041536/OV4689-OmniVision.pdf
>
>> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
>> gain is not linear across that range. Instead, it is piecewise linear
>> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
>> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
>> with more linear segments in between. Rockchip's camera engine code
>> chooses one of the above segments depenging on the desired gain
>> value. The question is, how should we proceed keeping in mind
>> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
>> libcamera will do the mapping, or the driver will do the mapping
>> itself and expose some logical gain units not tied to the actual gain
>> register value? Meanwhile, this driver conservatively exposes only
>> 0x0-0xf8 gain register range.
>
> The datasheet linked above says "for the gain formula, please contact
> your local OmniVision FAE" :-(
> I would assume that the range is from 1x rather than 0x - people
> rarely want a totally black image that 0x would give. Or is it ranges
> of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?

A picture is worth a thousand words, so I've attached the results of my
experimentation with the gain register. They were obtained with Rockchip
3399, with AEC, AGC and black level subtraction disabled. The image was
converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.

> Other sensors expose the full range of the register via
> V4L2_CID_ANALOGUE_GAIN, and require userspace (mainly libcamera now)
> to know how to convert a gain into the register value. If the gain
> range goes up to x16, then exposing that would be useful. I'd advocate
> just exposing the full range of 0x000 - 0x7ff, as then you can have
> the accuracy of 256 values between x1 to x2, but also the full range.

I also like this approach, although libcamera's CameraSensorHelper
doesn't support piecewise-linear gain code mapping yet. Nevertheless,
I believe exposing the full range is a good idea and will do so in v3.

> I might see if I can pick up one of these sensors and see if I can get
> it running on a Raspberry Pi. Thanks for trying to upstream this -
> it's nice to have such a range of sensor drivers to choose from.
>
> Dave
>

Thanks for your elucidating tips!

--
Best regards,
Mikhail Rudenko


Attachments:
gain.png (47.98 kB)

2022-09-15 22:41:56

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689


Hi Krzysztof,

On 2022-09-12 at 12:56 +02, Krzysztof Kozlowski <[email protected]> wrote:
> On 11/09/2022 22:01, Mikhail Rudenko wrote:
>> +static const struct i2c_device_id ov4689_id[] = {
>> + { "ov4689", 0 },
>> + {},
>> +};
>> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
>> +
>> +static const struct of_device_id ov4689_of_match[] = {
>> + { .compatible = "ovti,ov4689" },
>> + {},
>> +};
>> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
>> +
>> +static struct i2c_driver ov4689_i2c_driver = {
>> + .driver = {
>> + .name = "ov4689",
>> + .pm = &ov4689_pm_ops,
>> + .of_match_table = of_match_ptr(ov4689_of_match),
>
> of_match_ptr is usually paired with maybe_unused, otherwise you will
> have compile test warnings.

I see. I think we could also use `#if IS_ENABLED(CONFIG_OF)` around
`ov4689_of_match` and the corresponding `MODULE_DEVICE_TABLE`. Is it
appropriate here?

>> + },
>> + .probe = ov4689_probe,
>> + .remove = ov4689_remove,
>> + .id_table = ov4689_id,
>> +};
>> +
>> +module_i2c_driver(ov4689_i2c_driver);
>> +
>> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
>> +MODULE_LICENSE("GPL");
>
>
> Best regards,
> Krzysztof


--
Best regards,
Mikhail Rudenko

2022-09-16 10:08:56

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

On 15/09/2022 21:40, Mikhail Rudenko wrote:
>
> Hi Krzysztof,
>
> On 2022-09-12 at 12:56 +02, Krzysztof Kozlowski <[email protected]> wrote:
>> On 11/09/2022 22:01, Mikhail Rudenko wrote:
>>> +static const struct i2c_device_id ov4689_id[] = {
>>> + { "ov4689", 0 },
>>> + {},
>>> +};
>>> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
>>> +
>>> +static const struct of_device_id ov4689_of_match[] = {
>>> + { .compatible = "ovti,ov4689" },
>>> + {},
>>> +};
>>> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
>>> +
>>> +static struct i2c_driver ov4689_i2c_driver = {
>>> + .driver = {
>>> + .name = "ov4689",
>>> + .pm = &ov4689_pm_ops,
>>> + .of_match_table = of_match_ptr(ov4689_of_match),
>>
>> of_match_ptr is usually paired with maybe_unused, otherwise you will
>> have compile test warnings.
>
> I see. I think we could also use `#if IS_ENABLED(CONFIG_OF)` around
> `ov4689_of_match` and the corresponding `MODULE_DEVICE_TABLE`. Is it
> appropriate here?

Would work, but ifdefs are not nice. Just use maybe_unused. Warnings
should disappear.

Best regards,
Krzysztof

2022-09-16 13:44:20

by Tommaso Merciai

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,

On Thu, Sep 15, 2022 at 11:50:23PM +0300, Mikhail Rudenko wrote:
>
> Hi Tommaso,
>
> On 2022-09-14 at 17:51 +02, Tommaso Merciai <[email protected]> wrote:
> > Hi Mikhail,
> > I do a first round on reviewing your driver :)
> >
> > On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
> >> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
> >> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
> >> bus for data.
> >>
> >> This driver supports following features:
> >> - manual exposure and analog gain control support
> >> - test pattern support
> >> - media controller support
> >> - runtime PM support
> >> - support following resolutions:
> >> + 2688x1520 at 30 fps
> >>
> >> The driver provides all mandatory V4L2 controls for compatibility with
> >> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
> >> implements 4 lane mode only at this moment.
> >>
> >> Signed-off-by: Mikhail Rudenko <[email protected]>
> >> ---
> >> MAINTAINERS | 1 +
> >> drivers/media/i2c/Kconfig | 14 +
> >> drivers/media/i2c/Makefile | 1 +
> >> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
> >> 4 files changed, 967 insertions(+)
> >> create mode 100644 drivers/media/i2c/ov4689.c
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index 63c4844f26e6..1857f3864e1b 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -14529,6 +14529,7 @@ L: [email protected]
> >> S: Maintained
> >> T: git git://linuxtv.org/media_tree.git
> >> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
> >> +F: drivers/media/i2c/ov5647.c
> >>
> >> OMNIVISION OV5640 SENSOR DRIVER
> >> M: Steve Longerbeam <[email protected]>
> >> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
> >> index fae2baabb773..4993e1ae2ea8 100644
> >> --- a/drivers/media/i2c/Kconfig
> >> +++ b/drivers/media/i2c/Kconfig
> >> @@ -429,6 +429,20 @@ config VIDEO_OV2740
> >> To compile this driver as a module, choose M here: the
> >> module will be called ov2740.
> >>
> >> +config VIDEO_OV4689
> >> + tristate "OmniVision OV4689 sensor support"
> >> + depends on OF
> >> + depends on GPIOLIB && VIDEO_DEV && I2C
> >> + select MEDIA_CONTROLLER
> >> + select VIDEO_V4L2_SUBDEV_API
> >> + select V4L2_FWNODE
> >> + help
> >> + This is a Video4Linux2 sensor-level driver for the OmniVision
> >> + OV4689 camera.
> >> +
> >> + To compile this driver as a module, choose M here: the
> >> + module will be called ov4689.
> >> +
> >> config VIDEO_OV5640
> >> tristate "OmniVision OV5640 sensor support"
> >> depends on OF
> >> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
> >> index 3e1696963e7f..7446c0a1eed0 100644
> >> --- a/drivers/media/i2c/Makefile
> >> +++ b/drivers/media/i2c/Makefile
> >> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
> >> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
> >> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
> >> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
> >> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
> >> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
> >> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
> >> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
> >> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
> >> new file mode 100644
> >> index 000000000000..9f05e812acf8
> >> --- /dev/null
> >> +++ b/drivers/media/i2c/ov4689.c
> >> @@ -0,0 +1,951 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * ov4689 driver
> >> + *
> >> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
> >> + */
> >> +
> >> +#include <linux/clk.h>
> >> +#include <linux/device.h>
> >> +#include <linux/delay.h>
> >> +#include <linux/gpio/consumer.h>
> >> +#include <linux/i2c.h>
> >> +#include <linux/module.h>
> >> +#include <linux/pm_runtime.h>
> >> +#include <linux/regulator/consumer.h>
> >> +#include <media/media-entity.h>
> >> +#include <media/v4l2-async.h>
> >> +#include <media/v4l2-ctrls.h>
> >> +#include <media/v4l2-subdev.h>
> >> +#include <media/v4l2-fwnode.h>
> >> +
> >> +#define CHIP_ID 0x004688
> >> +#define OV4689_REG_CHIP_ID 0x300a
> >> +
> >> +#define OV4689_XVCLK_FREQ 24000000
> >> +
> >> +#define OV4689_REG_CTRL_MODE 0x0100
> >> +#define OV4689_MODE_SW_STANDBY 0x0
> >> +#define OV4689_MODE_STREAMING BIT(0)
> >> +
> >> +#define OV4689_REG_EXPOSURE 0x3500
> >> +#define OV4689_EXPOSURE_MIN 4
> >> +#define OV4689_EXPOSURE_STEP 1
> >> +#define OV4689_VTS_MAX 0x7fff
> >> +
> >> +#define OV4689_REG_GAIN_H 0x3508
> >> +#define OV4689_REG_GAIN_L 0x3509
> >> +#define OV4689_GAIN_H_MASK 0x07
> >> +#define OV4689_GAIN_H_SHIFT 8
> >> +#define OV4689_GAIN_L_MASK 0xff
> >> +#define OV4689_GAIN_MIN 0x10
> >> +#define OV4689_GAIN_MAX 0xf8
> >> +#define OV4689_GAIN_STEP 1
> >> +#define OV4689_GAIN_DEFAULT 0x10
> >> +
> >> +#define OV4689_REG_TEST_PATTERN 0x5040
> >> +#define OV4689_TEST_PATTERN_ENABLE 0x80
> >> +#define OV4689_TEST_PATTERN_DISABLE 0x0
> >> +
> >> +#define OV4689_REG_VTS 0x380e
> >> +
> >> +#define REG_NULL 0xFFFF
> >> +
> >> +#define OV4689_REG_VALUE_08BIT 1
> >> +#define OV4689_REG_VALUE_16BIT 2
> >> +#define OV4689_REG_VALUE_24BIT 3
> >> +
> >> +#define OV4689_LANES 4
> >> +#define OV4689_BITS_PER_SAMPLE 10
> >> +
> >> +static const char *const ov4689_supply_names[] = {
> >> + "avdd", /* Analog power */
> >> + "dovdd", /* Digital I/O power */
> >> + "dvdd", /* Digital core power */
> >> +};
> >> +
> >> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
> >> +
> >> +struct regval {
> >> + u16 addr;
> >> + u8 val;
> >> +};
> >
> > What about use ov4689_mode_id? I think could be usefull for the future:
> >
> > + enum ov4689_mode_id {
> > + OV4689_MODE_2688_1520 = 0,
> > + OV4689_NUM_MODES,
> > + };
>
> Looks like a good idea, will add in v3.
>
> >> +
> >> +struct ov4689_mode {
> > + enum ov4689_mode_id id;
>
> Same.
>
> >> + u32 width;
> >> + u32 height;
> >> + u32 max_fps;
> >> + u32 hts_def;
> >> + u32 vts_def;
> >> + u32 exp_def;
> >> + const struct regval *reg_list;
> >> +};
> >> +
> >> +struct ov4689 {
> >> + struct i2c_client *client;
> >> + struct clk *xvclk;
> >> + struct gpio_desc *reset_gpio;
> >> + struct gpio_desc *pwdn_gpio;
> >> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
> >> +
> >> + struct v4l2_subdev subdev;
> >> + struct media_pad pad;
> >> +
> >> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
> >> + bool streaming;
> >> + struct v4l2_ctrl_handler ctrl_handler;
> >> + struct v4l2_ctrl *exposure;
> >> + struct v4l2_ctrl *anal_gain;
> >> + struct v4l2_ctrl *digi_gain;
> >> + struct v4l2_ctrl *hblank;
> >> + struct v4l2_ctrl *vblank;
> >> + struct v4l2_ctrl *test_pattern;
> >> +
> >> + const struct ov4689_mode *cur_mode;
> >> +};
> >> +
> >> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
> >> +
> >> +/*
> >> + * Xclk 24Mhz
> >> + */
> >> +static const struct regval ov4689_global_regs[] = {
> >> + { REG_NULL, 0x00 },
> >> +};
> >> +
> >> +/*
> >> + * Xclk 24Mhz
> >> + * max_framerate 30fps
> >> + * mipi_datarate per lane 1008Mbps
> >> + */
> >> +static const struct regval ov4689_2688x1520_regs[] = {
> >> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
> >> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
> >> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
> >> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
> >> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
> >> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
> >> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
> >> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
> >> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
> >> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
> >> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
> >> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
> >> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
> >> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
> >> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
> >> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
> >> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
> >> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
> >> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
> >> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
> >> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
> >> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
> >> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
> >> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
> >> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
> >> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
> >> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
> >> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
> >> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
> >> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
> >> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
> >> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
> >> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
> >> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
> >> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
> >> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
> >> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
> >> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
> >> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
> >> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
> >> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
> >> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
> >> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
> >> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
> >> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
> >> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
> >> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
> >> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
> >> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
> >> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
> >> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
> >> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
> >> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
> >> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
> >> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
> >> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
> >> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
> >> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
> >> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
> >> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
> >> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
> >> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
> >> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
> >> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
> >> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
> >> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
> >> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
> >> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
> >> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
> >> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
> >> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
> >> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
> >> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
> >> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
> >> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
> >> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
> >> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
> >> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
> >> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
> >> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
> >> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
> >> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
> >> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
> >> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
> >> + {REG_NULL, 0x00},
> >> +};
> >> +
> >> +static const struct ov4689_mode supported_modes[] = {
> >> + {
> > + .id = OV4689_MODE_2688_1520,
>
> Same.
>
> >> + .width = 2688,
> >> + .height = 1520,
> >> + .max_fps = 30,
> >> + .exp_def = 0x0600,
> >> + .hts_def = 0x0a80,
> >> + .vts_def = 0x0612,
> >> + .reg_list = ov4689_2688x1520_regs,
> >> + },
> >> +};
> >> +
> >> +#define OV4689_LINK_FREQ_500MHZ 500000000
> >> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
> >> +
> >> +static const char *const ov4689_test_pattern_menu[] = {
> >> + "Disabled",
> >> + "Vertical Color Bar Type 1",
> >> + "Vertical Color Bar Type 2",
> >> + "Vertical Color Bar Type 3",
> >> + "Vertical Color Bar Type 4"
> >> +};
> >> +
> >> +/* Write registers up to 4 at a time */
> >> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
> >> + u32 val)
> >> +{
> >> + u32 buf_i, val_i;
> >> + __be32 val_be;
> >> + u8 *val_p;
> >> + u8 buf[6];
> >> +
> >> + if (len > 4)
> >> + return -EINVAL;
> >> +
> >> + buf[0] = reg >> 8;
> >> + buf[1] = reg & 0xff;
> >> +
> >> + val_be = cpu_to_be32(val);
> >> + val_p = (u8 *)&val_be;
> >> + buf_i = 2;
> >> + val_i = 4 - len;
> >> +
> >> + while (val_i < 4)
> >> + buf[buf_i++] = val_p[val_i++];
> >> +
> >> + if (i2c_master_send(client, buf, len + 2) != len + 2)
> >> + return -EIO;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_write_array(struct i2c_client *client,
> >> + const struct regval *regs)
> >> +{
> >> + int ret = 0;
> >> + u32 i;
> >> +
> >> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
> >> + ret = ov4689_write_reg(client, regs[i].addr,
> >> + OV4689_REG_VALUE_08BIT, regs[i].val);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +/* Read registers up to 4 at a time */
> >> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
> >> + u32 *val)
> >> +{
> >> + __be16 reg_addr_be = cpu_to_be16(reg);
> >> + struct i2c_msg msgs[2];
> >> + __be32 data_be = 0;
> >> + u8 *data_be_p;
> >> + int ret;
> >> +
> >> + if (len > 4 || !len)
> >> + return -EINVAL;
> >> +
> >> + data_be_p = (u8 *)&data_be;
> >> + /* Write register address */
> >> + msgs[0].addr = client->addr;
> >> + msgs[0].flags = 0;
> >> + msgs[0].len = 2;
> >> + msgs[0].buf = (u8 *)&reg_addr_be;
> >> +
> >> + /* Read data from register */
> >> + msgs[1].addr = client->addr;
> >> + msgs[1].flags = I2C_M_RD;
> >> + msgs[1].len = len;
> >> + msgs[1].buf = &data_be_p[4 - len];
> >> +
> >> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
> >> + if (ret != ARRAY_SIZE(msgs))
> >> + return -EIO;
> >> +
> >> + *val = be32_to_cpu(data_be);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
> >> + struct v4l2_mbus_framefmt *fmt)
> >> +{
> >> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> >> + fmt->width = mode->width;
> >> + fmt->height = mode->height;
> >> + fmt->field = V4L2_FIELD_NONE;
> >> +}
> >> +
> >> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *fmt)
> >> +{
> >> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> +
> >> + /* only one mode supported for now */
> >> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *fmt)
> >> +{
> >> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> +
> >> + /* only one mode supported for now */
> >> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> + if (code->index != 0)
> >> + return -EINVAL;
> >> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_frame_size_enum *fse)
> >> +{
> >> + if (fse->index >= ARRAY_SIZE(supported_modes))
> >> + return -EINVAL;
> >> +
> >> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
> >> + return -EINVAL;
> >> +
> >> + fse->min_width = supported_modes[fse->index].width;
> >> + fse->max_width = supported_modes[fse->index].width;
> >> + fse->max_height = supported_modes[fse->index].height;
> >> + fse->min_height = supported_modes[fse->index].height;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
> >> +{
> >> + u32 val;
> >> +
> >> + if (pattern)
> >> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
> >> + else
> >> + val = OV4689_TEST_PATTERN_DISABLE;
> >> +
> >> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
> >> + OV4689_REG_VALUE_08BIT, val);
> >> +}
> >> +
> >> +static int ov4689_get_selection(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_selection *sel)
> >> +{
> >> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> >> + return -EINVAL;
> >> +
> >> + switch (sel->target) {
> >> + case V4L2_SEL_TGT_CROP_BOUNDS:
> >> + sel->r.top = 0;
> >> + sel->r.left = 0;
> >> + sel->r.width = 2720;
> >> + sel->r.height = 1536;
> >> + return 0;
> >> + case V4L2_SEL_TGT_CROP:
> >> + case V4L2_SEL_TGT_CROP_DEFAULT:
> >> + sel->r.top = 8;
> >> + sel->r.left = 16;
> >> + sel->r.width = 2688;
> >> + sel->r.height = 1520;
> >> + return 0;
> >> + }
> >> + return -EINVAL;
> >> +}
> >> +
> >> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
> >> +{
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> + struct i2c_client *client = ov4689->client;
> >> + int ret = 0;
> >> +
> >> + mutex_lock(&ov4689->mutex);
> >> +
> >> + on = !!on;
> >> + if (on == ov4689->streaming)
> >> + goto unlock_and_return;
> >> +
> >> + if (on) {
> >> + ret = pm_runtime_resume_and_get(&client->dev);
> >> + if (ret < 0)
> >> + goto unlock_and_return;
> >> +
> >> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
> >> + if (ret) {
> >> + pm_runtime_put(&client->dev);
> >> + goto unlock_and_return;
> >> + }
> >> +
> >> + ret = ov4689_write_array(ov4689->client,
> >> + ov4689->cur_mode->reg_list);
> >> + if (ret) {
> >> + pm_runtime_put(&client->dev);
> >> + goto unlock_and_return;
> >> + }
> >> +
> >> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> >> + OV4689_REG_VALUE_08BIT,
> >> + OV4689_MODE_STREAMING);
> >> + if (ret) {
> >> + pm_runtime_put(&client->dev);
> >> + goto unlock_and_return;
> >> + }
> >> + } else {
> >> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> >> + OV4689_REG_VALUE_08BIT,
> >> + OV4689_MODE_SW_STANDBY);
> >> + pm_runtime_put(&client->dev);
> >> + }
> >> +
> >> + ov4689->streaming = on;
> >> +
> >> +unlock_and_return:
> >> + mutex_unlock(&ov4689->mutex);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +/* Calculate the delay in us by clock rate and clock cycles */
> >> +static inline u32 ov4689_cal_delay(u32 cycles)
> >> +{
> >> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
> >> +}
> >> +
> >> +static int __ov4689_power_on(struct ov4689 *ov4689)
> >
> > Just a doubt on this name function. Why __ ? Is this name reserved for?
>
> Just a leftover from BSP driver, will clean this up in v3.
>
> >> +{
> >> + struct device *dev = &ov4689->client->dev;
> >> + u32 delay_us;
> >> + int ret;
> >> +
> >> + ret = clk_prepare_enable(ov4689->xvclk);
> >> + if (ret < 0) {
> >> + dev_err(dev, "Failed to enable xvclk\n");
> >> + return ret;
> >> + }
> >> +
> >> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> >> +
> >> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> >> + if (ret < 0) {
> >> + dev_err(dev, "Failed to enable regulators\n");
> >> + goto disable_clk;
> >> + }
> >> +
> >> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
> >> + usleep_range(500, 1000);
> >> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
> >> +
> >> + /* 8192 cycles prior to first SCCB transaction */
> >> + delay_us = ov4689_cal_delay(8192);
> >> + usleep_range(delay_us, delay_us * 2);
> >> +
> >> + return 0;
> >> +
> >> +disable_clk:
> >> + clk_disable_unprepare(ov4689->xvclk);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static void __ov4689_power_off(struct ov4689 *ov4689)
> >> +{
> >> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
> >> + clk_disable_unprepare(ov4689->xvclk);
> >> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> >> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> >> +}
> >> +
> >> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
> >> +{
> >> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> +
> >> + return __ov4689_power_on(ov4689);
> >> +}
> >> +
> >> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
> >> +{
> >> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> +
> >> + __ov4689_power_off(ov4689);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> >> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> >> +{
> >> + struct ov4689 *ov4689 = to_ov4689(sd);
> >> + struct v4l2_mbus_framefmt *try_fmt;
> >> +
> >> + mutex_lock(&ov4689->mutex);
> >> +
> >> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
> >> + /* Initialize try_fmt */
> >> + ov4689_fill_fmt(&supported_modes[0], try_fmt);
> >
> > In this way instead of use magic number we can use this:
> >
> > ov4689_fill_fmt(&supported_modes[OV4689_MODE_2688_1520], try_fmt);
>
> Ack, will fix in v3.
>
> >> +
> >> + mutex_unlock(&ov4689->mutex);
> >> +
> >> + return 0;
> >> +}
> >> +#endif
> >> +
> >> +static const struct dev_pm_ops ov4689_pm_ops = {
> >> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
> >> +};
> >> +
> >> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> >> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
> >> + .open = ov4689_open,
> >> +};
> >> +#endif
> >> +
> >> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
> >> + .s_stream = ov4689_s_stream,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
> >> + .enum_mbus_code = ov4689_enum_mbus_code,
> >> + .enum_frame_size = ov4689_enum_frame_sizes,
> >> + .get_fmt = ov4689_get_fmt,
> >> + .set_fmt = ov4689_set_fmt,
> >> + .get_selection = ov4689_get_selection,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
> >> + .video = &ov4689_video_ops,
> >> + .pad = &ov4689_pad_ops,
> >> +};
> >> +
> >> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
> >> +{
> >> + struct ov4689 *ov4689 =
> >> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
> >> + struct i2c_client *client = ov4689->client;
> >> + s64 max_expo;
> >> + int ret;
> >> +
> >> + /* Propagate change of current control to all related controls */
> >> + switch (ctrl->id) {
> >> + case V4L2_CID_VBLANK:
> >> + /* Update max exposure while meeting expected vblanking */
> >> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
> >> + __v4l2_ctrl_modify_range(ov4689->exposure,
> >> + ov4689->exposure->minimum, max_expo,
> >> + ov4689->exposure->step,
> >> + ov4689->exposure->default_value);
> >> + break;
> >> + }
> >> +
> >> + if (!pm_runtime_get_if_in_use(&client->dev))
> >> + return 0;
> >> +
> >> + switch (ctrl->id) {
> >> + case V4L2_CID_EXPOSURE:
> >> + /* 4 least significant bits of expsoure are fractional part */
> >> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
> >> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
> >> + break;
> >> + case V4L2_CID_ANALOGUE_GAIN:
> >> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
> >> + OV4689_REG_VALUE_08BIT,
> >> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
> >> + OV4689_GAIN_H_MASK);
> >> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
> >> + OV4689_REG_VALUE_08BIT,
> >> + ctrl->val & OV4689_GAIN_L_MASK);
> >> + break;
> >> + case V4L2_CID_VBLANK:
> >> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
> >> + OV4689_REG_VALUE_16BIT,
> >> + ctrl->val + ov4689->cur_mode->height);
> >> + break;
> >> + case V4L2_CID_TEST_PATTERN:
> >> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
> >> + break;
> >> + default:
> >> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
> >> + __func__, ctrl->id, ctrl->val);
> >> + ret = -EINVAL;
> >> + break;
> >> + }
> >> +
> >> + pm_runtime_put(&client->dev);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
> >> + .s_ctrl = ov4689_set_ctrl,
> >> +};
> >> +
> >> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
> >> +{
> >> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
> >> + struct v4l2_fwnode_device_properties props;
> >> + struct v4l2_ctrl_handler *handler;
> >> + const struct ov4689_mode *mode;
> >> + s64 exposure_max, vblank_def;
> >> + struct v4l2_ctrl *ctrl;
> >> + u32 h_blank, pixel_rate;
> >> + int ret;
> >> +
> >> + handler = &ov4689->ctrl_handler;
> >> + mode = ov4689->cur_mode;
> >> + ret = v4l2_ctrl_handler_init(handler, 10);
> >> + if (ret)
> >> + return ret;
> >> + handler->lock = &ov4689->mutex;
> >> +
> >> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
> >> + link_freq_menu_items);
> >> + if (ctrl)
> >> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >> +
> >> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
> >> + OV4689_BITS_PER_SAMPLE;
> >> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
> >> + pixel_rate);
> >> +
> >> + h_blank = mode->hts_def - mode->width;
> >> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
> >> + h_blank, h_blank, 1, h_blank);
> >> + if (ov4689->hblank)
> >> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >> +
> >> + vblank_def = mode->vts_def - mode->height;
> >> + ov4689->vblank =
> >> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
> >> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
> >> + vblank_def);
> >> +
> >> + exposure_max = mode->vts_def - 4;
> >> + ov4689->exposure =
> >> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
> >> + OV4689_EXPOSURE_MIN, exposure_max,
> >> + OV4689_EXPOSURE_STEP, mode->exp_def);
> >> +
> >> + ov4689->anal_gain =
> >> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
> >> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
> >> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
> >> + OV4689_GAIN_DEFAULT);
> >> +
> >> + ov4689->test_pattern =
> >> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
> >> + V4L2_CID_TEST_PATTERN,
> >> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
> >> + 0, 0, ov4689_test_pattern_menu);
> >> +
> >> + if (handler->error) {
> >> + ret = handler->error;
> >> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
> >> + ret);
> >> + goto err_free_handler;
> >> + }
> >> +
> >> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
> >> + if (ret)
> >> + goto err_free_handler;
> >> +
> >> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
> >> + &props);
> >> + if (ret)
> >> + goto err_free_handler;
> >> +
> >> + ov4689->subdev.ctrl_handler = handler;
> >> +
> >> + return 0;
> >> +
> >> +err_free_handler:
> >> + v4l2_ctrl_handler_free(handler);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
> >> + struct i2c_client *client)
> >> +{
> >> + struct device *dev = &ov4689->client->dev;
> >> + u32 id = 0;
> >> + int ret;
> >> +
> >> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
> >> + OV4689_REG_VALUE_16BIT, &id);
> >> + if (id != CHIP_ID) {
> >> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
> >> + return -ENODEV;
> >> + }
> >> +
> >> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
> >> +{
> >> + unsigned int i;
> >> +
> >> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
> >> + ov4689->supplies[i].supply = ov4689_supply_names[i];
> >> +
> >> + return devm_regulator_bulk_get(&ov4689->client->dev,
> >> + OV4689_NUM_SUPPLIES, ov4689->supplies);
> >> +}
> >> +
> >> +static int ov4689_check_hwcfg(struct device *dev)
> >> +{
> >> + struct fwnode_handle *fwnode = dev_fwnode(dev);
> >> + struct v4l2_fwnode_endpoint bus_cfg = {
> >> + .bus_type = V4L2_MBUS_CSI2_DPHY,
> >> + };
> >> + struct fwnode_handle *endpoint;
> >> + unsigned int i;
> >> + int ret;
> >> +
> >> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
> >> + if (!endpoint)
> >> + return -EPROBE_DEFER;
> >> +
> >> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
> >> + fwnode_handle_put(endpoint);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
> >> + dev_err(dev, "only a 4-lane CSI2 config is supported");
> >> + ret = -EINVAL;
> >> + goto out_free_bus_cfg;
> >> + }
> >> +
> >> + if (!bus_cfg.nr_of_link_frequencies) {
> >> + dev_err(dev, "no link frequencies defined\n");
> >> + ret = -EINVAL;
> >> + goto out_free_bus_cfg;
> >> + }
> >> +
> >> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
> >> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
> >> + break;
> >> +
> >> + if (i == bus_cfg.nr_of_link_frequencies) {
> >> + dev_err(dev, "supported link freq %ull not found\n",
> >> + OV4689_LINK_FREQ_500MHZ);
> >> + ret = -EINVAL;
> >> + goto out_free_bus_cfg;
> >> + }
> >> +
> >> +out_free_bus_cfg:
> >> + v4l2_fwnode_endpoint_free(&bus_cfg);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +static int ov4689_probe(struct i2c_client *client,
> >> + const struct i2c_device_id *id)
> >
> > We are sure that we need need i2c_device_id *id?
>
> I see, will convert to single-argument .probe_new in v3.
>
> >> +{
> >> + struct device *dev = &client->dev;
> >> + struct v4l2_subdev *sd;
> >> + struct ov4689 *ov4689;
> >> + int ret;
> >> +
> >> + ret = ov4689_check_hwcfg(dev);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
> >> + if (!ov4689)
> >> + return -ENOMEM;
> >> +
> >> + ov4689->client = client;
> >> + ov4689->cur_mode = &supported_modes[0];
> >
> > Here aswell we can use:
> > ov4689->cur_mode = &supported_modes[OV4689_MODE_2688_1520]
>
> Ack.
>
> >> +
> >> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
> >> + if (IS_ERR(ov4689->xvclk)) {
> >> + dev_err(dev, "Failed to get xvclk\n");
> >> + return -EINVAL;
> >> + }
> >
> > ^ I think is better to use devm_clk_get_optional instead of clck_get.
> > clck_get can fail in CPU's that use ACPI
> >
> >> +
> >> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> >> + if (ret < 0) {
> >> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> >> + return ret;
> >> + }
> >> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> >> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> >
> >
> > What do you think about?
> > Thanks.
>
> Unfortunately, I have no experience with ACPI-based devices. :(
>
> Do you mean that in the case of an ACPI device and devm_clk_get_optional
> returning NULL we should assume that the clock is already enabled and
> will stay enabled during sensor operation? How should we distinguish it
> from the case of an OF-based system and clock just missing from device
> tree?

Not exaclty :)

I copy comment from [1]

if you use ov5693->xvclk to identify the ACPI vs OF use case shouldn't
you use the get_optionl() version ? Otherwise in the ACPI case you will have
-ENOENT if there's not 'xvclk' property and bail out.

Unless my understanding is wrong on ACPI we have "clock-frequency" and
on OF "xvclk" with an "assigned-clock-rates",

[1] https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

Let me know if you need more details.

Regards,
Tommaso

>
> As a note to myself:
>
> -static const struct ov4689_mode supported_modes[]
> +static const struct ov4689_mode supported_modes[OV4689_NUM_MODES]
>
> Overall, thanks for the review!
>
> > Regards,
> > Tommaso
> >
> --
> Best regards,
> Mikhail Rudenko

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
http://www.amarulasolutions.com

2022-09-16 14:16:23

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689


On 2022-09-16 at 15:34 +02, Tommaso Merciai <[email protected]> wrote:
> Hi Mikhail,
>
> On Thu, Sep 15, 2022 at 11:50:23PM +0300, Mikhail Rudenko wrote:
>>
>> Hi Tommaso,
>>
>> On 2022-09-14 at 17:51 +02, Tommaso Merciai <[email protected]> wrote:
>> > Hi Mikhail,
>> > I do a first round on reviewing your driver :)
>> >
>> > On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:

<snip>

>> >> +
>> >> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
>> >> + if (IS_ERR(ov4689->xvclk)) {
>> >> + dev_err(dev, "Failed to get xvclk\n");
>> >> + return -EINVAL;
>> >> + }
>> >
>> > ^ I think is better to use devm_clk_get_optional instead of clck_get.
>> > clck_get can fail in CPU's that use ACPI
>> >
>> >> +
>> >> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
>> >> + if (ret < 0) {
>> >> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
>> >> + return ret;
>> >> + }
>> >> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
>> >> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
>> >
>> >
>> > What do you think about?
>> > Thanks.
>>
>> Unfortunately, I have no experience with ACPI-based devices. :(
>>
>> Do you mean that in the case of an ACPI device and devm_clk_get_optional
>> returning NULL we should assume that the clock is already enabled and
>> will stay enabled during sensor operation? How should we distinguish it
>> from the case of an OF-based system and clock just missing from device
>> tree?
>
> Not exaclty :)
>
> I copy comment from [1]
>
> if you use ov5693->xvclk to identify the ACPI vs OF use case shouldn't
> you use the get_optionl() version ? Otherwise in the ACPI case you will have
> -ENOENT if there's not 'xvclk' property and bail out.
>
> Unless my understanding is wrong on ACPI we have "clock-frequency" and
> on OF "xvclk" with an "assigned-clock-rates",
>
> [1] https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/
>
> Let me know if you need more details.

Thanks for the pointer! I'll try to implement something along the lines
of your ov5693 series.

But I'm not sure that will be enough to support ACPI systems
correctly. What about lanes number and link frequency checks? Should
they be made conditional on CONFIG_OF? Anything else I don't know?

>
> Regards,
> Tommaso
>
--
Best regards,
Mikhail Rudenko

2022-09-16 22:23:33

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Krzysztof, Mikhail,

On Fri, Sep 16, 2022 at 10:43:11AM +0100, Krzysztof Kozlowski wrote:
> On 15/09/2022 21:40, Mikhail Rudenko wrote:
> >
> > Hi Krzysztof,
> >
> > On 2022-09-12 at 12:56 +02, Krzysztof Kozlowski <[email protected]> wrote:
> >> On 11/09/2022 22:01, Mikhail Rudenko wrote:
> >>> +static const struct i2c_device_id ov4689_id[] = {
> >>> + { "ov4689", 0 },
> >>> + {},
> >>> +};
> >>> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
> >>> +
> >>> +static const struct of_device_id ov4689_of_match[] = {
> >>> + { .compatible = "ovti,ov4689" },
> >>> + {},
> >>> +};
> >>> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
> >>> +
> >>> +static struct i2c_driver ov4689_i2c_driver = {
> >>> + .driver = {
> >>> + .name = "ov4689",
> >>> + .pm = &ov4689_pm_ops,
> >>> + .of_match_table = of_match_ptr(ov4689_of_match),
> >>
> >> of_match_ptr is usually paired with maybe_unused, otherwise you will
> >> have compile test warnings.
> >
> > I see. I think we could also use `#if IS_ENABLED(CONFIG_OF)` around
> > `ov4689_of_match` and the corresponding `MODULE_DEVICE_TABLE`. Is it
> > appropriate here?
>
> Would work, but ifdefs are not nice. Just use maybe_unused. Warnings
> should disappear.

Neither is needed. Just drop of_match_ptr().

Apart from clock handling, it would probably work on ACPI, too. No need to
address that though.

--
Regards,

Sakari Ailus

2022-09-19 06:45:50

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail,

On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
>
> Hi Dave,
>
> On 2022-09-14 at 10:58 +01, Dave Stevenson <[email protected]> wrote:
> > Hi Mikhail
> >
> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
> >>
> >> Hello,
> >>
> >> this series implements support for Omnivision OV4689 image
> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> >> connection.
> >>
> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
> >>
> >> While porting the driver, I stumbled upon two issues:
> >>
> >> (1) In the original driver, horizontal total size (HTS) was set to a
> >> value (2584) lower then the frame width (2688), resulting in negative
> >> hblank. In this driver, I increased HTS to 2688, but fps dropped from
> >> 29.88 to 28.73. What is the preferred way to handle this?
> >
> > This is one of the joys of sensors - they don't all work in the same way.
> >
> > I don't have an official datasheet for OV4689 from Omnivision, but
> > found one on the internet [1]. That should allow you to reverse the
> > PLL configuration to confirm that the pixel rate is the value you've
> > computed based on link frequency (they aren't necessarily related). Do
> > the frame rate calculations work using width + HBLANK, height +
> > VBLANK, and pixel rate?
> > The datasheet claims the sensor supports 2688x1520 @ 90 fps, so
> > something doesn't hold true between 4 data lanes at 500MHz/1Gbit/s per
> > lane when your default hts/vts is 2688x1554 and it only gives
> > 28.73fps.
>
> Seems like those 90 fps is about CSI throughput, not actual sensor
> performance. I've checked the datasheet and the register values, and it
> seems like the pixel clock is 126 Mhz in this configuration (the maximum
> is 150 MHz according to the datasheet). This corresponds to a
> theoretical fps of 30.16 at hts=2688 and vts=1554. At the same time the
> observed fps is 28.73. I'm not sure where those 1.43 frames are lost,
> hope to do more experimentation with VTS and HTS over the weekend.
>
> > I have seen modes in sensors where the HTS register is in units of 2
> > pixels, so what range of HTS (and VTS) values actually works on this
> > sensor? (I don't see it documented, but I'm not surprised).
> >
> > [1] https://cdn.hackaday.io/files/19354828041536/OV4689-OmniVision.pdf
> >
> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> >> gain is not linear across that range. Instead, it is piecewise linear
> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> >> with more linear segments in between. Rockchip's camera engine code
> >> chooses one of the above segments depenging on the desired gain
> >> value. The question is, how should we proceed keeping in mind
> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> >> libcamera will do the mapping, or the driver will do the mapping
> >> itself and expose some logical gain units not tied to the actual gain
> >> register value? Meanwhile, this driver conservatively exposes only
> >> 0x0-0xf8 gain register range.
> >
> > The datasheet linked above says "for the gain formula, please contact
> > your local OmniVision FAE" :-(
> > I would assume that the range is from 1x rather than 0x - people
> > rarely want a totally black image that 0x would give. Or is it ranges
> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
>
> A picture is worth a thousand words, so I've attached the results of my
> experimentation with the gain register. They were obtained with Rockchip
> 3399, with AEC, AGC and black level subtraction disabled. The image was
> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.

Based on that it looks like their medication may have been a little too
strong.

Could this be implemented so that the control value would be linear linear
but its range would correspond 1x--16x values?

libcamera will be able to cope with that.

>
> > Other sensors expose the full range of the register via
> > V4L2_CID_ANALOGUE_GAIN, and require userspace (mainly libcamera now)
> > to know how to convert a gain into the register value. If the gain
> > range goes up to x16, then exposing that would be useful. I'd advocate
> > just exposing the full range of 0x000 - 0x7ff, as then you can have
> > the accuracy of 256 values between x1 to x2, but also the full range.
>
> I also like this approach, although libcamera's CameraSensorHelper
> doesn't support piecewise-linear gain code mapping yet. Nevertheless,
> I believe exposing the full range is a good idea and will do so in v3.
>
> > I might see if I can pick up one of these sensors and see if I can get
> > it running on a Raspberry Pi. Thanks for trying to upstream this -
> > it's nice to have such a range of sensor drivers to choose from.
> >
> > Dave
> >
>
> Thanks for your elucidating tips!
>
> --
> Best regards,
> Mikhail Rudenko
>



--
Sakari Ailus

2022-09-19 07:13:26

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Tommaso,

On Fri, Sep 16, 2022 at 03:34:01PM +0200, Tommaso Merciai wrote:
> > >> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> > >> + if (ret < 0) {
> > >> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> > >> + return ret;
> > >> + }
> > >> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> > >> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> > >
> > >
> > > What do you think about?
> > > Thanks.
> >
> > Unfortunately, I have no experience with ACPI-based devices. :(
> >
> > Do you mean that in the case of an ACPI device and devm_clk_get_optional
> > returning NULL we should assume that the clock is already enabled and
> > will stay enabled during sensor operation? How should we distinguish it
> > from the case of an OF-based system and clock just missing from device
> > tree?
>
> Not exaclty :)
>
> I copy comment from [1]
>
> if you use ov5693->xvclk to identify the ACPI vs OF use case shouldn't
> you use the get_optionl() version ? Otherwise in the ACPI case you will have
> -ENOENT if there's not 'xvclk' property and bail out.
>
> Unless my understanding is wrong on ACPI we have "clock-frequency" and
> on OF "xvclk" with an "assigned-clock-rates",

Generally yes. It's also possible to have a clock in ACPI based system
although those clocks do not come from ACPI. See e.g.
drivers/platform/x86/intel/int3472/clk_and_regulator.c .

--
Sakari Ailus

2022-09-19 07:39:03

by Tommaso Merciai

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,

On Fri, Sep 16, 2022 at 04:44:31PM +0300, Mikhail Rudenko wrote:
>
> On 2022-09-16 at 15:34 +02, Tommaso Merciai <[email protected]> wrote:
> > Hi Mikhail,
> >
> > On Thu, Sep 15, 2022 at 11:50:23PM +0300, Mikhail Rudenko wrote:
> >>
> >> Hi Tommaso,
> >>
> >> On 2022-09-14 at 17:51 +02, Tommaso Merciai <[email protected]> wrote:
> >> > Hi Mikhail,
> >> > I do a first round on reviewing your driver :)
> >> >
> >> > On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
>
> <snip>
>
> >> >> +
> >> >> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
> >> >> + if (IS_ERR(ov4689->xvclk)) {
> >> >> + dev_err(dev, "Failed to get xvclk\n");
> >> >> + return -EINVAL;
> >> >> + }
> >> >
> >> > ^ I think is better to use devm_clk_get_optional instead of clck_get.
> >> > clck_get can fail in CPU's that use ACPI
> >> >
> >> >> +
> >> >> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> >> >> + if (ret < 0) {
> >> >> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> >> >> + return ret;
> >> >> + }
> >> >> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> >> >> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> >> >
> >> >
> >> > What do you think about?
> >> > Thanks.
> >>
> >> Unfortunately, I have no experience with ACPI-based devices. :(
> >>
> >> Do you mean that in the case of an ACPI device and devm_clk_get_optional
> >> returning NULL we should assume that the clock is already enabled and
> >> will stay enabled during sensor operation? How should we distinguish it
> >> from the case of an OF-based system and clock just missing from device
> >> tree?
> >
> > Not exaclty :)
> >
> > I copy comment from [1]
> >
> > if you use ov5693->xvclk to identify the ACPI vs OF use case shouldn't
> > you use the get_optionl() version ? Otherwise in the ACPI case you will have
> > -ENOENT if there's not 'xvclk' property and bail out.
> >
> > Unless my understanding is wrong on ACPI we have "clock-frequency" and
> > on OF "xvclk" with an "assigned-clock-rates",
> >
> > [1] https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/
> >
> > Let me know if you need more details.
>
> Thanks for the pointer! I'll try to implement something along the lines
> of your ov5693 series.
>
> But I'm not sure that will be enough to support ACPI systems
> correctly. What about lanes number and link frequency checks? Should
> they be made conditional on CONFIG_OF? Anything else I don't know?

In my opinion, lanes number and link frequency checks are ok :)
We don't need conditional CONFIG_OF.

fwnode* function support both ACPI and dts.

Thanks,
Tommaso

>
> >
> > Regards,
> > Tommaso
> >
> --
> Best regards,
> Mikhail Rudenko

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
http://www.amarulasolutions.com

2022-09-19 07:45:08

by Tommaso Merciai

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Sakari,

On Mon, Sep 19, 2022 at 06:33:12AM +0000, Sakari Ailus wrote:
> Hi Tommaso,
>
> On Fri, Sep 16, 2022 at 03:34:01PM +0200, Tommaso Merciai wrote:
> > > >> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> > > >> + if (ret < 0) {
> > > >> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> > > >> + return ret;
> > > >> + }
> > > >> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> > > >> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> > > >
> > > >
> > > > What do you think about?
> > > > Thanks.
> > >
> > > Unfortunately, I have no experience with ACPI-based devices. :(
> > >
> > > Do you mean that in the case of an ACPI device and devm_clk_get_optional
> > > returning NULL we should assume that the clock is already enabled and
> > > will stay enabled during sensor operation? How should we distinguish it
> > > from the case of an OF-based system and clock just missing from device
> > > tree?
> >
> > Not exaclty :)
> >
> > I copy comment from [1]
> >
> > if you use ov5693->xvclk to identify the ACPI vs OF use case shouldn't
> > you use the get_optionl() version ? Otherwise in the ACPI case you will have
> > -ENOENT if there's not 'xvclk' property and bail out.
> >
> > Unless my understanding is wrong on ACPI we have "clock-frequency" and
> > on OF "xvclk" with an "assigned-clock-rates",
>
> Generally yes. It's also possible to have a clock in ACPI based system
> although those clocks do not come from ACPI. See e.g.
> drivers/platform/x86/intel/int3472/clk_and_regulator.c .

I save this :)
Thanks for sharing.

Regards,
Tommaso

>
> --
> Sakari Ailus

--
Tommaso Merciai
Embedded Linux Engineer
[email protected]
__________________________________

Amarula Solutions SRL
Via Le Canevare 30, 31100 Treviso, Veneto, IT
T. +39 042 243 5310
[email protected]
http://www.amarulasolutions.com

2022-09-19 08:25:36

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver


Hi Sakari,

On 2022-09-19 at 06:40 GMT, Sakari Ailus <[email protected]> wrote:

> Hi Mikhail,
>
> On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
>>
>> Hi Dave,
>>
>> On 2022-09-14 at 10:58 +01, Dave Stevenson <[email protected]> wrote:
>> > Hi Mikhail
>> >
>> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
>> >>
>> >> Hello,
>> >>
>> >> this series implements support for Omnivision OV4689 image
>> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
>> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
>> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
>> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
>> >> connection.
>> >>
>> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
>> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
>> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera
>> >> module.
>> >> While porting the driver, I stumbled upon two issues:
[snip]
>> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
>> >> gain is not linear across that range. Instead, it is piecewise linear
>> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
>> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
>> >> with more linear segments in between. Rockchip's camera engine code
>> >> chooses one of the above segments depenging on the desired gain
>> >> value. The question is, how should we proceed keeping in mind
>> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
>> >> libcamera will do the mapping, or the driver will do the mapping
>> >> itself and expose some logical gain units not tied to the actual gain
>> >> register value? Meanwhile, this driver conservatively exposes only
>> >> 0x0-0xf8 gain register range.
>> >
>> > The datasheet linked above says "for the gain formula, please contact
>> > your local OmniVision FAE" :-(
>> > I would assume that the range is from 1x rather than 0x - people
>> > rarely want a totally black image that 0x would give. Or is it ranges
>> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
>>
>> A picture is worth a thousand words, so I've attached the results of my
>> experimentation with the gain register. They were obtained with Rockchip
>> 3399, with AEC, AGC and black level subtraction disabled. The image was
>> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.
>
> Based on that it looks like their medication may have been a little too
> strong.
>
> Could this be implemented so that the control value would be linear linear
> but its range would correspond 1x--16x values?
>
> libcamera will be able to cope with that.
>

According to the following fragment of the Rockchip camera engine sensor
configuration file for ov4689 [1]

<Linear index="1" type="double" size="[4 7]">
[1 2 128 0 1 128 255
2 4 64 -248 1 376 504
4 8 32 -756 1 884 1012
8 16 16 -1784 1 1912 2040]
</Linear>,

it uses gain register value range 128-255 for gain 1x-2x, 376-504 for
gain 2x-4x, 884-1024 for 4x-8x, and 1912-2040 for 8x-16x. Do you suggest
to implement this calculation in the sensor driver and expose some
linear "logical" gain to userspace (ranging, e.g., 128-2048 for gains
1x-16x)?


[1] https://github.com/aosp-rockchip/android_external_camera_engine_rkaiq/blob/quartz64/iqfiles/ov4689_JSD3425-C1_JSD3425-C1-36IRC-4M-F20.xml

--
Best regards,
Mikhail

2022-09-19 11:07:46

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail,

On Mon, Sep 19, 2022 at 10:01:06AM +0300, Mikhail Rudenko wrote:
>
> Hi Sakari,
>
> On 2022-09-19 at 06:40 GMT, Sakari Ailus <[email protected]> wrote:
>
> > Hi Mikhail,
> >
> > On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
> >>
> >> Hi Dave,
> >>
> >> On 2022-09-14 at 10:58 +01, Dave Stevenson <[email protected]> wrote:
> >> > Hi Mikhail
> >> >
> >> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
> >> >>
> >> >> Hello,
> >> >>
> >> >> this series implements support for Omnivision OV4689 image
> >> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> >> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
> >> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> >> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> >> >> connection.
> >> >>
> >> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> >> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> >> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera
> >> >> module.
> >> >> While porting the driver, I stumbled upon two issues:
> [snip]
> >> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> >> >> gain is not linear across that range. Instead, it is piecewise linear
> >> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> >> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> >> >> with more linear segments in between. Rockchip's camera engine code
> >> >> chooses one of the above segments depenging on the desired gain
> >> >> value. The question is, how should we proceed keeping in mind
> >> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> >> >> libcamera will do the mapping, or the driver will do the mapping
> >> >> itself and expose some logical gain units not tied to the actual gain
> >> >> register value? Meanwhile, this driver conservatively exposes only
> >> >> 0x0-0xf8 gain register range.
> >> >
> >> > The datasheet linked above says "for the gain formula, please contact
> >> > your local OmniVision FAE" :-(
> >> > I would assume that the range is from 1x rather than 0x - people
> >> > rarely want a totally black image that 0x would give. Or is it ranges
> >> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
> >>
> >> A picture is worth a thousand words, so I've attached the results of my
> >> experimentation with the gain register. They were obtained with Rockchip
> >> 3399, with AEC, AGC and black level subtraction disabled. The image was
> >> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.
> >
> > Based on that it looks like their medication may have been a little too
> > strong.
> >
> > Could this be implemented so that the control value would be linear linear
> > but its range would correspond 1x--16x values?
> >
> > libcamera will be able to cope with that.
> >
>
> According to the following fragment of the Rockchip camera engine sensor
> configuration file for ov4689 [1]
>
> <Linear index="1" type="double" size="[4 7]">
> [1 2 128 0 1 128 255
> 2 4 64 -248 1 376 504
> 4 8 32 -756 1 884 1012
> 8 16 16 -1784 1 1912 2040]
> </Linear>,
>
> it uses gain register value range 128-255 for gain 1x-2x, 376-504 for
> gain 2x-4x, 884-1024 for 4x-8x, and 1912-2040 for 8x-16x. Do you suggest
> to implement this calculation in the sensor driver and expose some
> linear "logical" gain to userspace (ranging, e.g., 128-2048 for gains
> 1x-16x)?

Yes. This way the user space can somehow work without knowing this special
implementation, even though the granularity changes over the range. I guess
the granularity would need to be known in libcamera but that's a separate
issue.

--
Sakari Ailus

2022-09-19 14:06:04

by Laurent Pinchart

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hello,

On Mon, Sep 19, 2022 at 10:31:02AM +0000, Sakari Ailus wrote:
> On Mon, Sep 19, 2022 at 10:01:06AM +0300, Mikhail Rudenko wrote:
> > On 2022-09-19 at 06:40 GMT, Sakari Ailus wrote:
> > > On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
> > >> On 2022-09-14 at 10:58 +01, Dave Stevenson wrote:
> > >> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko wrote:
> > >> >>
> > >> >> Hello,
> > >> >>
> > >> >> this series implements support for Omnivision OV4689 image
> > >> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> > >> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
> > >> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> > >> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> > >> >> connection.
> > >> >>
> > >> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> > >> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> > >> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera
> > >> >> module.
> > >> >> While porting the driver, I stumbled upon two issues:
> >
> > [snip]
> >
> > >> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> > >> >> gain is not linear across that range. Instead, it is piecewise linear
> > >> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> > >> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> > >> >> with more linear segments in between. Rockchip's camera engine code
> > >> >> chooses one of the above segments depenging on the desired gain
> > >> >> value. The question is, how should we proceed keeping in mind
> > >> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> > >> >> libcamera will do the mapping, or the driver will do the mapping
> > >> >> itself and expose some logical gain units not tied to the actual gain
> > >> >> register value? Meanwhile, this driver conservatively exposes only
> > >> >> 0x0-0xf8 gain register range.
> > >> >
> > >> > The datasheet linked above says "for the gain formula, please contact
> > >> > your local OmniVision FAE" :-(
> > >> > I would assume that the range is from 1x rather than 0x - people
> > >> > rarely want a totally black image that 0x would give. Or is it ranges
> > >> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
> > >>
> > >> A picture is worth a thousand words, so I've attached the results of my
> > >> experimentation with the gain register. They were obtained with Rockchip
> > >> 3399, with AEC, AGC and black level subtraction disabled. The image was
> > >> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.

Is that full or limited range YUV ?

> > > Based on that it looks like their medication may have been a little too
> > > strong.
> > >
> > > Could this be implemented so that the control value would be linear linear
> > > but its range would correspond 1x--16x values?
> > >
> > > libcamera will be able to cope with that.
> >
> > According to the following fragment of the Rockchip camera engine sensor
> > configuration file for ov4689 [1]
> >
> > <Linear index="1" type="double" size="[4 7]">
> > [1 2 128 0 1 128 255
> > 2 4 64 -248 1 376 504
> > 4 8 32 -756 1 884 1012
> > 8 16 16 -1784 1 1912 2040]
> > </Linear>,
> >
> > it uses gain register value range 128-255 for gain 1x-2x, 376-504 for
> > gain 2x-4x, 884-1024 for 4x-8x, and 1912-2040 for 8x-16x. Do you suggest

That looks *really* weird. I would have understood [384, 511], [896,
1023] and [1920, 2047], but not those intervals.

The driver hardcodes bit 0x3503[2] to 1, which means "sensor gain
format". Maybe setting it to 0 ("real gain format") would produce saner
results ?

> > to implement this calculation in the sensor driver and expose some
> > linear "logical" gain to userspace (ranging, e.g., 128-2048 for gains
> > 1x-16x)?
>
> Yes. This way the user space can somehow work without knowing this special
> implementation, even though the granularity changes over the range. I guess
> the granularity would need to be known in libcamera but that's a separate
> issue.

I can live with that.

--
Regards,

Laurent Pinchart

2022-09-20 16:52:57

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

16 21.69
17 21.76
18 21.81
19 21.65
20 21.94
21 21.99
22 22.05
23 22.04
24 21.96
25 22.20
26 22.33
27 22.41
28 22.39
29 22.55
30 22.38
31 22.62
32 22.77
33 22.80
34 22.87
35 23.01
36 23.08
37 23.09
38 23.01
39 23.19
40 23.41
41 23.27
42 23.36
43 23.43
44 23.52
45 23.61
46 23.70
47 23.78
48 23.87
49 23.95
50 24.03
51 24.27
52 24.21
53 24.31
54 24.40
55 24.48
56 24.58
57 24.68
58 24.77
59 24.87
60 24.95
61 25.05
62 25.13
63 25.22
64 25.33
65 25.44
66 25.53
67 25.63
68 25.72
69 25.82
70 25.92
71 26.02
72 26.12
73 26.22
74 26.32
75 26.41
76 26.51
77 26.62
78 26.71
79 26.80
80 26.91
81 27.01
82 27.11
83 27.21
84 27.30
85 27.41
86 27.50
87 27.60
88 27.71
89 27.81
90 27.91
91 28.01
92 28.11
93 28.21
94 28.30
95 28.39
96 28.49
97 28.59
98 28.69
99 28.79
100 28.89
101 28.99
102 29.09
103 29.18
104 29.29
105 29.40
106 29.49
107 29.59
108 29.68
109 29.78
110 29.87
111 29.97
112 30.07
113 30.17
114 30.27
115 30.37
116 30.47
117 30.57
118 30.66
119 30.75
120 30.86
121 30.96
122 31.06
123 31.15
124 31.25
125 31.35
126 31.45
127 31.55
128 31.65
129 31.75
130 31.85
131 31.94
132 32.04
133 32.14
134 32.23
135 32.33
136 32.43
137 32.54
138 32.64
139 32.73
140 32.82
141 32.92
142 33.15
143 33.11
144 33.21
145 33.31
146 33.41
147 33.51
148 33.61
149 33.71
150 33.87
151 33.89
152 33.99
153 34.10
154 34.20
155 34.42
156 34.38
157 34.48
158 34.58
159 34.67
160 34.99
161 34.88
162 34.97
163 35.20
164 35.16
165 35.40
166 35.50
167 35.59
168 35.70
169 35.67
170 35.99
171 35.93
172 36.09
173 36.19
174 36.29
175 36.46
176 36.49
177 36.59
178 36.69
179 36.79
180 36.89
181 37.07
182 36.96
183 37.26
184 37.28
185 37.47
186 37.49
187 37.58
188 37.76
189 37.65
190 37.75
191 38.06
192 38.09
193 38.13
194 38.16
195 38.26
196 38.42
197 38.68
198 38.57
199 38.73
200 38.77
201 38.87
202 39.10
203 39.20
204 39.17
205 39.41
206 39.51
207 39.47
208 39.58
209 39.68
210 39.78
211 39.88
212 39.98
213 40.09
214 40.19
215 40.29
216 40.39
217 40.50
218 40.60
219 40.70
220 40.89
221 40.90
222 41.00
223 41.10
224 41.20
225 41.31
226 41.41
227 41.51
228 41.61
229 41.71
230 41.81
231 41.91
232 42.01
233 42.12
234 42.22
235 42.32
236 42.42
237 42.52
238 42.62
239 42.72
240 42.82
241 42.92
242 43.03
243 43.12
244 43.22
245 43.24
246 43.42
247 43.52
248 43.62
249 43.72
250 43.82
251 43.83
252 44.01
253 44.11
254 44.21
255 44.22
256 22.33
257 22.40
258 22.47
259 22.55
260 22.62
261 22.71
262 22.79
263 22.87
264 22.99
265 23.09
266 23.19
267 23.28
268 23.38
269 23.50
270 23.61
271 23.72
272 23.87
273 23.99
274 24.12
275 24.25
276 24.36
277 24.53
278 24.67
279 24.82
280 24.98
281 25.13
282 25.28
283 25.43
284 25.59
285 25.76
286 25.92
287 26.08
288 26.26
289 26.44
290 26.61
291 26.78
292 26.95
293 27.14
294 27.31
295 27.48
296 27.68
297 27.87
298 28.05
299 28.22
300 28.39
301 28.58
302 28.76
303 28.94
304 29.14
305 29.34
306 29.53
307 29.71
308 29.90
309 30.10
310 30.29
311 30.48
312 30.68
313 30.88
314 31.08
315 31.27
316 31.46
317 31.67
318 31.80
319 32.05
320 32.29
321 32.49
322 32.69
323 32.89
324 33.08
325 33.29
326 33.49
327 33.69
328 33.91
329 34.12
330 34.29
331 34.46
332 34.70
333 34.84
334 35.11
335 35.23
336 35.45
337 35.74
338 35.87
339 36.08
340 36.14
341 36.57
342 36.63
343 36.89
344 37.10
345 37.31
346 37.51
347 37.71
348 37.78
349 38.05
350 38.19
351 38.39
352 38.60
353 38.82
354 39.03
355 39.22
356 39.42
357 39.64
358 39.84
359 40.03
360 40.25
361 40.47
362 40.67
363 40.87
364 41.07
365 41.28
366 41.49
367 41.68
368 41.88
369 42.10
370 42.31
371 42.50
372 42.70
373 42.92
374 43.13
375 43.33
376 43.53
377 43.74
378 43.95
379 44.14
380 44.34
381 44.56
382 44.75
383 44.94
384 45.18
385 45.39
386 45.60
387 45.80
388 46.00
389 46.21
390 46.41
391 46.60
392 46.81
393 47.02
394 47.22
395 47.42
396 47.62
397 47.83
398 48.03
399 48.23
400 48.44
401 48.66
402 48.86
403 49.06
404 49.25
405 49.46
406 49.57
407 49.77
408 49.98
409 50.28
410 50.39
411 50.68
412 50.80
413 51.08
414 51.28
415 51.46
416 51.60
417 51.90
418 51.88
419 52.21
420 52.50
421 52.62
422 52.90
423 53.09
424 53.31
425 53.52
426 53.72
427 53.92
428 54.11
429 54.32
430 54.52
431 54.71
432 54.86
433 55.14
434 55.33
435 55.53
436 55.73
437 55.84
438 56.04
439 56.32
440 56.53
441 56.74
442 56.95
443 57.14
444 57.33
445 57.54
446 57.74
447 57.93
448 58.15
449 58.36
450 58.70
451 58.77
452 59.09
453 59.17
454 59.37
455 59.57
456 59.78
457 60.00
458 60.20
459 60.39
460 60.59
461 60.81
462 61.00
463 61.19
464 61.42
465 61.63
466 61.84
467 62.16
468 62.23
469 62.44
470 62.64
471 62.84
472 63.05
473 63.26
474 63.47
475 63.66
476 63.86
477 64.07
478 64.27
479 64.38
480 64.59
481 64.89
482 65.10
483 65.29
484 65.49
485 65.61
486 65.81
487 66.00
488 66.22
489 66.45
490 66.64
491 66.84
492 67.04
493 67.25
494 67.46
495 67.59
496 67.88
497 68.08
498 68.36
499 68.47
500 68.67
501 68.88
502 68.96
503 69.28
504 69.49
505 69.70
506 69.99
507 70.10
508 70.23
509 70.44
510 70.57
511 70.78
512 22.11
513 22.17
514 22.22
515 22.27
516 22.33
517 22.40
518 22.46
519 22.53
520 22.60
521 22.67
522 22.73
523 22.81
524 22.87
525 22.94
526 23.01
527 23.08
528 23.17
529 23.25
530 23.33
531 23.41
532 23.49
533 23.58
534 23.66
535 23.73
536 23.83
537 23.92
538 24.01
539 24.10
540 24.19
541 24.28
542 24.37
543 24.46
544 24.58
545 24.69
546 24.78
547 24.89
548 24.99
549 25.10
550 25.20
551 25.30
552 25.42
553 25.53
554 25.63
555 25.73
556 25.84
557 25.96
558 26.06
559 26.16
560 26.28
561 26.40
562 26.51
563 26.61
564 26.72
565 26.83
566 26.94
567 27.05
568 27.17
569 27.28
570 27.40
571 27.51
572 27.63
573 27.75
574 27.87
575 27.98
576 28.12
577 28.25
578 28.36
579 28.48
580 28.59
581 28.72
582 28.83
583 28.95
584 29.09
585 29.21
586 29.34
587 29.46
588 29.58
589 29.71
590 29.83
591 29.94
592 30.08
593 30.21
594 30.33
595 30.46
596 30.58
597 30.71
598 30.83
599 30.95
600 31.09
601 31.22
602 31.34
603 31.46
604 31.59
605 31.72
606 31.84
607 31.96
608 32.07
609 32.23
610 32.36
611 32.41
612 32.61
613 32.74
614 32.87
615 33.00
616 33.08
617 33.28
618 33.32
619 33.53
620 33.65
621 33.71
622 33.91
623 34.04
624 34.17
625 34.31
626 34.44
627 34.57
628 34.69
629 34.82
630 34.95
631 35.00
632 35.21
633 35.35
634 35.40
635 35.39
636 35.73
637 35.87
638 36.00
639 36.04
640 36.28
641 36.42
642 36.55
643 36.60
644 36.81
645 36.95
646 37.00
647 37.21
648 37.35
649 37.49
650 37.62
651 37.75
652 37.88
653 37.93
654 38.06
655 38.19
656 38.33
657 38.56
658 38.61
659 38.73
660 38.86
661 39.08
662 39.00
663 39.26
664 39.39
665 39.40
666 39.67
667 39.66
668 39.92
669 40.03
670 40.06
671 40.19
672 40.47
673 40.48
674 40.61
675 40.96
676 40.88
677 41.02
678 41.15
679 41.27
680 41.44
681 41.56
682 41.69
683 41.82
684 41.95
685 42.09
686 42.35
687 42.35
688 42.49
689 42.69
690 42.76
691 42.89
692 43.02
693 43.16
694 43.29
695 43.42
696 43.56
697 43.70
698 43.84
699 43.97
700 44.10
701 44.24
702 44.37
703 44.50
704 44.65
705 44.79
706 44.92
707 45.04
708 45.17
709 45.31
710 45.44
711 45.57
712 45.72
713 45.85
714 45.91
715 46.12
716 46.25
717 46.39
718 46.52
719 46.65
720 46.80
721 46.93
722 47.07
723 47.20
724 47.33
725 47.47
726 47.61
727 47.65
728 47.87
729 48.02
730 48.06
731 48.28
732 48.33
733 48.54
734 48.60
735 48.73
736 48.88
737 49.00
738 49.14
739 49.26
740 49.39
741 49.62
742 49.66
743 49.79
744 49.94
745 50.16
746 50.21
747 50.43
748 50.55
749 50.60
750 50.80
751 50.87
752 51.00
753 51.15
754 51.29
755 51.50
756 51.55
757 51.70
758 51.82
759 51.89
760 51.96
761 52.23
762 52.45
763 52.50
764 52.62
765 52.63
766 52.77
767 52.95
768 24.08
769 24.23
770 24.37
771 24.52
772 24.69
773 24.87
774 25.06
775 25.26
776 25.51
777 25.76
778 25.99
779 26.20
780 26.40
781 26.70
782 27.04
783 27.33
784 27.66
785 27.91
786 28.30
787 28.62
788 28.94
789 29.25
790 29.57
791 29.99
792 30.36
793 30.68
794 31.11
795 31.42
796 31.87
797 32.19
798 32.58
799 32.95
800 33.44
801 33.86
802 34.19
803 34.45
804 34.85
805 35.42
806 35.69
807 36.09
808 36.53
809 36.96
810 37.50
811 37.77
812 38.17
813 38.74
814 39.01
815 39.42
816 39.97
817 40.28
818 40.71
819 41.12
820 41.53
821 41.97
822 42.39
823 42.93
824 43.37
825 43.68
826 44.11
827 44.52
828 44.92
829 45.37
830 45.79
831 46.20
832 46.81
833 47.14
834 47.57
835 47.99
836 48.40
837 48.98
838 49.28
839 49.68
840 50.14
841 50.59
842 51.02
843 51.35
844 51.86
845 52.30
846 52.64
847 53.14
848 53.50
849 53.95
850 54.39
851 54.80
852 55.30
853 55.67
854 56.17
855 56.58
856 57.02
857 57.38
858 57.81
859 58.22
860 58.51
861 59.08
862 59.38
863 59.91
864 60.22
865 60.80
866 61.30
867 61.52
868 61.93
869 62.50
870 62.93
871 63.21
872 63.79
873 64.11
874 64.60
875 64.95
876 65.42
877 65.80
878 66.34
879 66.62
880 67.04
881 67.49
882 67.92
883 68.33
884 68.75
885 69.19
886 69.74
887 70.15
888 70.46
889 70.90
890 71.32
891 71.73
892 72.13
893 72.58
894 73.00
895 73.31
896 73.88
897 74.38
898 74.74
899 75.14
900 75.46
901 75.97
902 76.38
903 76.77
904 77.20
905 77.54
906 77.94
907 78.32
908 78.77
909 79.21
910 79.60
911 79.90
912 80.40
913 80.73
914 81.21
915 81.57
916 81.98
917 82.44
918 82.76
919 83.13
920 83.64
921 83.83
922 84.30
923 84.64
924 85.03
925 85.43
926 85.80
927 86.17
928 86.56
929 86.95
930 87.31
931 87.59
932 87.95
933 88.42
934 88.90
935 89.10
936 89.42
937 89.84
938 90.17
939 90.60
940 90.95
941 91.24
942 91.67
943 92.09
944 92.30
945 92.75
946 93.11
947 93.45
948 93.71
949 94.07
950 94.41
951 94.75
952 95.10
953 95.47
954 95.81
955 96.14
956 96.47
957 96.70
958 97.16
959 97.48
960 97.84
961 98.20
962 98.54
963 98.66
964 99.18
965 99.54
966 99.86
967 100.18
968 100.53
969 100.76
970 101.20
971 101.46
972 101.82
973 102.04
974 102.35
975 102.66
976 102.99
977 103.31
978 103.57
979 103.88
980 104.26
981 104.59
982 104.87
983 105.25
984 105.48
985 105.76
986 106.12
987 106.34
988 106.65
989 107.03
990 107.31
991 107.59
992 107.84
993 108.15
994 108.44
995 108.73
996 109.02
997 109.33
998 109.62
999 109.89
1000 110.19
1001 110.49
1002 110.77
1003 111.05
1004 111.32
1005 111.61
1006 111.88
1007 111.95
1008 112.42
1009 112.73
1010 112.99
1011 113.25
1012 113.52
1013 113.79
1014 114.07
1015 114.31
1016 114.58
1017 114.86
1018 115.11
1019 115.37
1020 115.55
1021 115.77
1022 116.14
1023 116.38
1024 29.68
1025 29.90
1026 29.92
1027 29.94
1028 30.03
1029 30.34
1030 30.21
1031 30.30
1032 30.55
1033 30.59
1034 30.70
1035 30.74
1036 30.85
1037 30.97
1038 31.09
1039 31.20
1040 31.18
1041 31.12
1042 31.09
1043 30.79
1044 30.30
1045 29.58
1046 28.91
1047 28.09
1048 27.34
1049 26.65
1050 26.36
1051 26.28
1052 26.42
1053 26.57
1054 26.71
1055 26.85
1056 27.03
1057 27.18
1058 27.33
1059 27.40
1060 27.62
1061 27.78
1062 27.92
1063 28.07
1064 28.24
1065 28.40
1066 28.55
1067 28.69
1068 28.85
1069 28.94
1070 29.09
1071 29.33
1072 29.50
1073 29.68
1074 29.85
1075 30.02
1076 30.15
1077 30.33
1078 30.52
1079 30.68
1080 30.86
1081 31.04
1082 31.22
1083 31.39
1084 31.56
1085 31.67
1086 31.92
1087 32.09
1088 32.30
1089 32.49
1090 32.67
1091 32.84
1092 33.02
1093 33.21
1094 33.39
1095 33.49
1096 33.76
1097 33.75
1098 34.14
1099 34.32
1100 34.47
1101 34.69
1102 34.86
1103 34.89
1104 35.17
1105 35.45
1106 35.63
1107 35.81
1108 35.97
1109 36.20
1110 36.38
1111 36.56
1112 36.76
1113 36.88
1114 37.06
1115 37.33
1116 37.31
1117 37.63
1118 37.81
1119 37.87
1120 38.20
1121 38.40
1122 38.46
1123 38.77
1124 39.04
1125 39.16
1126 39.34
1127 39.39
1128 39.73
1129 39.80
1130 40.20
1131 40.31
1132 40.50
1133 40.62
1134 40.88
1135 40.94
1136 41.14
1137 41.34
1138 41.54
1139 41.73
1140 41.92
1141 42.12
1142 42.31
1143 42.50
1144 42.83
1145 42.90
1146 43.23
1147 43.28
1148 43.54
1149 43.67
1150 44.00
1151 44.05
1152 44.40
1153 44.48
1154 44.67
1155 44.81
1156 45.05
1157 45.26
1158 45.45
1159 45.63
1160 45.84
1161 46.04
1162 46.23
1163 46.43
1164 46.62
1165 46.82
1166 46.93
1167 47.20
1168 47.40
1169 47.53
1170 47.81
1171 48.00
1172 48.19
1173 48.31
1174 48.58
1175 48.77
1176 48.95
1177 49.17
1178 49.37
1179 49.47
1180 49.75
1181 49.86
1182 50.13
1183 50.24
1184 50.45
1185 50.75
1186 50.86
1187 51.05
1188 51.33
1189 51.45
1190 51.65
1191 51.92
1192 52.05
1193 52.13
1194 52.45
1195 52.64
1196 52.70
1197 53.04
1198 53.22
1199 53.41
1200 53.62
1201 53.70
1202 54.02
1203 54.21
1204 54.40
1205 54.49
1206 54.68
1207 54.99
1208 55.20
1209 55.40
1210 55.60
1211 55.66
1212 55.98
1213 56.05
1214 56.25
1215 56.58
1216 56.66
1217 56.87
1218 57.06
1219 57.26
1220 57.57
1221 57.65
1222 57.85
1223 58.04
1224 58.31
1225 58.45
1226 58.65
1227 58.97
1228 59.03
1229 59.24
1230 59.62
1231 59.61
1232 59.82
1233 60.04
1234 60.23
1235 60.33
1236 60.53
1237 60.95
1238 61.01
1239 61.21
1240 61.47
1241 61.61
1242 61.81
1243 62.13
1244 62.18
1245 62.40
1246 62.57
1247 62.77
1248 62.89
1249 63.19
1250 63.29
1251 63.57
1252 63.76
1253 63.97
1254 64.13
1255 64.26
1256 64.55
1257 64.76
1258 64.88
1259 65.15
1260 65.34
1261 65.55
1262 65.74
1263 65.92
1264 66.13
1265 66.24
1266 66.44
1267 66.64
1268 66.82
1269 67.02
1270 67.21
1271 67.50
1272 67.61
1273 67.81
1274 68.01
1275 68.20
1276 68.38
1277 68.58
1278 68.77
1279 68.97
1280 25.64
1281 26.00
1282 26.08
1283 26.40
1284 26.37
1285 26.58
1286 26.98
1287 27.17
1288 27.30
1289 27.53
1290 27.72
1291 28.10
1292 28.33
1293 28.53
1294 28.62
1295 28.87
1296 29.18
1297 29.26
1298 29.52
1299 29.31
1300 29.45
1301 29.69
1302 29.97
1303 30.26
1304 30.52
1305 30.93
1306 31.21
1307 31.54
1308 31.86
1309 32.20
1310 32.51
1311 32.84
1312 33.01
1313 33.59
1314 33.86
1315 34.20
1316 34.41
1317 35.01
1318 35.27
1319 35.70
1320 36.10
1321 36.40
1322 36.84
1323 36.98
1324 37.34
1325 37.86
1326 38.30
1327 38.59
1328 38.98
1329 39.39
1330 39.77
1331 40.07
1332 40.52
1333 40.79
1334 41.17
1335 41.67
1336 41.93
1337 42.48
1338 42.93
1339 43.11
1340 43.49
1341 44.03
1342 44.34
1343 44.66
1344 45.24
1345 45.53
1346 45.93
1347 46.30
1348 46.82
1349 47.11
1350 47.49
1351 47.87
1352 48.30
1353 48.71
1354 49.11
1355 49.50
1356 49.88
1357 50.31
1358 50.61
1359 51.08
1360 51.43
1361 51.84
1362 52.34
1363 52.64
1364 53.06
1365 53.46
1366 53.85
1367 54.25
1368 54.71
1369 55.16
1370 55.36
1371 55.74
1372 56.13
1373 56.69
1374 56.95
1375 57.46
1376 57.89
1377 58.18
1378 58.59
1379 58.97
1380 59.37
1381 59.92
1382 60.18
1383 60.57
1384 61.01
1385 61.42
1386 61.96
1387 62.23
1388 62.61
1389 62.95
1390 63.43
1391 63.72
1392 64.20
1393 64.66
1394 64.96
1395 65.35
1396 65.84
1397 66.25
1398 66.64
1399 67.04
1400 67.41
1401 67.77
1402 68.18
1403 68.56
1404 68.95
1405 69.38
1406 69.77
1407 70.24
1408 70.62
1409 71.02
1410 71.45
1411 71.83
1412 72.20
1413 72.71
1414 73.09
1415 73.40
1416 73.82
1417 74.12
1418 74.63
1419 75.01
1420 75.39
1421 75.80
1422 76.18
1423 76.54
1424 76.95
1425 77.36
1426 77.80
1427 78.11
1428 78.48
1429 78.81
1430 79.24
1431 79.61
1432 79.99
1433 80.38
1434 80.63
1435 81.20
1436 81.46
1437 81.86
1438 82.14
1439 82.55
1440 82.95
1441 83.32
1442 83.69
1443 84.04
1444 84.27
1445 84.64
1446 85.11
1447 85.34
1448 85.71
1449 86.21
1450 86.56
1451 86.89
1452 87.23
1453 87.48
1454 87.72
1455 88.27
1456 88.49
1457 88.98
1458 89.33
1459 89.53
1460 89.92
1461 90.22
1462 90.67
1463 90.94
1464 91.23
1465 91.57
1466 91.96
1467 92.34
1468 92.66
1469 93.00
1470 93.32
1471 93.43
1472 93.99
1473 94.34
1474 94.46
1475 94.78
1476 95.17
1477 95.59
1478 95.94
1479 96.05
1480 96.47
1481 96.80
1482 97.04
1483 97.35
1484 97.85
1485 98.16
1486 98.48
1487 98.67
1488 98.98
1489 99.37
1490 99.65
1491 100.06
1492 100.15
1493 100.57
1494 100.98
1495 101.07
1496 101.53
1497 101.87
1498 102.00
1499 102.41
1500 102.58
1501 103.09
1502 103.24
1503 103.47
1504 103.90
1505 104.24
1506 104.51
1507 104.67
1508 105.07
1509 105.45
1510 105.68
1511 105.88
1512 106.12
1513 106.42
1514 106.72
1515 107.05
1516 107.34
1517 107.66
1518 107.82
1519 108.29
1520 108.55
1521 108.74
1522 109.05
1523 109.27
1524 109.47
1525 109.80
1526 110.02
1527 110.35
1528 110.55
1529 110.91
1530 111.09
1531 111.37
1532 111.61
1533 111.88
1534 112.14
1535 112.46
1536 29.40
1537 29.60
1538 29.56
1539 29.74
1540 29.82
1541 30.05
1542 30.11
1543 30.34
1544 30.41
1545 30.46
1546 30.79
1547 30.82
1548 30.90
1549 31.19
1550 31.21
1551 31.38
1552 31.59
1553 31.58
1554 31.55
1555 31.16
1556 30.64
1557 29.83
1558 28.96
1559 28.17
1560 27.57
1561 27.71
1562 27.89
1563 28.08
1564 28.27
1565 28.48
1566 28.68
1567 28.86
1568 29.10
1569 29.31
1570 29.52
1571 29.73
1572 29.94
1573 30.17
1574 30.38
1575 30.59
1576 30.83
1577 31.06
1578 31.20
1579 31.42
1580 31.64
1581 31.96
1582 32.11
1583 32.40
1584 32.65
1585 32.90
1586 33.13
1587 33.36
1588 33.57
1589 33.85
1590 34.00
1591 34.29
1592 34.56
1593 34.74
1594 35.06
1595 35.30
1596 35.46
1597 35.80
1598 35.96
1599 36.27
1600 36.49
1601 36.83
1602 36.87
1603 37.24
1604 37.35
1605 37.61
1606 38.07
1607 38.23
1608 38.58
1609 38.81
1610 38.87
1611 39.33
1612 39.49
1613 39.77
1614 39.88
1615 40.26
1616 40.41
1617 40.75
1618 40.95
1619 41.20
1620 41.51
1621 41.72
1622 41.96
1623 42.35
1624 42.48
1625 42.75
1626 43.01
1627 43.26
1628 43.51
1629 43.95
1630 44.04
1631 44.29
1632 44.57
1633 44.84
1634 45.10
1635 45.36
1636 45.61
1637 45.88
1638 46.14
1639 46.39
1640 46.67
1641 46.94
1642 47.20
1643 47.45
1644 47.68
1645 47.98
1646 48.24
1647 48.49
1648 48.76
1649 49.04
1650 49.29
1651 49.46
1652 49.81
1653 49.99
1654 50.34
1655 50.60
1656 50.87
1657 51.14
1658 51.41
1659 51.58
1660 51.84
1661 52.05
1662 52.24
1663 52.62
1664 52.92
1665 53.20
1666 53.34
1667 53.73
1668 53.85
1669 54.26
1670 54.51
1671 54.64
1672 55.06
1673 55.33
1674 55.59
1675 55.86
1676 56.10
1677 56.25
1678 56.52
1679 56.77
1680 57.05
1681 57.34
1682 57.72
1683 57.98
1684 58.11
1685 58.52
1686 58.65
1687 59.03
1688 59.17
1689 59.45
1690 59.71
1691 59.97
1692 60.22
1693 60.51
1694 60.76
1695 61.02
1696 61.30
1697 61.58
1698 61.84
1699 62.10
1700 62.36
1701 62.55
1702 62.81
1703 63.06
1704 63.41
1705 63.71
1706 63.95
1707 64.14
1708 64.48
1709 64.76
1710 65.01
1711 65.27
1712 65.54
1713 65.73
1714 66.08
1715 66.31
1716 66.50
1717 66.78
1718 67.12
1719 67.37
1720 67.55
1721 67.83
1722 68.10
1723 68.35
1724 68.60
1725 68.88
1726 69.13
1727 69.39
1728 69.68
1729 70.04
1730 70.22
1731 70.48
1732 70.74
1733 71.01
1734 71.26
1735 71.52
1736 71.80
1737 72.07
1738 72.33
1739 72.59
1740 72.84
1741 72.99
1742 73.37
1743 73.62
1744 73.77
1745 74.05
1746 74.44
1747 74.68
1748 74.93
1749 75.07
1750 75.46
1751 75.63
1752 75.96
1753 76.02
1754 76.42
1755 76.73
1756 76.98
1757 77.23
1758 77.47
1759 77.60
1760 77.97
1761 78.11
1762 78.49
1763 78.72
1764 78.90
1765 79.24
1766 79.47
1767 79.70
1768 79.85
1769 80.23
1770 80.34
1771 80.65
1772 80.93
1773 81.06
1774 81.42
1775 81.65
1776 81.90
1777 82.15
1778 82.27
1779 82.62
1780 82.85
1781 82.98
1782 83.21
1783 83.51
1784 83.74
1785 83.99
1786 84.18
1787 84.39
1788 84.74
1789 85.00
1790 85.22
1791 85.32
1792 26.55
1793 26.82
1794 27.22
1795 27.51
1796 27.90
1797 28.20
1798 28.77
1799 29.22
1800 29.64
1801 30.20
1802 30.74
1803 31.43
1804 31.88
1805 32.65
1806 33.34
1807 33.89
1808 34.63
1809 35.43
1810 36.06
1811 36.63
1812 37.46
1813 38.23
1814 38.83
1815 39.69
1816 40.35
1817 41.15
1818 42.07
1819 42.71
1820 43.48
1821 44.31
1822 45.10
1823 46.00
1824 46.74
1825 47.61
1826 48.44
1827 49.24
1828 49.95
1829 50.91
1830 51.73
1831 52.46
1832 53.44
1833 54.32
1834 55.17
1835 55.98
1836 56.58
1837 57.58
1838 58.41
1839 59.23
1840 60.01
1841 60.96
1842 61.70
1843 62.51
1844 63.32
1845 64.22
1846 65.03
1847 65.85
1848 66.70
1849 67.59
1850 68.34
1851 69.16
1852 70.06
1853 70.94
1854 71.67
1855 72.57
1856 73.43
1857 74.40
1858 75.14
1859 75.93
1860 76.72
1861 77.57
1862 78.35
1863 79.00
1864 79.96
1865 80.78
1866 81.56
1867 82.31
1868 83.04
1869 83.83
1870 84.45
1871 85.30
1872 86.08
1873 86.73
1874 87.48
1875 88.30
1876 89.01
1877 89.64
1878 90.46
1879 91.14
1880 91.80
1881 92.59
1882 93.17
1883 93.87
1884 94.61
1885 95.33
1886 95.77
1887 96.62
1888 97.30
1889 98.01
1890 98.68
1891 99.25
1892 99.82
1893 100.51
1894 101.13
1895 101.73
1896 102.54
1897 103.15
1898 103.83
1899 104.30
1900 105.02
1901 105.65
1902 106.12
1903 106.63
1904 107.43
1905 108.04
1906 108.44
1907 109.00
1908 109.64
1909 110.30
1910 110.71
1911 111.25
1912 111.81
1913 112.39
1914 112.93
1915 113.46
1916 113.97
1917 114.52
1918 115.00
1919 115.52
1920 116.12
1921 116.65
1922 117.15
1923 117.63
1924 118.05
1925 118.63
1926 118.98
1927 119.50
1928 119.90
1929 120.51
1930 120.86
1931 121.42
1932 121.77
1933 122.33
1934 122.77
1935 123.09
1936 123.58
1937 124.03
1938 124.46
1939 124.89
1940 125.29
1941 125.71
1942 126.17
1943 126.54
1944 126.89
1945 127.41
1946 127.75
1947 128.01
1948 128.57
1949 128.93
1950 129.22
1951 129.69
1952 130.12
1953 130.45
1954 130.78
1955 131.11
1956 131.47
1957 131.86
1958 132.23
1959 132.58
1960 132.96
1961 133.34
1962 133.75
1963 134.05
1964 134.39
1965 134.87
1966 135.21
1967 135.31
1968 135.87
1969 136.15
1970 136.46
1971 136.79
1972 137.14
1973 137.47
1974 137.79
1975 138.12
1976 138.43
1977 138.77
1978 139.16
1979 139.34
1980 139.74
1981 140.04
1982 140.34
1983 140.71
1984 141.00
1985 141.30
1986 141.67
1987 141.93
1988 142.27
1989 142.47
1990 142.76
1991 143.06
1992 143.43
1993 143.61
1994 143.89
1995 144.20
1996 144.56
1997 144.78
1998 145.04
1999 145.25
2000 145.64
2001 145.93
2002 146.25
2003 146.41
2004 146.68
2005 146.96
2006 147.30
2007 147.46
2008 147.85
2009 148.09
2010 148.26
2011 148.51
2012 148.76
2013 149.06
2014 149.26
2015 149.54
2016 149.84
2017 150.10
2018 150.34
2019 150.58
2020 150.68
2021 151.07
2022 151.31
2023 151.47
2024 151.69
2025 151.89
2026 152.16
2027 152.34
2028 152.68
2029 152.81
2030 153.03
2031 153.26
2032 153.36
2033 153.72
2034 153.82
2035 154.14
2036 154.28
2037 154.52
2038 154.85
2039 154.88
2040 155.19


Attachments:
gain-test.3503_00.pdf (17.11 kB)
gain-test.3503_04.pdf (18.33 kB)
gain-test.3503_00.log (22.89 kB)
gain-test.3503_04.log (22.93 kB)
Download all attachments

2022-09-20 21:00:29

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver


Hi Laurent, Sakari,

On 2022-09-19 at 16:49 +03, Laurent Pinchart <[email protected]> wrote:

> Hello,
>
> On Mon, Sep 19, 2022 at 10:31:02AM +0000, Sakari Ailus wrote:
>> On Mon, Sep 19, 2022 at 10:01:06AM +0300, Mikhail Rudenko wrote:
>> > On 2022-09-19 at 06:40 GMT, Sakari Ailus wrote:
>> > > On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
>> > >> On 2022-09-14 at 10:58 +01, Dave Stevenson wrote:
>> > >> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko wrote:
>> > >> >>
>> > >> >> Hello,
>> > >> >>
>> > >> >> this series implements support for Omnivision OV4689 image
>> > >> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
>> > >> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
>> > >> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
>> > >> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
>> > >> >> connection.
>> > >> >>
>> > >> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
>> > >> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
>> > >> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera
>> > >> >> module.
>> > >> >> While porting the driver, I stumbled upon two issues:
>> >
>> > [snip]
>> >
>> > >> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
>> > >> >> gain is not linear across that range. Instead, it is piecewise linear
>> > >> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
>> > >> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
>> > >> >> with more linear segments in between. Rockchip's camera engine code
>> > >> >> chooses one of the above segments depenging on the desired gain
>> > >> >> value. The question is, how should we proceed keeping in mind
>> > >> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
>> > >> >> libcamera will do the mapping, or the driver will do the mapping
>> > >> >> itself and expose some logical gain units not tied to the actual gain
>> > >> >> register value? Meanwhile, this driver conservatively exposes only
>> > >> >> 0x0-0xf8 gain register range.
>> > >> >
>> > >> > The datasheet linked above says "for the gain formula, please contact
>> > >> > your local OmniVision FAE" :-(
>> > >> > I would assume that the range is from 1x rather than 0x - people
>> > >> > rarely want a totally black image that 0x would give. Or is it ranges
>> > >> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
>> > >>
>> > >> A picture is worth a thousand words, so I've attached the results of my
>> > >> experimentation with the gain register. They were obtained with Rockchip
>> > >> 3399, with AEC, AGC and black level subtraction disabled. The image was
>> > >> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.
>
> Is that full or limited range YUV ?
>
>> > > Based on that it looks like their medication may have been a little too
>> > > strong.
>> > >
>> > > Could this be implemented so that the control value would be linear linear
>> > > but its range would correspond 1x--16x values?
>> > >
>> > > libcamera will be able to cope with that.
>> >
>> > According to the following fragment of the Rockchip camera engine sensor
>> > configuration file for ov4689 [1]
>> >
>> > <Linear index="1" type="double" size="[4 7]">
>> > [1 2 128 0 1 128 255
>> > 2 4 64 -248 1 376 504
>> > 4 8 32 -756 1 884 1012
>> > 8 16 16 -1784 1 1912 2040]
>> > </Linear>,
>> >
>> > it uses gain register value range 128-255 for gain 1x-2x, 376-504 for
>> > gain 2x-4x, 884-1024 for 4x-8x, and 1912-2040 for 8x-16x. Do you suggest
>
> That looks *really* weird. I would have understood [384, 511], [896,
> 1023] and [1920, 2047], but not those intervals.
>
> The driver hardcodes bit 0x3503[2] to 1, which means "sensor gain
> format". Maybe setting it to 0 ("real gain format") would produce saner
> results ?
>
>> > to implement this calculation in the sensor driver and expose some
>> > linear "logical" gain to userspace (ranging, e.g., 128-2048 for gains
>> > 1x-16x)?
>>
>> Yes. This way the user space can somehow work without knowing this special
>> implementation, even though the granularity changes over the range. I guess
>> the granularity would need to be known in libcamera but that's a separate
>> issue.
>
> I can live with that.

I got some fresh data regarding gain setting, with gain register value
ranging from 0 to 4096, please check the attached plot. What is the best
way to expose this to userspace in your opinion?

--
Best regards,
Mikhail Rudenko


Attachments:
gain_2.pdf (43.69 kB)

2022-09-21 13:39:35

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail,

On Tue, Sep 20, 2022 at 11:31:01PM +0300, Mikhail Rudenko wrote:
>
> Hi Laurent, Sakari,
>
> On 2022-09-19 at 16:49 +03, Laurent Pinchart <[email protected]> wrote:
>
> > Hello,
> >
> > On Mon, Sep 19, 2022 at 10:31:02AM +0000, Sakari Ailus wrote:
> >> On Mon, Sep 19, 2022 at 10:01:06AM +0300, Mikhail Rudenko wrote:
> >> > On 2022-09-19 at 06:40 GMT, Sakari Ailus wrote:
> >> > > On Fri, Sep 16, 2022 at 12:27:42AM +0300, Mikhail Rudenko wrote:
> >> > >> On 2022-09-14 at 10:58 +01, Dave Stevenson wrote:
> >> > >> > On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko wrote:
> >> > >> >>
> >> > >> >> Hello,
> >> > >> >>
> >> > >> >> this series implements support for Omnivision OV4689 image
> >> > >> >> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> >> > >> >> megapixel image sensor. Ihis chip supports high frame rate speeds up
> >> > >> >> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> >> > >> >> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> >> > >> >> connection.
> >> > >> >>
> >> > >> >> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> >> > >> >> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> >> > >> >> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera
> >> > >> >> module.
> >> > >> >> While porting the driver, I stumbled upon two issues:
> >> >
> >> > [snip]
> >> >
> >> > >> >> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> >> > >> >> gain is not linear across that range. Instead, it is piecewise linear
> >> > >> >> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> >> > >> >> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> >> > >> >> with more linear segments in between. Rockchip's camera engine code
> >> > >> >> chooses one of the above segments depenging on the desired gain
> >> > >> >> value. The question is, how should we proceed keeping in mind
> >> > >> >> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> >> > >> >> libcamera will do the mapping, or the driver will do the mapping
> >> > >> >> itself and expose some logical gain units not tied to the actual gain
> >> > >> >> register value? Meanwhile, this driver conservatively exposes only
> >> > >> >> 0x0-0xf8 gain register range.
> >> > >> >
> >> > >> > The datasheet linked above says "for the gain formula, please contact
> >> > >> > your local OmniVision FAE" :-(
> >> > >> > I would assume that the range is from 1x rather than 0x - people
> >> > >> > rarely want a totally black image that 0x would give. Or is it ranges
> >> > >> > of 1x - 2x, 2x - 4x, 4x - 8x, and 8x - 16x?
> >> > >>
> >> > >> A picture is worth a thousand words, so I've attached the results of my
> >> > >> experimentation with the gain register. They were obtained with Rockchip
> >> > >> 3399, with AEC, AGC and black level subtraction disabled. The image was
> >> > >> converted from 10-bit RGGB to 8-bit YUV 4:2:0 by the Rockchip ISP.
> >
> > Is that full or limited range YUV ?
> >
> >> > > Based on that it looks like their medication may have been a little too
> >> > > strong.
> >> > >
> >> > > Could this be implemented so that the control value would be linear linear
> >> > > but its range would correspond 1x--16x values?
> >> > >
> >> > > libcamera will be able to cope with that.
> >> >
> >> > According to the following fragment of the Rockchip camera engine sensor
> >> > configuration file for ov4689 [1]
> >> >
> >> > <Linear index="1" type="double" size="[4 7]">
> >> > [1 2 128 0 1 128 255
> >> > 2 4 64 -248 1 376 504
> >> > 4 8 32 -756 1 884 1012
> >> > 8 16 16 -1784 1 1912 2040]
> >> > </Linear>,
> >> >
> >> > it uses gain register value range 128-255 for gain 1x-2x, 376-504 for
> >> > gain 2x-4x, 884-1024 for 4x-8x, and 1912-2040 for 8x-16x. Do you suggest
> >
> > That looks *really* weird. I would have understood [384, 511], [896,
> > 1023] and [1920, 2047], but not those intervals.
> >
> > The driver hardcodes bit 0x3503[2] to 1, which means "sensor gain
> > format". Maybe setting it to 0 ("real gain format") would produce saner
> > results ?
> >
> >> > to implement this calculation in the sensor driver and expose some
> >> > linear "logical" gain to userspace (ranging, e.g., 128-2048 for gains
> >> > 1x-16x)?
> >>
> >> Yes. This way the user space can somehow work without knowing this special
> >> implementation, even though the granularity changes over the range. I guess
> >> the granularity would need to be known in libcamera but that's a separate
> >> issue.
> >
> > I can live with that.
>
> I got some fresh data regarding gain setting, with gain register value
> ranging from 0 to 4096, please check the attached plot. What is the best
> way to expose this to userspace in your opinion?

I know I requested this to be changed to be as linear as possible, but it
would seem that the gain values do not seem to match exactly what is
documented. So you'd need to experimentally find where you'd need to switch
the range and you might arrive at better values after the initial
implementation. There might be differences between units, too, and if there
were tuning values, you'd find this in sensor EEPROM.

Therefore I think this appears to fit less well for driver implementation.
I'm fine with exposing this to the user space as-is although it doesn't
make a great user space inteface. It's just a poor hardware implementation
but there's nothing we can do about it.

--
Kind regards,

Sakari Ailus

2022-09-22 10:02:54

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,

On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
> bus for data.
>
> This driver supports following features:
> - manual exposure and analog gain control support
> - test pattern support
> - media controller support
> - runtime PM support
> - support following resolutions:
> + 2688x1520 at 30 fps
>
> The driver provides all mandatory V4L2 controls for compatibility with
> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
> implements 4 lane mode only at this moment.
>
> Signed-off-by: Mikhail Rudenko <[email protected]>
> ---
> MAINTAINERS | 1 +
> drivers/media/i2c/Kconfig | 14 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 967 insertions(+)
> create mode 100644 drivers/media/i2c/ov4689.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 63c4844f26e6..1857f3864e1b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -14529,6 +14529,7 @@ L: [email protected]
> S: Maintained
> T: git git://linuxtv.org/media_tree.git
> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
> +F: drivers/media/i2c/ov5647.c
>
> OMNIVISION OV5640 SENSOR DRIVER
> M: Steve Longerbeam <[email protected]>
> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
> index fae2baabb773..4993e1ae2ea8 100644
> --- a/drivers/media/i2c/Kconfig
> +++ b/drivers/media/i2c/Kconfig
> @@ -429,6 +429,20 @@ config VIDEO_OV2740
> To compile this driver as a module, choose M here: the
> module will be called ov2740.
>
> +config VIDEO_OV4689
> + tristate "OmniVision OV4689 sensor support"
> + depends on OF
> + depends on GPIOLIB && VIDEO_DEV && I2C
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + This is a Video4Linux2 sensor-level driver for the OmniVision
> + OV4689 camera.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called ov4689.
> +
> config VIDEO_OV5640
> tristate "OmniVision OV5640 sensor support"
> depends on OF
> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
> index 3e1696963e7f..7446c0a1eed0 100644
> --- a/drivers/media/i2c/Makefile
> +++ b/drivers/media/i2c/Makefile
> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
> new file mode 100644
> index 000000000000..9f05e812acf8
> --- /dev/null
> +++ b/drivers/media/i2c/ov4689.c
> @@ -0,0 +1,951 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ov4689 driver
> + *
> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/device.h>
> +#include <linux/delay.h>
> +#include <linux/gpio/consumer.h>
> +#include <linux/i2c.h>
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regulator/consumer.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#define CHIP_ID 0x004688
> +#define OV4689_REG_CHIP_ID 0x300a
> +
> +#define OV4689_XVCLK_FREQ 24000000
> +
> +#define OV4689_REG_CTRL_MODE 0x0100
> +#define OV4689_MODE_SW_STANDBY 0x0
> +#define OV4689_MODE_STREAMING BIT(0)
> +
> +#define OV4689_REG_EXPOSURE 0x3500
> +#define OV4689_EXPOSURE_MIN 4
> +#define OV4689_EXPOSURE_STEP 1
> +#define OV4689_VTS_MAX 0x7fff
> +
> +#define OV4689_REG_GAIN_H 0x3508
> +#define OV4689_REG_GAIN_L 0x3509
> +#define OV4689_GAIN_H_MASK 0x07
> +#define OV4689_GAIN_H_SHIFT 8
> +#define OV4689_GAIN_L_MASK 0xff
> +#define OV4689_GAIN_MIN 0x10
> +#define OV4689_GAIN_MAX 0xf8
> +#define OV4689_GAIN_STEP 1
> +#define OV4689_GAIN_DEFAULT 0x10
> +
> +#define OV4689_REG_TEST_PATTERN 0x5040
> +#define OV4689_TEST_PATTERN_ENABLE 0x80
> +#define OV4689_TEST_PATTERN_DISABLE 0x0
> +
> +#define OV4689_REG_VTS 0x380e
> +
> +#define REG_NULL 0xFFFF
> +
> +#define OV4689_REG_VALUE_08BIT 1
> +#define OV4689_REG_VALUE_16BIT 2
> +#define OV4689_REG_VALUE_24BIT 3
> +
> +#define OV4689_LANES 4
> +#define OV4689_BITS_PER_SAMPLE 10
> +
> +static const char *const ov4689_supply_names[] = {
> + "avdd", /* Analog power */
> + "dovdd", /* Digital I/O power */
> + "dvdd", /* Digital core power */
> +};
> +
> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)

I think it'd be cleaner to use ARRAY_SIZE(ov4689_supply_names) instead.

> +
> +struct regval {
> + u16 addr;
> + u8 val;
> +};
> +
> +struct ov4689_mode {
> + u32 width;
> + u32 height;
> + u32 max_fps;
> + u32 hts_def;
> + u32 vts_def;
> + u32 exp_def;
> + const struct regval *reg_list;
> +};
> +
> +struct ov4689 {
> + struct i2c_client *client;
> + struct clk *xvclk;
> + struct gpio_desc *reset_gpio;
> + struct gpio_desc *pwdn_gpio;
> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
> +
> + struct v4l2_subdev subdev;
> + struct media_pad pad;
> +
> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
> + bool streaming;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct v4l2_ctrl *exposure;
> + struct v4l2_ctrl *anal_gain;
> + struct v4l2_ctrl *digi_gain;
> + struct v4l2_ctrl *hblank;
> + struct v4l2_ctrl *vblank;
> + struct v4l2_ctrl *test_pattern;

Only keep the controls you need elsewhere.

> +
> + const struct ov4689_mode *cur_mode;
> +};
> +
> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
> +
> +/*
> + * Xclk 24Mhz
> + */
> +static const struct regval ov4689_global_regs[] = {
> + { REG_NULL, 0x00 },
> +};
> +
> +/*
> + * Xclk 24Mhz
> + * max_framerate 30fps
> + * mipi_datarate per lane 1008Mbps
> + */
> +static const struct regval ov4689_2688x1520_regs[] = {
> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
> + {REG_NULL, 0x00},
> +};
> +
> +static const struct ov4689_mode supported_modes[] = {
> + {
> + .width = 2688,
> + .height = 1520,
> + .max_fps = 30,
> + .exp_def = 0x0600,
> + .hts_def = 0x0a80,
> + .vts_def = 0x0612,
> + .reg_list = ov4689_2688x1520_regs,
> + },
> +};
> +
> +#define OV4689_LINK_FREQ_500MHZ 500000000

Please use the plain number --- see also comments in probe.

> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
> +
> +static const char *const ov4689_test_pattern_menu[] = {
> + "Disabled",
> + "Vertical Color Bar Type 1",
> + "Vertical Color Bar Type 2",
> + "Vertical Color Bar Type 3",
> + "Vertical Color Bar Type 4"
> +};
> +
> +/* Write registers up to 4 at a time */
> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
> + u32 val)
> +{
> + u32 buf_i, val_i;
> + __be32 val_be;
> + u8 *val_p;
> + u8 buf[6];
> +
> + if (len > 4)
> + return -EINVAL;
> +
> + buf[0] = reg >> 8;
> + buf[1] = reg & 0xff;
> +
> + val_be = cpu_to_be32(val);
> + val_p = (u8 *)&val_be;
> + buf_i = 2;
> + val_i = 4 - len;
> +
> + while (val_i < 4)
> + buf[buf_i++] = val_p[val_i++];
> +
> + if (i2c_master_send(client, buf, len + 2) != len + 2)
> + return -EIO;
> +
> + return 0;
> +}
> +
> +static int ov4689_write_array(struct i2c_client *client,
> + const struct regval *regs)
> +{
> + int ret = 0;
> + u32 i;
> +
> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
> + ret = ov4689_write_reg(client, regs[i].addr,
> + OV4689_REG_VALUE_08BIT, regs[i].val);
> +
> + return ret;
> +}
> +
> +/* Read registers up to 4 at a time */
> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
> + u32 *val)
> +{
> + __be16 reg_addr_be = cpu_to_be16(reg);
> + struct i2c_msg msgs[2];
> + __be32 data_be = 0;
> + u8 *data_be_p;
> + int ret;
> +
> + if (len > 4 || !len)
> + return -EINVAL;
> +
> + data_be_p = (u8 *)&data_be;
> + /* Write register address */
> + msgs[0].addr = client->addr;
> + msgs[0].flags = 0;
> + msgs[0].len = 2;
> + msgs[0].buf = (u8 *)&reg_addr_be;
> +
> + /* Read data from register */
> + msgs[1].addr = client->addr;
> + msgs[1].flags = I2C_M_RD;
> + msgs[1].len = len;
> + msgs[1].buf = &data_be_p[4 - len];
> +
> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
> + if (ret != ARRAY_SIZE(msgs))
> + return -EIO;
> +
> + *val = be32_to_cpu(data_be);
> +
> + return 0;
> +}
> +
> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
> + struct v4l2_mbus_framefmt *fmt)
> +{
> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> + fmt->width = mode->width;
> + fmt->height = mode->height;
> + fmt->field = V4L2_FIELD_NONE;
> +}
> +
> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index != 0)
> + return -EINVAL;
> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + if (fse->index >= ARRAY_SIZE(supported_modes))
> + return -EINVAL;
> +
> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
> + return -EINVAL;
> +
> + fse->min_width = supported_modes[fse->index].width;
> + fse->max_width = supported_modes[fse->index].width;
> + fse->max_height = supported_modes[fse->index].height;
> + fse->min_height = supported_modes[fse->index].height;
> +
> + return 0;
> +}
> +
> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
> +{
> + u32 val;
> +
> + if (pattern)
> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
> + else
> + val = OV4689_TEST_PATTERN_DISABLE;
> +
> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
> + OV4689_REG_VALUE_08BIT, val);
> +}
> +
> +static int ov4689_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> + return -EINVAL;
> +
> + switch (sel->target) {
> + case V4L2_SEL_TGT_CROP_BOUNDS:
> + sel->r.top = 0;
> + sel->r.left = 0;
> + sel->r.width = 2720;
> + sel->r.height = 1536;
> + return 0;
> + case V4L2_SEL_TGT_CROP:
> + case V4L2_SEL_TGT_CROP_DEFAULT:
> + sel->r.top = 8;
> + sel->r.left = 16;
> + sel->r.width = 2688;
> + sel->r.height = 1520;
> + return 0;
> + }
> + return -EINVAL;
> +}
> +
> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct i2c_client *client = ov4689->client;
> + int ret = 0;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + on = !!on;
> + if (on == ov4689->streaming)
> + goto unlock_and_return;
> +
> + if (on) {
> + ret = pm_runtime_resume_and_get(&client->dev);
> + if (ret < 0)
> + goto unlock_and_return;
> +
> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_array(ov4689->client,
> + ov4689->cur_mode->reg_list);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_STREAMING);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> + } else {
> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_SW_STANDBY);
> + pm_runtime_put(&client->dev);
> + }
> +
> + ov4689->streaming = on;
> +
> +unlock_and_return:
> + mutex_unlock(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +/* Calculate the delay in us by clock rate and clock cycles */
> +static inline u32 ov4689_cal_delay(u32 cycles)
> +{
> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);

Please use the actual rate instead.

> +}
> +
> +static int __ov4689_power_on(struct ov4689 *ov4689)
> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 delay_us;
> + int ret;
> +
> + ret = clk_prepare_enable(ov4689->xvclk);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable xvclk\n");
> + return ret;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> +
> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable regulators\n");
> + goto disable_clk;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
> + usleep_range(500, 1000);
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
> +
> + /* 8192 cycles prior to first SCCB transaction */
> + delay_us = ov4689_cal_delay(8192);
> + usleep_range(delay_us, delay_us * 2);
> +
> + return 0;
> +
> +disable_clk:
> + clk_disable_unprepare(ov4689->xvclk);
> +
> + return ret;
> +}
> +
> +static void __ov4689_power_off(struct ov4689 *ov4689)
> +{
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
> + clk_disable_unprepare(ov4689->xvclk);
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}

Please merge these two and the wrappers below.

> +
> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + return __ov4689_power_on(ov4689);
> +}
> +
> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + __ov4689_power_off(ov4689);
> +
> + return 0;
> +}
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct v4l2_mbus_framefmt *try_fmt;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
> + /* Initialize try_fmt */
> + ov4689_fill_fmt(&supported_modes[0], try_fmt);
> +
> + mutex_unlock(&ov4689->mutex);
> +
> + return 0;
> +}
> +#endif
> +
> +static const struct dev_pm_ops ov4689_pm_ops = {
> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
> +};
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
> + .open = ov4689_open,
> +};
> +#endif
> +
> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
> + .s_stream = ov4689_s_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
> + .enum_mbus_code = ov4689_enum_mbus_code,
> + .enum_frame_size = ov4689_enum_frame_sizes,
> + .get_fmt = ov4689_get_fmt,
> + .set_fmt = ov4689_set_fmt,
> + .get_selection = ov4689_get_selection,
> +};
> +
> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
> + .video = &ov4689_video_ops,
> + .pad = &ov4689_pad_ops,
> +};
> +
> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct ov4689 *ov4689 =
> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
> + struct i2c_client *client = ov4689->client;
> + s64 max_expo;
> + int ret;
> +
> + /* Propagate change of current control to all related controls */
> + switch (ctrl->id) {
> + case V4L2_CID_VBLANK:
> + /* Update max exposure while meeting expected vblanking */
> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
> + __v4l2_ctrl_modify_range(ov4689->exposure,
> + ov4689->exposure->minimum, max_expo,
> + ov4689->exposure->step,
> + ov4689->exposure->default_value);
> + break;
> + }
> +
> + if (!pm_runtime_get_if_in_use(&client->dev))
> + return 0;
> +
> + switch (ctrl->id) {
> + case V4L2_CID_EXPOSURE:
> + /* 4 least significant bits of expsoure are fractional part */
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
> + break;
> + case V4L2_CID_ANALOGUE_GAIN:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
> + OV4689_REG_VALUE_08BIT,
> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
> + OV4689_GAIN_H_MASK);
> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,

ret = ret ?: ...;

> + OV4689_REG_VALUE_08BIT,
> + ctrl->val & OV4689_GAIN_L_MASK);
> + break;
> + case V4L2_CID_VBLANK:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
> + OV4689_REG_VALUE_16BIT,
> + ctrl->val + ov4689->cur_mode->height);
> + break;
> + case V4L2_CID_TEST_PATTERN:
> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
> + break;
> + default:
> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
> + __func__, ctrl->id, ctrl->val);
> + ret = -EINVAL;
> + break;
> + }
> +
> + pm_runtime_put(&client->dev);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
> + .s_ctrl = ov4689_set_ctrl,
> +};
> +
> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
> +{
> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
> + struct v4l2_fwnode_device_properties props;
> + struct v4l2_ctrl_handler *handler;
> + const struct ov4689_mode *mode;
> + s64 exposure_max, vblank_def;
> + struct v4l2_ctrl *ctrl;
> + u32 h_blank, pixel_rate;
> + int ret;
> +
> + handler = &ov4689->ctrl_handler;
> + mode = ov4689->cur_mode;
> + ret = v4l2_ctrl_handler_init(handler, 10);
> + if (ret)
> + return ret;
> + handler->lock = &ov4689->mutex;
> +
> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
> + link_freq_menu_items);
> + if (ctrl)
> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
> + OV4689_BITS_PER_SAMPLE;
> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
> + pixel_rate);
> +
> + h_blank = mode->hts_def - mode->width;
> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
> + h_blank, h_blank, 1, h_blank);
> + if (ov4689->hblank)
> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + vblank_def = mode->vts_def - mode->height;
> + ov4689->vblank =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
> + vblank_def);
> +
> + exposure_max = mode->vts_def - 4;
> + ov4689->exposure =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
> + OV4689_EXPOSURE_MIN, exposure_max,
> + OV4689_EXPOSURE_STEP, mode->exp_def);
> +
> + ov4689->anal_gain =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
> + OV4689_GAIN_DEFAULT);
> +
> + ov4689->test_pattern =
> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
> + V4L2_CID_TEST_PATTERN,
> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
> + 0, 0, ov4689_test_pattern_menu);
> +
> + if (handler->error) {
> + ret = handler->error;
> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
> + ret);
> + goto err_free_handler;
> + }
> +
> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
> + &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ov4689->subdev.ctrl_handler = handler;
> +
> + return 0;
> +
> +err_free_handler:
> + v4l2_ctrl_handler_free(handler);
> +
> + return ret;
> +}
> +
> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
> + struct i2c_client *client)
> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 id = 0;
> + int ret;
> +
> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
> + OV4689_REG_VALUE_16BIT, &id);
> + if (id != CHIP_ID) {
> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
> + return -ENODEV;
> + }
> +
> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
> +
> + return 0;
> +}
> +
> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
> + ov4689->supplies[i].supply = ov4689_supply_names[i];
> +
> + return devm_regulator_bulk_get(&ov4689->client->dev,
> + OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}
> +
> +static int ov4689_check_hwcfg(struct device *dev)
> +{
> + struct fwnode_handle *fwnode = dev_fwnode(dev);
> + struct v4l2_fwnode_endpoint bus_cfg = {
> + .bus_type = V4L2_MBUS_CSI2_DPHY,
> + };
> + struct fwnode_handle *endpoint;
> + unsigned int i;
> + int ret;
> +
> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
> + if (!endpoint)
> + return -EPROBE_DEFER;
> +
> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
> + fwnode_handle_put(endpoint);
> + if (ret)
> + return ret;
> +
> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
> + dev_err(dev, "only a 4-lane CSI2 config is supported");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + if (!bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "no link frequencies defined\n");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)

Please instead compare with array entries.

> + break;
> +
> + if (i == bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "supported link freq %ull not found\n",
> + OV4689_LINK_FREQ_500MHZ);
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> +out_free_bus_cfg:
> + v4l2_fwnode_endpoint_free(&bus_cfg);
> +
> + return ret;
> +}
> +
> +static int ov4689_probe(struct i2c_client *client,
> + const struct i2c_device_id *id)
> +{
> + struct device *dev = &client->dev;
> + struct v4l2_subdev *sd;
> + struct ov4689 *ov4689;
> + int ret;
> +
> + ret = ov4689_check_hwcfg(dev);
> + if (ret)
> + return ret;
> +
> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
> + if (!ov4689)
> + return -ENOMEM;
> +
> + ov4689->client = client;
> + ov4689->cur_mode = &supported_modes[0];
> +
> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
> + if (IS_ERR(ov4689->xvclk)) {
> + dev_err(dev, "Failed to get xvclk\n");
> + return -EINVAL;
> + }
> +
> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);

Please see handling clocks in:

<URL:https://hverkuil.home.xs4all.nl/spec/driver-api/camera-sensor.html>

> + if (ret < 0) {
> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> + return ret;
> + }
> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> +
> + ov4689->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->reset_gpio)) {
> + dev_err(dev, "Failed to get reset-gpios\n");
> + return -EINVAL;
> + }
> +
> + ov4689->pwdn_gpio = devm_gpiod_get(dev, "pwdn", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->pwdn_gpio)) {
> + dev_err(dev, "Failed to get pwdn-gpios\n");
> + return -EINVAL;
> + }
> +
> + ret = ov4689_configure_regulators(ov4689);
> + if (ret) {
> + dev_err(dev, "Failed to get power regulators\n");
> + return ret;
> + }
> +
> + mutex_init(&ov4689->mutex);
> +
> + sd = &ov4689->subdev;
> + v4l2_i2c_subdev_init(sd, client, &ov4689_subdev_ops);
> + ret = ov4689_initialize_controls(ov4689);
> + if (ret)
> + goto err_destroy_mutex;
> +
> + ret = __ov4689_power_on(ov4689);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = ov4689_check_sensor_id(ov4689, client);
> + if (ret)
> + goto err_power_off;
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> + sd->internal_ops = &ov4689_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> +#endif
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + ov4689->pad.flags = MEDIA_PAD_FL_SOURCE;
> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
> + ret = media_entity_pads_init(&sd->entity, 1, &ov4689->pad);
> + if (ret < 0)
> + goto err_power_off;
> +#endif
> +
> + ret = v4l2_async_register_subdev_sensor(sd);
> + if (ret) {
> + dev_err(dev, "v4l2 async register subdev failed\n");
> + goto err_clean_entity;
> + }
> +
> + pm_runtime_set_active(dev);
> + pm_runtime_enable(dev);
> + pm_runtime_idle(dev);
> +
> + return 0;
> +
> +err_clean_entity:
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + media_entity_cleanup(&sd->entity);
> +#endif
> +err_power_off:
> + __ov4689_power_off(ov4689);
> +err_free_handler:
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> +err_destroy_mutex:
> + mutex_destroy(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +static int ov4689_remove(struct i2c_client *client)
> +{
> + struct v4l2_subdev *sd = i2c_get_clientdata(client);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + v4l2_async_unregister_subdev(sd);
> +#if defined(CONFIG_MEDIA_CONTROLLER)

No need for #if here, please drop.

> + media_entity_cleanup(&sd->entity);
> +#endif
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> + mutex_destroy(&ov4689->mutex);
> +
> + pm_runtime_disable(&client->dev);
> + if (!pm_runtime_status_suspended(&client->dev))
> + __ov4689_power_off(ov4689);
> + pm_runtime_set_suspended(&client->dev);
> +
> + return 0;
> +}
> +
> +static const struct i2c_device_id ov4689_id[] = {
> + { "ov4689", 0 },
> + {},
> +};
> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
> +
> +static const struct of_device_id ov4689_of_match[] = {
> + { .compatible = "ovti,ov4689" },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
> +
> +static struct i2c_driver ov4689_i2c_driver = {
> + .driver = {
> + .name = "ov4689",
> + .pm = &ov4689_pm_ops,
> + .of_match_table = of_match_ptr(ov4689_of_match),
> + },
> + .probe = ov4689_probe,
> + .remove = ov4689_remove,
> + .id_table = ov4689_id,
> +};
> +
> +module_i2c_driver(ov4689_i2c_driver);
> +
> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
> +MODULE_LICENSE("GPL");
> --
> 2.37.3
>

--
Sakari Ailus

2022-09-22 10:32:55

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail,

On Sun, Sep 11, 2022 at 11:01:33PM +0300, Mikhail Rudenko wrote:
> Hello,
>
> this series implements support for Omnivision OV4689 image
> sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> megapixel image sensor. Ihis chip supports high frame rate speeds up
> to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> connection.
>
> The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
>
> While porting the driver, I stumbled upon two issues:
>
> (1) In the original driver, horizontal total size (HTS) was set to a
> value (2584) lower then the frame width (2688), resulting in negative
> hblank. In this driver, I increased HTS to 2688, but fps dropped from
> 29.88 to 28.73. What is the preferred way to handle this?

If horizontal total size is less than the frame width, something is
certainly wrong there. You can't have negative horizontal blanking. Neither
it can be zero.

>
> (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> gain is not linear across that range. Instead, it is piecewise linear
> (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> with more linear segments in between. Rockchip's camera engine code
> chooses one of the above segments depenging on the desired gain
> value. The question is, how should we proceed keeping in mind
> libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> libcamera will do the mapping, or the driver will do the mapping
> itself and expose some logical gain units not tied to the actual gain
> register value? Meanwhile, this driver conservatively exposes only
> 0x0-0xf8 gain register range.

--
Sakari Ailus

2022-09-22 11:08:54

by Dave Stevenson

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail

On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
>
> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
> bus for data.
>
> This driver supports following features:
> - manual exposure and analog gain control support
> - test pattern support
> - media controller support
> - runtime PM support
> - support following resolutions:
> + 2688x1520 at 30 fps
>
> The driver provides all mandatory V4L2 controls for compatibility with
> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
> implements 4 lane mode only at this moment.
>
> Signed-off-by: Mikhail Rudenko <[email protected]>
> ---
> MAINTAINERS | 1 +
> drivers/media/i2c/Kconfig | 14 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 967 insertions(+)
> create mode 100644 drivers/media/i2c/ov4689.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 63c4844f26e6..1857f3864e1b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -14529,6 +14529,7 @@ L: [email protected]
> S: Maintained
> T: git git://linuxtv.org/media_tree.git
> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
> +F: drivers/media/i2c/ov5647.c
>
> OMNIVISION OV5640 SENSOR DRIVER
> M: Steve Longerbeam <[email protected]>
> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
> index fae2baabb773..4993e1ae2ea8 100644
> --- a/drivers/media/i2c/Kconfig
> +++ b/drivers/media/i2c/Kconfig
> @@ -429,6 +429,20 @@ config VIDEO_OV2740
> To compile this driver as a module, choose M here: the
> module will be called ov2740.
>
> +config VIDEO_OV4689
> + tristate "OmniVision OV4689 sensor support"
> + depends on OF
> + depends on GPIOLIB && VIDEO_DEV && I2C
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + This is a Video4Linux2 sensor-level driver for the OmniVision
> + OV4689 camera.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called ov4689.
> +
> config VIDEO_OV5640
> tristate "OmniVision OV5640 sensor support"
> depends on OF
> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
> index 3e1696963e7f..7446c0a1eed0 100644
> --- a/drivers/media/i2c/Makefile
> +++ b/drivers/media/i2c/Makefile
> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
> new file mode 100644
> index 000000000000..9f05e812acf8
> --- /dev/null
> +++ b/drivers/media/i2c/ov4689.c
> @@ -0,0 +1,951 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * ov4689 driver
> + *
> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/device.h>
> +#include <linux/delay.h>
> +#include <linux/gpio/consumer.h>
> +#include <linux/i2c.h>
> +#include <linux/module.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regulator/consumer.h>
> +#include <media/media-entity.h>
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-fwnode.h>
> +
> +#define CHIP_ID 0x004688
> +#define OV4689_REG_CHIP_ID 0x300a
> +
> +#define OV4689_XVCLK_FREQ 24000000
> +
> +#define OV4689_REG_CTRL_MODE 0x0100
> +#define OV4689_MODE_SW_STANDBY 0x0
> +#define OV4689_MODE_STREAMING BIT(0)
> +
> +#define OV4689_REG_EXPOSURE 0x3500
> +#define OV4689_EXPOSURE_MIN 4
> +#define OV4689_EXPOSURE_STEP 1
> +#define OV4689_VTS_MAX 0x7fff
> +
> +#define OV4689_REG_GAIN_H 0x3508
> +#define OV4689_REG_GAIN_L 0x3509
> +#define OV4689_GAIN_H_MASK 0x07
> +#define OV4689_GAIN_H_SHIFT 8
> +#define OV4689_GAIN_L_MASK 0xff
> +#define OV4689_GAIN_MIN 0x10
> +#define OV4689_GAIN_MAX 0xf8
> +#define OV4689_GAIN_STEP 1
> +#define OV4689_GAIN_DEFAULT 0x10
> +
> +#define OV4689_REG_TEST_PATTERN 0x5040
> +#define OV4689_TEST_PATTERN_ENABLE 0x80
> +#define OV4689_TEST_PATTERN_DISABLE 0x0
> +
> +#define OV4689_REG_VTS 0x380e
> +
> +#define REG_NULL 0xFFFF
> +
> +#define OV4689_REG_VALUE_08BIT 1
> +#define OV4689_REG_VALUE_16BIT 2
> +#define OV4689_REG_VALUE_24BIT 3
> +
> +#define OV4689_LANES 4
> +#define OV4689_BITS_PER_SAMPLE 10
> +
> +static const char *const ov4689_supply_names[] = {
> + "avdd", /* Analog power */
> + "dovdd", /* Digital I/O power */
> + "dvdd", /* Digital core power */
> +};
> +
> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
> +
> +struct regval {
> + u16 addr;
> + u8 val;
> +};
> +
> +struct ov4689_mode {
> + u32 width;
> + u32 height;
> + u32 max_fps;
> + u32 hts_def;
> + u32 vts_def;
> + u32 exp_def;
> + const struct regval *reg_list;
> +};
> +
> +struct ov4689 {
> + struct i2c_client *client;
> + struct clk *xvclk;
> + struct gpio_desc *reset_gpio;
> + struct gpio_desc *pwdn_gpio;
> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
> +
> + struct v4l2_subdev subdev;
> + struct media_pad pad;
> +
> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
> + bool streaming;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct v4l2_ctrl *exposure;
> + struct v4l2_ctrl *anal_gain;
> + struct v4l2_ctrl *digi_gain;
> + struct v4l2_ctrl *hblank;
> + struct v4l2_ctrl *vblank;
> + struct v4l2_ctrl *test_pattern;
> +
> + const struct ov4689_mode *cur_mode;
> +};
> +
> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
> +
> +/*
> + * Xclk 24Mhz
> + */
> +static const struct regval ov4689_global_regs[] = {
> + { REG_NULL, 0x00 },
> +};
> +
> +/*
> + * Xclk 24Mhz
> + * max_framerate 30fps
> + * mipi_datarate per lane 1008Mbps

Data rate stated as 1008Mbps here....

> + */
> +static const struct regval ov4689_2688x1520_regs[] = {
> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
> + {REG_NULL, 0x00},
> +};
> +
> +static const struct ov4689_mode supported_modes[] = {
> + {
> + .width = 2688,
> + .height = 1520,
> + .max_fps = 30,
> + .exp_def = 0x0600,
> + .hts_def = 0x0a80,
> + .vts_def = 0x0612,
> + .reg_list = ov4689_2688x1520_regs,
> + },
> +};
> +
> +#define OV4689_LINK_FREQ_500MHZ 500000000
> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };

... but a link frequency of 500MHz (ie 1000Mbit/s) here.
Seeing as you compute the pixel rate based on the link frequency,
that's going to mean that the pixel rate is incorrect.
Link frequency should be 504MHz.

Your PLL settings appear to match the 24MHz configuration in table
2-11 "sample PLL configuration" of the datasheet, so it would confirm
that MIPI_SCLK is 1008MHz and MIPI_PCLK is 126MHz (at 1008/8 seems to
be more byte clock than pixel (10bpp) clock).

Dave

> +
> +static const char *const ov4689_test_pattern_menu[] = {
> + "Disabled",
> + "Vertical Color Bar Type 1",
> + "Vertical Color Bar Type 2",
> + "Vertical Color Bar Type 3",
> + "Vertical Color Bar Type 4"
> +};
> +
> +/* Write registers up to 4 at a time */
> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
> + u32 val)
> +{
> + u32 buf_i, val_i;
> + __be32 val_be;
> + u8 *val_p;
> + u8 buf[6];
> +
> + if (len > 4)
> + return -EINVAL;
> +
> + buf[0] = reg >> 8;
> + buf[1] = reg & 0xff;
> +
> + val_be = cpu_to_be32(val);
> + val_p = (u8 *)&val_be;
> + buf_i = 2;
> + val_i = 4 - len;
> +
> + while (val_i < 4)
> + buf[buf_i++] = val_p[val_i++];
> +
> + if (i2c_master_send(client, buf, len + 2) != len + 2)
> + return -EIO;
> +
> + return 0;
> +}
> +
> +static int ov4689_write_array(struct i2c_client *client,
> + const struct regval *regs)
> +{
> + int ret = 0;
> + u32 i;
> +
> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
> + ret = ov4689_write_reg(client, regs[i].addr,
> + OV4689_REG_VALUE_08BIT, regs[i].val);
> +
> + return ret;
> +}
> +
> +/* Read registers up to 4 at a time */
> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
> + u32 *val)
> +{
> + __be16 reg_addr_be = cpu_to_be16(reg);
> + struct i2c_msg msgs[2];
> + __be32 data_be = 0;
> + u8 *data_be_p;
> + int ret;
> +
> + if (len > 4 || !len)
> + return -EINVAL;
> +
> + data_be_p = (u8 *)&data_be;
> + /* Write register address */
> + msgs[0].addr = client->addr;
> + msgs[0].flags = 0;
> + msgs[0].len = 2;
> + msgs[0].buf = (u8 *)&reg_addr_be;
> +
> + /* Read data from register */
> + msgs[1].addr = client->addr;
> + msgs[1].flags = I2C_M_RD;
> + msgs[1].len = len;
> + msgs[1].buf = &data_be_p[4 - len];
> +
> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
> + if (ret != ARRAY_SIZE(msgs))
> + return -EIO;
> +
> + *val = be32_to_cpu(data_be);
> +
> + return 0;
> +}
> +
> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
> + struct v4l2_mbus_framefmt *fmt)
> +{
> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> + fmt->width = mode->width;
> + fmt->height = mode->height;
> + fmt->field = V4L2_FIELD_NONE;
> +}
> +
> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + /* only one mode supported for now */
> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index != 0)
> + return -EINVAL;
> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
> +
> + return 0;
> +}
> +
> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_frame_size_enum *fse)
> +{
> + if (fse->index >= ARRAY_SIZE(supported_modes))
> + return -EINVAL;
> +
> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
> + return -EINVAL;
> +
> + fse->min_width = supported_modes[fse->index].width;
> + fse->max_width = supported_modes[fse->index].width;
> + fse->max_height = supported_modes[fse->index].height;
> + fse->min_height = supported_modes[fse->index].height;
> +
> + return 0;
> +}
> +
> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
> +{
> + u32 val;
> +
> + if (pattern)
> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
> + else
> + val = OV4689_TEST_PATTERN_DISABLE;
> +
> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
> + OV4689_REG_VALUE_08BIT, val);
> +}
> +
> +static int ov4689_get_selection(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_selection *sel)
> +{
> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> + return -EINVAL;
> +
> + switch (sel->target) {
> + case V4L2_SEL_TGT_CROP_BOUNDS:
> + sel->r.top = 0;
> + sel->r.left = 0;
> + sel->r.width = 2720;
> + sel->r.height = 1536;
> + return 0;
> + case V4L2_SEL_TGT_CROP:
> + case V4L2_SEL_TGT_CROP_DEFAULT:
> + sel->r.top = 8;
> + sel->r.left = 16;
> + sel->r.width = 2688;
> + sel->r.height = 1520;
> + return 0;
> + }
> + return -EINVAL;
> +}
> +
> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct i2c_client *client = ov4689->client;
> + int ret = 0;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + on = !!on;
> + if (on == ov4689->streaming)
> + goto unlock_and_return;
> +
> + if (on) {
> + ret = pm_runtime_resume_and_get(&client->dev);
> + if (ret < 0)
> + goto unlock_and_return;
> +
> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_array(ov4689->client,
> + ov4689->cur_mode->reg_list);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> +
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_STREAMING);
> + if (ret) {
> + pm_runtime_put(&client->dev);
> + goto unlock_and_return;
> + }
> + } else {
> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
> + OV4689_REG_VALUE_08BIT,
> + OV4689_MODE_SW_STANDBY);
> + pm_runtime_put(&client->dev);
> + }
> +
> + ov4689->streaming = on;
> +
> +unlock_and_return:
> + mutex_unlock(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +/* Calculate the delay in us by clock rate and clock cycles */
> +static inline u32 ov4689_cal_delay(u32 cycles)
> +{
> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
> +}
> +
> +static int __ov4689_power_on(struct ov4689 *ov4689)
> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 delay_us;
> + int ret;
> +
> + ret = clk_prepare_enable(ov4689->xvclk);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable xvclk\n");
> + return ret;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> +
> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> + if (ret < 0) {
> + dev_err(dev, "Failed to enable regulators\n");
> + goto disable_clk;
> + }
> +
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
> + usleep_range(500, 1000);
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
> +
> + /* 8192 cycles prior to first SCCB transaction */
> + delay_us = ov4689_cal_delay(8192);
> + usleep_range(delay_us, delay_us * 2);
> +
> + return 0;
> +
> +disable_clk:
> + clk_disable_unprepare(ov4689->xvclk);
> +
> + return ret;
> +}
> +
> +static void __ov4689_power_off(struct ov4689 *ov4689)
> +{
> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
> + clk_disable_unprepare(ov4689->xvclk);
> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}
> +
> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + return __ov4689_power_on(ov4689);
> +}
> +
> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + __ov4689_power_off(ov4689);
> +
> + return 0;
> +}
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> +{
> + struct ov4689 *ov4689 = to_ov4689(sd);
> + struct v4l2_mbus_framefmt *try_fmt;
> +
> + mutex_lock(&ov4689->mutex);
> +
> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
> + /* Initialize try_fmt */
> + ov4689_fill_fmt(&supported_modes[0], try_fmt);
> +
> + mutex_unlock(&ov4689->mutex);
> +
> + return 0;
> +}
> +#endif
> +
> +static const struct dev_pm_ops ov4689_pm_ops = {
> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
> +};
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
> + .open = ov4689_open,
> +};
> +#endif
> +
> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
> + .s_stream = ov4689_s_stream,
> +};
> +
> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
> + .enum_mbus_code = ov4689_enum_mbus_code,
> + .enum_frame_size = ov4689_enum_frame_sizes,
> + .get_fmt = ov4689_get_fmt,
> + .set_fmt = ov4689_set_fmt,
> + .get_selection = ov4689_get_selection,
> +};
> +
> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
> + .video = &ov4689_video_ops,
> + .pad = &ov4689_pad_ops,
> +};
> +
> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct ov4689 *ov4689 =
> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
> + struct i2c_client *client = ov4689->client;
> + s64 max_expo;
> + int ret;
> +
> + /* Propagate change of current control to all related controls */
> + switch (ctrl->id) {
> + case V4L2_CID_VBLANK:
> + /* Update max exposure while meeting expected vblanking */
> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
> + __v4l2_ctrl_modify_range(ov4689->exposure,
> + ov4689->exposure->minimum, max_expo,
> + ov4689->exposure->step,
> + ov4689->exposure->default_value);
> + break;
> + }
> +
> + if (!pm_runtime_get_if_in_use(&client->dev))
> + return 0;
> +
> + switch (ctrl->id) {
> + case V4L2_CID_EXPOSURE:
> + /* 4 least significant bits of expsoure are fractional part */
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
> + break;
> + case V4L2_CID_ANALOGUE_GAIN:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
> + OV4689_REG_VALUE_08BIT,
> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
> + OV4689_GAIN_H_MASK);
> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
> + OV4689_REG_VALUE_08BIT,
> + ctrl->val & OV4689_GAIN_L_MASK);
> + break;
> + case V4L2_CID_VBLANK:
> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
> + OV4689_REG_VALUE_16BIT,
> + ctrl->val + ov4689->cur_mode->height);
> + break;
> + case V4L2_CID_TEST_PATTERN:
> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
> + break;
> + default:
> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
> + __func__, ctrl->id, ctrl->val);
> + ret = -EINVAL;
> + break;
> + }
> +
> + pm_runtime_put(&client->dev);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
> + .s_ctrl = ov4689_set_ctrl,
> +};
> +
> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
> +{
> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
> + struct v4l2_fwnode_device_properties props;
> + struct v4l2_ctrl_handler *handler;
> + const struct ov4689_mode *mode;
> + s64 exposure_max, vblank_def;
> + struct v4l2_ctrl *ctrl;
> + u32 h_blank, pixel_rate;
> + int ret;
> +
> + handler = &ov4689->ctrl_handler;
> + mode = ov4689->cur_mode;
> + ret = v4l2_ctrl_handler_init(handler, 10);
> + if (ret)
> + return ret;
> + handler->lock = &ov4689->mutex;
> +
> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
> + link_freq_menu_items);
> + if (ctrl)
> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
> + OV4689_BITS_PER_SAMPLE;
> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
> + pixel_rate);
> +
> + h_blank = mode->hts_def - mode->width;
> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
> + h_blank, h_blank, 1, h_blank);
> + if (ov4689->hblank)
> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +
> + vblank_def = mode->vts_def - mode->height;
> + ov4689->vblank =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
> + vblank_def);
> +
> + exposure_max = mode->vts_def - 4;
> + ov4689->exposure =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
> + OV4689_EXPOSURE_MIN, exposure_max,
> + OV4689_EXPOSURE_STEP, mode->exp_def);
> +
> + ov4689->anal_gain =
> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
> + OV4689_GAIN_DEFAULT);
> +
> + ov4689->test_pattern =
> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
> + V4L2_CID_TEST_PATTERN,
> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
> + 0, 0, ov4689_test_pattern_menu);
> +
> + if (handler->error) {
> + ret = handler->error;
> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
> + ret);
> + goto err_free_handler;
> + }
> +
> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
> + &props);
> + if (ret)
> + goto err_free_handler;
> +
> + ov4689->subdev.ctrl_handler = handler;
> +
> + return 0;
> +
> +err_free_handler:
> + v4l2_ctrl_handler_free(handler);
> +
> + return ret;
> +}
> +
> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
> + struct i2c_client *client)
> +{
> + struct device *dev = &ov4689->client->dev;
> + u32 id = 0;
> + int ret;
> +
> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
> + OV4689_REG_VALUE_16BIT, &id);
> + if (id != CHIP_ID) {
> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
> + return -ENODEV;
> + }
> +
> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
> +
> + return 0;
> +}
> +
> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
> + ov4689->supplies[i].supply = ov4689_supply_names[i];
> +
> + return devm_regulator_bulk_get(&ov4689->client->dev,
> + OV4689_NUM_SUPPLIES, ov4689->supplies);
> +}
> +
> +static int ov4689_check_hwcfg(struct device *dev)
> +{
> + struct fwnode_handle *fwnode = dev_fwnode(dev);
> + struct v4l2_fwnode_endpoint bus_cfg = {
> + .bus_type = V4L2_MBUS_CSI2_DPHY,
> + };
> + struct fwnode_handle *endpoint;
> + unsigned int i;
> + int ret;
> +
> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
> + if (!endpoint)
> + return -EPROBE_DEFER;
> +
> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
> + fwnode_handle_put(endpoint);
> + if (ret)
> + return ret;
> +
> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
> + dev_err(dev, "only a 4-lane CSI2 config is supported");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + if (!bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "no link frequencies defined\n");
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
> + break;
> +
> + if (i == bus_cfg.nr_of_link_frequencies) {
> + dev_err(dev, "supported link freq %ull not found\n",
> + OV4689_LINK_FREQ_500MHZ);
> + ret = -EINVAL;
> + goto out_free_bus_cfg;
> + }
> +
> +out_free_bus_cfg:
> + v4l2_fwnode_endpoint_free(&bus_cfg);
> +
> + return ret;
> +}
> +
> +static int ov4689_probe(struct i2c_client *client,
> + const struct i2c_device_id *id)
> +{
> + struct device *dev = &client->dev;
> + struct v4l2_subdev *sd;
> + struct ov4689 *ov4689;
> + int ret;
> +
> + ret = ov4689_check_hwcfg(dev);
> + if (ret)
> + return ret;
> +
> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
> + if (!ov4689)
> + return -ENOMEM;
> +
> + ov4689->client = client;
> + ov4689->cur_mode = &supported_modes[0];
> +
> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
> + if (IS_ERR(ov4689->xvclk)) {
> + dev_err(dev, "Failed to get xvclk\n");
> + return -EINVAL;
> + }
> +
> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
> + if (ret < 0) {
> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
> + return ret;
> + }
> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
> +
> + ov4689->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->reset_gpio)) {
> + dev_err(dev, "Failed to get reset-gpios\n");
> + return -EINVAL;
> + }
> +
> + ov4689->pwdn_gpio = devm_gpiod_get(dev, "pwdn", GPIOD_OUT_LOW);
> + if (IS_ERR(ov4689->pwdn_gpio)) {
> + dev_err(dev, "Failed to get pwdn-gpios\n");
> + return -EINVAL;
> + }
> +
> + ret = ov4689_configure_regulators(ov4689);
> + if (ret) {
> + dev_err(dev, "Failed to get power regulators\n");
> + return ret;
> + }
> +
> + mutex_init(&ov4689->mutex);
> +
> + sd = &ov4689->subdev;
> + v4l2_i2c_subdev_init(sd, client, &ov4689_subdev_ops);
> + ret = ov4689_initialize_controls(ov4689);
> + if (ret)
> + goto err_destroy_mutex;
> +
> + ret = __ov4689_power_on(ov4689);
> + if (ret)
> + goto err_free_handler;
> +
> + ret = ov4689_check_sensor_id(ov4689, client);
> + if (ret)
> + goto err_power_off;
> +
> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
> + sd->internal_ops = &ov4689_internal_ops;
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> +#endif
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + ov4689->pad.flags = MEDIA_PAD_FL_SOURCE;
> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
> + ret = media_entity_pads_init(&sd->entity, 1, &ov4689->pad);
> + if (ret < 0)
> + goto err_power_off;
> +#endif
> +
> + ret = v4l2_async_register_subdev_sensor(sd);
> + if (ret) {
> + dev_err(dev, "v4l2 async register subdev failed\n");
> + goto err_clean_entity;
> + }
> +
> + pm_runtime_set_active(dev);
> + pm_runtime_enable(dev);
> + pm_runtime_idle(dev);
> +
> + return 0;
> +
> +err_clean_entity:
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + media_entity_cleanup(&sd->entity);
> +#endif
> +err_power_off:
> + __ov4689_power_off(ov4689);
> +err_free_handler:
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> +err_destroy_mutex:
> + mutex_destroy(&ov4689->mutex);
> +
> + return ret;
> +}
> +
> +static int ov4689_remove(struct i2c_client *client)
> +{
> + struct v4l2_subdev *sd = i2c_get_clientdata(client);
> + struct ov4689 *ov4689 = to_ov4689(sd);
> +
> + v4l2_async_unregister_subdev(sd);
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> + media_entity_cleanup(&sd->entity);
> +#endif
> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
> + mutex_destroy(&ov4689->mutex);
> +
> + pm_runtime_disable(&client->dev);
> + if (!pm_runtime_status_suspended(&client->dev))
> + __ov4689_power_off(ov4689);
> + pm_runtime_set_suspended(&client->dev);
> +
> + return 0;
> +}
> +
> +static const struct i2c_device_id ov4689_id[] = {
> + { "ov4689", 0 },
> + {},
> +};
> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
> +
> +static const struct of_device_id ov4689_of_match[] = {
> + { .compatible = "ovti,ov4689" },
> + {},
> +};
> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
> +
> +static struct i2c_driver ov4689_i2c_driver = {
> + .driver = {
> + .name = "ov4689",
> + .pm = &ov4689_pm_ops,
> + .of_match_table = of_match_ptr(ov4689_of_match),
> + },
> + .probe = ov4689_probe,
> + .remove = ov4689_remove,
> + .id_table = ov4689_id,
> +};
> +
> +module_i2c_driver(ov4689_i2c_driver);
> +
> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
> +MODULE_LICENSE("GPL");
> --
> 2.37.3
>

2022-09-22 11:09:30

by Dave Stevenson

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver

Hi Mikhail & Sakari

On Thu, 22 Sept 2022 at 10:55, Sakari Ailus
<[email protected]> wrote:
>
> Hi Mikhail,
>
> On Sun, Sep 11, 2022 at 11:01:33PM +0300, Mikhail Rudenko wrote:
> > Hello,
> >
> > this series implements support for Omnivision OV4689 image
> > sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
> > megapixel image sensor. Ihis chip supports high frame rate speeds up
> > to 90 fps at 2688x1520 resolution. It is programmable through an I2C
> > interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
> > connection.
> >
> > The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
> > and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
> > 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
> >
> > While porting the driver, I stumbled upon two issues:
> >
> > (1) In the original driver, horizontal total size (HTS) was set to a
> > value (2584) lower then the frame width (2688), resulting in negative
> > hblank. In this driver, I increased HTS to 2688, but fps dropped from
> > 29.88 to 28.73. What is the preferred way to handle this?
>
> If horizontal total size is less than the frame width, something is
> certainly wrong there. You can't have negative horizontal blanking. Neither
> it can be zero.

Something certainly seems odd.

To continue my thoughts from earlier in this patch set, Omnivision's
Product Brief [1] states:
The 1/3-inch OV4689 can capture full-resolution 4-megapixel high
definition (HD) video at 90 frames per second (fps), 1080p HD at 120
fps, and binned 720p HD at 180 fps

The datasheet section 2.1 states:
The OV4689 color image sensor is a high performance, 4 megapixel RAW
image sensor that delivers 2688x1520 at 90 fps using OmniBSI-2™ pixel
technology.

So 4MP 90fps or 1080p120 should be achievable somehow.

2688x1520 @ 90fps is 367.7MPix/s, and that tallies quite nicely with
table 2-9 listing the DAC PLL speed limitation of 360-378MHz. Exactly
how that is then converted into PCLK or SCLK is unclear.
Ideally you'd be able to contact an Omnivision FAE to confirm, but
that means you need to be buying modules directly from them or
otherwise have a business relationship.
I do note that there is an NVidia Tegra driver for OV4689 at [2]. I
wonder if analysis of that would reveal anything.

I have just been looking at the ov9282 driver and the timings don't
tally there either - configure it for 60fps and you get 30fps. The
TIMING_HTS register appears to be in units of 2 pixels. The default is
0x2d8 (728 decimal) on a 1280x720 mode, but consider it as units of 2
pixels and HTS of 1456 (1280 active and hblank of 176) does match up.
It works in the general case too.

Looking at the OV4689 datasheet again, the default for TIMING_HTS
(0x380c/d) is 0x5f8 (1528 decimal) when HOUTPUT_SIZE (0x3808/9) is
0x1200 (4608 decimal). Whilst there are no guarantees that default
register settings will stream in any sensible form, it does imply
TIMING_HTS is not in units of 1 pixel, and potentially 4 pixels.
From the Rockchip BSP driver it plainly does stream at 30 (or 29.88)
fps with TIMING_HTS < HOUTPUT_SIZE, so a quick test of reducing it
further would be worthwhile. What does the default of 0x2d8 give you
as a frame rate, and are the images correct?

Just some thoughts.
Dave

[1] https://www.ovt.com/wp-content/uploads/2022/01/OV4689-PB-v1.7-WEB.pdf
[2] https://github.com/bogsen/STLinux-Kernel/blob/master/drivers/media/platform/tegra/ov4689.c


> >
> > (2) The original driver exposes analog gain range 0x0 - 0x7ff, but the
> > gain is not linear across that range. Instead, it is piecewise linear
> > (and discontinuous). 0x0-0xff register values result in 0x-2x gain,
> > 0x100-0x1ff to 0x-4x, 0x300-0x3ff to 0x-8x, and 0x700-0x7ff to 0x-16x,
> > with more linear segments in between. Rockchip's camera engine code
> > chooses one of the above segments depenging on the desired gain
> > value. The question is, how should we proceed keeping in mind
> > libcamera use case? Should the whole 0x0-0x7ff be exposed as-is and
> > libcamera will do the mapping, or the driver will do the mapping
> > itself and expose some logical gain units not tied to the actual gain
> > register value? Meanwhile, this driver conservatively exposes only
> > 0x0-0xf8 gain register range.
>
> --
> Sakari Ailus

2022-09-22 15:18:41

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Dave,

and thanks for reviewing this!

On 2022-09-22 at 11:54 +01, Dave Stevenson <[email protected]> wrote:

> Hi Mikhail
>
> On Sun, 11 Sept 2022 at 21:02, Mikhail Rudenko <[email protected]> wrote:
>>
>> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
>> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
>> bus for data.
>>
>> This driver supports following features:
>> - manual exposure and analog gain control support
>> - test pattern support
>> - media controller support
>> - runtime PM support
>> - support following resolutions:
>> + 2688x1520 at 30 fps
>>
>> The driver provides all mandatory V4L2 controls for compatibility with
>> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
>> implements 4 lane mode only at this moment.
>>
>> Signed-off-by: Mikhail Rudenko <[email protected]>
>> ---
>> MAINTAINERS | 1 +
>> drivers/media/i2c/Kconfig | 14 +
>> drivers/media/i2c/Makefile | 1 +
>> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
>> 4 files changed, 967 insertions(+)
>> create mode 100644 drivers/media/i2c/ov4689.c
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 63c4844f26e6..1857f3864e1b 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -14529,6 +14529,7 @@ L: [email protected]
>> S: Maintained
>> T: git git://linuxtv.org/media_tree.git
>> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
>> +F: drivers/media/i2c/ov5647.c
>>
>> OMNIVISION OV5640 SENSOR DRIVER
>> M: Steve Longerbeam <[email protected]>
>> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
>> index fae2baabb773..4993e1ae2ea8 100644
>> --- a/drivers/media/i2c/Kconfig
>> +++ b/drivers/media/i2c/Kconfig
>> @@ -429,6 +429,20 @@ config VIDEO_OV2740
>> To compile this driver as a module, choose M here: the
>> module will be called ov2740.
>>
>> +config VIDEO_OV4689
>> + tristate "OmniVision OV4689 sensor support"
>> + depends on OF
>> + depends on GPIOLIB && VIDEO_DEV && I2C
>> + select MEDIA_CONTROLLER
>> + select VIDEO_V4L2_SUBDEV_API
>> + select V4L2_FWNODE
>> + help
>> + This is a Video4Linux2 sensor-level driver for the OmniVision
>> + OV4689 camera.
>> +
>> + To compile this driver as a module, choose M here: the
>> + module will be called ov4689.
>> +
>> config VIDEO_OV5640
>> tristate "OmniVision OV5640 sensor support"
>> depends on OF
>> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
>> index 3e1696963e7f..7446c0a1eed0 100644
>> --- a/drivers/media/i2c/Makefile
>> +++ b/drivers/media/i2c/Makefile
>> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
>> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
>> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
>> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
>> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
>> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
>> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
>> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
>> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
>> new file mode 100644
>> index 000000000000..9f05e812acf8
>> --- /dev/null
>> +++ b/drivers/media/i2c/ov4689.c
>> @@ -0,0 +1,951 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ov4689 driver
>> + *
>> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
>> + */
>> +
>> +#include <linux/clk.h>
>> +#include <linux/device.h>
>> +#include <linux/delay.h>
>> +#include <linux/gpio/consumer.h>
>> +#include <linux/i2c.h>
>> +#include <linux/module.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/regulator/consumer.h>
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-async.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/v4l2-fwnode.h>
>> +
>> +#define CHIP_ID 0x004688
>> +#define OV4689_REG_CHIP_ID 0x300a
>> +
>> +#define OV4689_XVCLK_FREQ 24000000
>> +
>> +#define OV4689_REG_CTRL_MODE 0x0100
>> +#define OV4689_MODE_SW_STANDBY 0x0
>> +#define OV4689_MODE_STREAMING BIT(0)
>> +
>> +#define OV4689_REG_EXPOSURE 0x3500
>> +#define OV4689_EXPOSURE_MIN 4
>> +#define OV4689_EXPOSURE_STEP 1
>> +#define OV4689_VTS_MAX 0x7fff
>> +
>> +#define OV4689_REG_GAIN_H 0x3508
>> +#define OV4689_REG_GAIN_L 0x3509
>> +#define OV4689_GAIN_H_MASK 0x07
>> +#define OV4689_GAIN_H_SHIFT 8
>> +#define OV4689_GAIN_L_MASK 0xff
>> +#define OV4689_GAIN_MIN 0x10
>> +#define OV4689_GAIN_MAX 0xf8
>> +#define OV4689_GAIN_STEP 1
>> +#define OV4689_GAIN_DEFAULT 0x10
>> +
>> +#define OV4689_REG_TEST_PATTERN 0x5040
>> +#define OV4689_TEST_PATTERN_ENABLE 0x80
>> +#define OV4689_TEST_PATTERN_DISABLE 0x0
>> +
>> +#define OV4689_REG_VTS 0x380e
>> +
>> +#define REG_NULL 0xFFFF
>> +
>> +#define OV4689_REG_VALUE_08BIT 1
>> +#define OV4689_REG_VALUE_16BIT 2
>> +#define OV4689_REG_VALUE_24BIT 3
>> +
>> +#define OV4689_LANES 4
>> +#define OV4689_BITS_PER_SAMPLE 10
>> +
>> +static const char *const ov4689_supply_names[] = {
>> + "avdd", /* Analog power */
>> + "dovdd", /* Digital I/O power */
>> + "dvdd", /* Digital core power */
>> +};
>> +
>> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
>> +
>> +struct regval {
>> + u16 addr;
>> + u8 val;
>> +};
>> +
>> +struct ov4689_mode {
>> + u32 width;
>> + u32 height;
>> + u32 max_fps;
>> + u32 hts_def;
>> + u32 vts_def;
>> + u32 exp_def;
>> + const struct regval *reg_list;
>> +};
>> +
>> +struct ov4689 {
>> + struct i2c_client *client;
>> + struct clk *xvclk;
>> + struct gpio_desc *reset_gpio;
>> + struct gpio_desc *pwdn_gpio;
>> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
>> +
>> + struct v4l2_subdev subdev;
>> + struct media_pad pad;
>> +
>> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
>> + bool streaming;
>> + struct v4l2_ctrl_handler ctrl_handler;
>> + struct v4l2_ctrl *exposure;
>> + struct v4l2_ctrl *anal_gain;
>> + struct v4l2_ctrl *digi_gain;
>> + struct v4l2_ctrl *hblank;
>> + struct v4l2_ctrl *vblank;
>> + struct v4l2_ctrl *test_pattern;
>> +
>> + const struct ov4689_mode *cur_mode;
>> +};
>> +
>> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
>> +
>> +/*
>> + * Xclk 24Mhz
>> + */
>> +static const struct regval ov4689_global_regs[] = {
>> + { REG_NULL, 0x00 },
>> +};
>> +
>> +/*
>> + * Xclk 24Mhz
>> + * max_framerate 30fps
>> + * mipi_datarate per lane 1008Mbps
>
> Data rate stated as 1008Mbps here....
>
>> + */
>> +static const struct regval ov4689_2688x1520_regs[] = {
>> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
>> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
>> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
>> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
>> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
>> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
>> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
>> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
>> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
>> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
>> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
>> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
>> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
>> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
>> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
>> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
>> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
>> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
>> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
>> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
>> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
>> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
>> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
>> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
>> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
>> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
>> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
>> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
>> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
>> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
>> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
>> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
>> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
>> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
>> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
>> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
>> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
>> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
>> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
>> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
>> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
>> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
>> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
>> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
>> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
>> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
>> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
>> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
>> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
>> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
>> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
>> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
>> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
>> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
>> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
>> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
>> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
>> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
>> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
>> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
>> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
>> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
>> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
>> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
>> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
>> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
>> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
>> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
>> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
>> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
>> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
>> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
>> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
>> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
>> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
>> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
>> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
>> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
>> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
>> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
>> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
>> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
>> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
>> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
>> + {REG_NULL, 0x00},
>> +};
>> +
>> +static const struct ov4689_mode supported_modes[] = {
>> + {
>> + .width = 2688,
>> + .height = 1520,
>> + .max_fps = 30,
>> + .exp_def = 0x0600,
>> + .hts_def = 0x0a80,
>> + .vts_def = 0x0612,
>> + .reg_list = ov4689_2688x1520_regs,
>> + },
>> +};
>> +
>> +#define OV4689_LINK_FREQ_500MHZ 500000000
>> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
>
> ... but a link frequency of 500MHz (ie 1000Mbit/s) here.
> Seeing as you compute the pixel rate based on the link frequency,
> that's going to mean that the pixel rate is incorrect.
> Link frequency should be 504MHz.
>
> Your PLL settings appear to match the 24MHz configuration in table
> 2-11 "sample PLL configuration" of the datasheet, so it would confirm
> that MIPI_SCLK is 1008MHz and MIPI_PCLK is 126MHz (at 1008/8 seems to
> be more byte clock than pixel (10bpp) clock).

Those are leftovers from the bsp driver, and they will be definitely
changed in v3. Recovering correct pixel clock, hblank and vblank
matching the observed fps needs some experimentation, which I hope to
carry out over the weekend.

>
> Dave
>

--
Best regards,
Mikhail

2022-09-22 15:54:13

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver


Hi Dave,

On 2022-09-22 at 11:43 +01, Dave Stevenson <[email protected]> wrote:

> Hi Mikhail & Sakari
>
> On Thu, 22 Sept 2022 at 10:55, Sakari Ailus
> <[email protected]> wrote:
>>
>> Hi Mikhail,
>>
>> On Sun, Sep 11, 2022 at 11:01:33PM +0300, Mikhail Rudenko wrote:
>> > Hello,
>> >
>> > this series implements support for Omnivision OV4689 image
>> > sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
>> > megapixel image sensor. Ihis chip supports high frame rate speeds up
>> > to 90 fps at 2688x1520 resolution. It is programmable through an I2C
>> > interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
>> > connection.
>> >
>> > The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
>> > and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
>> > 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
>> >
>> > While porting the driver, I stumbled upon two issues:
>> >
>> > (1) In the original driver, horizontal total size (HTS) was set to a
>> > value (2584) lower then the frame width (2688), resulting in negative
>> > hblank. In this driver, I increased HTS to 2688, but fps dropped from
>> > 29.88 to 28.73. What is the preferred way to handle this?
>>
>> If horizontal total size is less than the frame width, something is
>> certainly wrong there. You can't have negative horizontal blanking. Neither
>> it can be zero.
>
> Something certainly seems odd.
>
> To continue my thoughts from earlier in this patch set, Omnivision's
> Product Brief [1] states:
> The 1/3-inch OV4689 can capture full-resolution 4-megapixel high
> definition (HD) video at 90 frames per second (fps), 1080p HD at 120
> fps, and binned 720p HD at 180 fps
>
> The datasheet section 2.1 states:
> The OV4689 color image sensor is a high performance, 4 megapixel RAW
> image sensor that delivers 2688x1520 at 90 fps using OmniBSI-2™ pixel
> technology.
>
> So 4MP 90fps or 1080p120 should be achievable somehow.
>
> 2688x1520 @ 90fps is 367.7MPix/s, and that tallies quite nicely with
> table 2-9 listing the DAC PLL speed limitation of 360-378MHz. Exactly
> how that is then converted into PCLK or SCLK is unclear.
> Ideally you'd be able to contact an Omnivision FAE to confirm, but
> that means you need to be buying modules directly from them or
> otherwise have a business relationship.
> I do note that there is an NVidia Tegra driver for OV4689 at [2]. I
> wonder if analysis of that would reveal anything.
>
> I have just been looking at the ov9282 driver and the timings don't
> tally there either - configure it for 60fps and you get 30fps. The
> TIMING_HTS register appears to be in units of 2 pixels. The default is
> 0x2d8 (728 decimal) on a 1280x720 mode, but consider it as units of 2
> pixels and HTS of 1456 (1280 active and hblank of 176) does match up.
> It works in the general case too.
>
> Looking at the OV4689 datasheet again, the default for TIMING_HTS
> (0x380c/d) is 0x5f8 (1528 decimal) when HOUTPUT_SIZE (0x3808/9) is
> 0x1200 (4608 decimal). Whilst there are no guarantees that default
> register settings will stream in any sensible form, it does imply
> TIMING_HTS is not in units of 1 pixel, and potentially 4 pixels.
> From the Rockchip BSP driver it plainly does stream at 30 (or 29.88)
> fps with TIMING_HTS < HOUTPUT_SIZE, so a quick test of reducing it
> further would be worthwhile. What does the default of 0x2d8 give you
> as a frame rate, and are the images correct?

Thanks for sharing this! Actually, I'm going to do some experimentation
with these registers next weekend, and post the result here.

> Just some thoughts.
> Dave

--
Best regards,
Mikhail Rudenko

2022-09-22 16:05:45

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689


Hi Sakari,

and thanks for reviewing this!

Please see my comments below:

On 2022-09-22 at 09:53 GMT, Sakari Ailus <[email protected]> wrote:

> Hi Mikhail,
>
> On Sun, Sep 11, 2022 at 11:01:35PM +0300, Mikhail Rudenko wrote:
>> Add a V4L2 sub-device driver for OmniVision OV4689 image sensor. This
>> is a 4 Mpx image sensor using the I2C bus for control and the CSI-2
>> bus for data.
>>
>> This driver supports following features:
>> - manual exposure and analog gain control support
>> - test pattern support
>> - media controller support
>> - runtime PM support
>> - support following resolutions:
>> + 2688x1520 at 30 fps
>>
>> The driver provides all mandatory V4L2 controls for compatibility with
>> libcamera. The sensor supports 1/2/4-lane CSI-2 modes, but the driver
>> implements 4 lane mode only at this moment.
>>
>> Signed-off-by: Mikhail Rudenko <[email protected]>
>> ---
>> MAINTAINERS | 1 +
>> drivers/media/i2c/Kconfig | 14 +
>> drivers/media/i2c/Makefile | 1 +
>> drivers/media/i2c/ov4689.c | 951 +++++++++++++++++++++++++++++++++++++
>> 4 files changed, 967 insertions(+)
>> create mode 100644 drivers/media/i2c/ov4689.c
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 63c4844f26e6..1857f3864e1b 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -14529,6 +14529,7 @@ L: [email protected]
>> S: Maintained
>> T: git git://linuxtv.org/media_tree.git
>> F: Documentation/devicetree/bindings/media/i2c/ovti,ov4689.yaml
>> +F: drivers/media/i2c/ov5647.c
>>
>> OMNIVISION OV5640 SENSOR DRIVER
>> M: Steve Longerbeam <[email protected]>
>> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
>> index fae2baabb773..4993e1ae2ea8 100644
>> --- a/drivers/media/i2c/Kconfig
>> +++ b/drivers/media/i2c/Kconfig
>> @@ -429,6 +429,20 @@ config VIDEO_OV2740
>> To compile this driver as a module, choose M here: the
>> module will be called ov2740.
>>
>> +config VIDEO_OV4689
>> + tristate "OmniVision OV4689 sensor support"
>> + depends on OF
>> + depends on GPIOLIB && VIDEO_DEV && I2C
>> + select MEDIA_CONTROLLER
>> + select VIDEO_V4L2_SUBDEV_API
>> + select V4L2_FWNODE
>> + help
>> + This is a Video4Linux2 sensor-level driver for the OmniVision
>> + OV4689 camera.
>> +
>> + To compile this driver as a module, choose M here: the
>> + module will be called ov4689.
>> +
>> config VIDEO_OV5640
>> tristate "OmniVision OV5640 sensor support"
>> depends on OF
>> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
>> index 3e1696963e7f..7446c0a1eed0 100644
>> --- a/drivers/media/i2c/Makefile
>> +++ b/drivers/media/i2c/Makefile
>> @@ -78,6 +78,7 @@ obj-$(CONFIG_VIDEO_OV2659) += ov2659.o
>> obj-$(CONFIG_VIDEO_OV2680) += ov2680.o
>> obj-$(CONFIG_VIDEO_OV2685) += ov2685.o
>> obj-$(CONFIG_VIDEO_OV2740) += ov2740.o
>> +obj-$(CONFIG_VIDEO_OV4689) += ov4689.o
>> obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
>> obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
>> obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
>> diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
>> new file mode 100644
>> index 000000000000..9f05e812acf8
>> --- /dev/null
>> +++ b/drivers/media/i2c/ov4689.c
>> @@ -0,0 +1,951 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * ov4689 driver
>> + *
>> + * Copyright (C) 2017 Fuzhou Rockchip Electronics Co., Ltd.
>> + */
>> +
>> +#include <linux/clk.h>
>> +#include <linux/device.h>
>> +#include <linux/delay.h>
>> +#include <linux/gpio/consumer.h>
>> +#include <linux/i2c.h>
>> +#include <linux/module.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/regulator/consumer.h>
>> +#include <media/media-entity.h>
>> +#include <media/v4l2-async.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/v4l2-fwnode.h>
>> +
>> +#define CHIP_ID 0x004688
>> +#define OV4689_REG_CHIP_ID 0x300a
>> +
>> +#define OV4689_XVCLK_FREQ 24000000
>> +
>> +#define OV4689_REG_CTRL_MODE 0x0100
>> +#define OV4689_MODE_SW_STANDBY 0x0
>> +#define OV4689_MODE_STREAMING BIT(0)
>> +
>> +#define OV4689_REG_EXPOSURE 0x3500
>> +#define OV4689_EXPOSURE_MIN 4
>> +#define OV4689_EXPOSURE_STEP 1
>> +#define OV4689_VTS_MAX 0x7fff
>> +
>> +#define OV4689_REG_GAIN_H 0x3508
>> +#define OV4689_REG_GAIN_L 0x3509
>> +#define OV4689_GAIN_H_MASK 0x07
>> +#define OV4689_GAIN_H_SHIFT 8
>> +#define OV4689_GAIN_L_MASK 0xff
>> +#define OV4689_GAIN_MIN 0x10
>> +#define OV4689_GAIN_MAX 0xf8
>> +#define OV4689_GAIN_STEP 1
>> +#define OV4689_GAIN_DEFAULT 0x10
>> +
>> +#define OV4689_REG_TEST_PATTERN 0x5040
>> +#define OV4689_TEST_PATTERN_ENABLE 0x80
>> +#define OV4689_TEST_PATTERN_DISABLE 0x0
>> +
>> +#define OV4689_REG_VTS 0x380e
>> +
>> +#define REG_NULL 0xFFFF
>> +
>> +#define OV4689_REG_VALUE_08BIT 1
>> +#define OV4689_REG_VALUE_16BIT 2
>> +#define OV4689_REG_VALUE_24BIT 3
>> +
>> +#define OV4689_LANES 4
>> +#define OV4689_BITS_PER_SAMPLE 10
>> +
>> +static const char *const ov4689_supply_names[] = {
>> + "avdd", /* Analog power */
>> + "dovdd", /* Digital I/O power */
>> + "dvdd", /* Digital core power */
>> +};
>> +
>> +#define OV4689_NUM_SUPPLIES ARRAY_SIZE(ov4689_supply_names)
>
> I think it'd be cleaner to use ARRAY_SIZE(ov4689_supply_names) instead.
>

Ack, will fix in v3.


>> +
>> +struct regval {
>> + u16 addr;
>> + u8 val;
>> +};
>> +
>> +struct ov4689_mode {
>> + u32 width;
>> + u32 height;
>> + u32 max_fps;
>> + u32 hts_def;
>> + u32 vts_def;
>> + u32 exp_def;
>> + const struct regval *reg_list;
>> +};
>> +
>> +struct ov4689 {
>> + struct i2c_client *client;
>> + struct clk *xvclk;
>> + struct gpio_desc *reset_gpio;
>> + struct gpio_desc *pwdn_gpio;
>> + struct regulator_bulk_data supplies[OV4689_NUM_SUPPLIES];
>> +
>> + struct v4l2_subdev subdev;
>> + struct media_pad pad;
>> +
>> + struct mutex mutex; /* lock to protect streaming, ctrls and cur_mode */
>> + bool streaming;
>> + struct v4l2_ctrl_handler ctrl_handler;
>> + struct v4l2_ctrl *exposure;
>> + struct v4l2_ctrl *anal_gain;
>> + struct v4l2_ctrl *digi_gain;
>> + struct v4l2_ctrl *hblank;
>> + struct v4l2_ctrl *vblank;
>> + struct v4l2_ctrl *test_pattern;
>
> Only keep the controls you need elsewhere.
>

Ack, will remove the unused controls.

>> +
>> + const struct ov4689_mode *cur_mode;
>> +};
>> +
>> +#define to_ov4689(sd) container_of(sd, struct ov4689, subdev)
>> +
>> +/*
>> + * Xclk 24Mhz
>> + */
>> +static const struct regval ov4689_global_regs[] = {
>> + { REG_NULL, 0x00 },
>> +};
>> +
>> +/*
>> + * Xclk 24Mhz
>> + * max_framerate 30fps
>> + * mipi_datarate per lane 1008Mbps
>> + */
>> +static const struct regval ov4689_2688x1520_regs[] = {
>> + {0x0103, 0x01}, {0x3638, 0x00}, {0x0300, 0x00},
>> + {0x0302, 0x2a}, {0x0303, 0x00}, {0x0304, 0x03},
>> + {0x030b, 0x00}, {0x030d, 0x1e}, {0x030e, 0x04},
>> + {0x030f, 0x01}, {0x0312, 0x01}, {0x031e, 0x00},
>> + {0x3000, 0x20}, {0x3002, 0x00}, {0x3018, 0x72},
>> + {0x3020, 0x93}, {0x3021, 0x03}, {0x3022, 0x01},
>> + {0x3031, 0x0a}, {0x303f, 0x0c}, {0x3305, 0xf1},
>> + {0x3307, 0x04}, {0x3309, 0x29}, {0x3500, 0x00},
>> + {0x3501, 0x60}, {0x3502, 0x00}, {0x3503, 0x04},
>> + {0x3504, 0x00}, {0x3505, 0x00}, {0x3506, 0x00},
>> + {0x3507, 0x00}, {0x3508, 0x00}, {0x3509, 0x80},
>> + {0x350a, 0x00}, {0x350b, 0x00}, {0x350c, 0x00},
>> + {0x350d, 0x00}, {0x350e, 0x00}, {0x350f, 0x80},
>> + {0x3510, 0x00}, {0x3511, 0x00}, {0x3512, 0x00},
>> + {0x3513, 0x00}, {0x3514, 0x00}, {0x3515, 0x80},
>> + {0x3516, 0x00}, {0x3517, 0x00}, {0x3518, 0x00},
>> + {0x3519, 0x00}, {0x351a, 0x00}, {0x351b, 0x80},
>> + {0x351c, 0x00}, {0x351d, 0x00}, {0x351e, 0x00},
>> + {0x351f, 0x00}, {0x3520, 0x00}, {0x3521, 0x80},
>> + {0x3522, 0x08}, {0x3524, 0x08}, {0x3526, 0x08},
>> + {0x3528, 0x08}, {0x352a, 0x08}, {0x3602, 0x00},
>> + {0x3603, 0x40}, {0x3604, 0x02}, {0x3605, 0x00},
>> + {0x3606, 0x00}, {0x3607, 0x00}, {0x3609, 0x12},
>> + {0x360a, 0x40}, {0x360c, 0x08}, {0x360f, 0xe5},
>> + {0x3608, 0x8f}, {0x3611, 0x00}, {0x3613, 0xf7},
>> + {0x3616, 0x58}, {0x3619, 0x99}, {0x361b, 0x60},
>> + {0x361c, 0x7a}, {0x361e, 0x79}, {0x361f, 0x02},
>> + {0x3632, 0x00}, {0x3633, 0x10}, {0x3634, 0x10},
>> + {0x3635, 0x10}, {0x3636, 0x15}, {0x3646, 0x86},
>> + {0x364a, 0x0b}, {0x3700, 0x17}, {0x3701, 0x22},
>> + {0x3703, 0x10}, {0x370a, 0x37}, {0x3705, 0x00},
>> + {0x3706, 0x63}, {0x3709, 0x3c}, {0x370b, 0x01},
>> + {0x370c, 0x30}, {0x3710, 0x24}, {0x3711, 0x0c},
>> + {0x3716, 0x00}, {0x3720, 0x28}, {0x3729, 0x7b},
>> + {0x372a, 0x84}, {0x372b, 0xbd}, {0x372c, 0xbc},
>> + {0x372e, 0x52}, {0x373c, 0x0e}, {0x373e, 0x33},
>> + {0x3743, 0x10}, {0x3744, 0x88}, {0x3745, 0xc0},
>> + {0x374a, 0x43}, {0x374c, 0x00}, {0x374e, 0x23},
>> + {0x3751, 0x7b}, {0x3752, 0x84}, {0x3753, 0xbd},
>> + {0x3754, 0xbc}, {0x3756, 0x52}, {0x375c, 0x00},
>> + {0x3760, 0x00}, {0x3761, 0x00}, {0x3762, 0x00},
>> + {0x3763, 0x00}, {0x3764, 0x00}, {0x3767, 0x04},
>> + {0x3768, 0x04}, {0x3769, 0x08}, {0x376a, 0x08},
>> + {0x376b, 0x20}, {0x376c, 0x00}, {0x376d, 0x00},
>> + {0x376e, 0x00}, {0x3773, 0x00}, {0x3774, 0x51},
>> + {0x3776, 0xbd}, {0x3777, 0xbd}, {0x3781, 0x18},
>> + {0x3783, 0x25}, {0x3798, 0x1b}, {0x3800, 0x00},
>> + {0x3801, 0x08}, {0x3802, 0x00}, {0x3803, 0x04},
>> + {0x3804, 0x0a}, {0x3805, 0x97}, {0x3806, 0x05},
>> + {0x3807, 0xfb}, {0x3808, 0x0a}, {0x3809, 0x80},
>> + {0x380a, 0x05}, {0x380b, 0xf0}, {0x380c, 0x0a},
>> + {0x380d, 0x80}, {0x380e, 0x06}, {0x380f, 0x12},
>> + {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00},
>> + {0x3813, 0x04}, {0x3814, 0x01}, {0x3815, 0x01},
>> + {0x3819, 0x01}, {0x3820, 0x00}, {0x3821, 0x06},
>> + {0x3829, 0x00}, {0x382a, 0x01}, {0x382b, 0x01},
>> + {0x382d, 0x7f}, {0x3830, 0x04}, {0x3836, 0x01},
>> + {0x3837, 0x00}, {0x3841, 0x02}, {0x3846, 0x08},
>> + {0x3847, 0x07}, {0x3d85, 0x36}, {0x3d8c, 0x71},
>> + {0x3d8d, 0xcb}, {0x3f0a, 0x00}, {0x4000, 0xf1},
>> + {0x4001, 0x40}, {0x4002, 0x04}, {0x4003, 0x14},
>> + {0x400e, 0x00}, {0x4011, 0x00}, {0x401a, 0x00},
>> + {0x401b, 0x00}, {0x401c, 0x00}, {0x401d, 0x00},
>> + {0x401f, 0x00}, {0x4020, 0x00}, {0x4021, 0x10},
>> + {0x4022, 0x07}, {0x4023, 0xcf}, {0x4024, 0x09},
>> + {0x4025, 0x60}, {0x4026, 0x09}, {0x4027, 0x6f},
>> + {0x4028, 0x00}, {0x4029, 0x02}, {0x402a, 0x06},
>> + {0x402b, 0x04}, {0x402c, 0x02}, {0x402d, 0x02},
>> + {0x402e, 0x0e}, {0x402f, 0x04}, {0x4302, 0xff},
>> + {0x4303, 0xff}, {0x4304, 0x00}, {0x4305, 0x00},
>> + {0x4306, 0x00}, {0x4308, 0x02}, {0x4500, 0x6c},
>> + {0x4501, 0xc4}, {0x4502, 0x40}, {0x4503, 0x01},
>> + {0x4601, 0xa7}, {0x4800, 0x04}, {0x4813, 0x08},
>> + {0x481f, 0x40}, {0x4829, 0x78}, {0x4837, 0x10},
>> + {0x4b00, 0x2a}, {0x4b0d, 0x00}, {0x4d00, 0x04},
>> + {0x4d01, 0x42}, {0x4d02, 0xd1}, {0x4d03, 0x93},
>> + {0x4d04, 0xf5}, {0x4d05, 0xc1}, {0x5000, 0xf3},
>> + {0x5001, 0x11}, {0x5004, 0x00}, {0x500a, 0x00},
>> + {0x500b, 0x00}, {0x5032, 0x00}, {0x5040, 0x00},
>> + {0x5050, 0x0c}, {0x5500, 0x00}, {0x5501, 0x10},
>> + {0x5502, 0x01}, {0x5503, 0x0f}, {0x8000, 0x00},
>> + {0x8001, 0x00}, {0x8002, 0x00}, {0x8003, 0x00},
>> + {0x8004, 0x00}, {0x8005, 0x00}, {0x8006, 0x00},
>> + {0x8007, 0x00}, {0x8008, 0x00}, {0x3638, 0x00},
>> + {REG_NULL, 0x00},
>> +};
>> +
>> +static const struct ov4689_mode supported_modes[] = {
>> + {
>> + .width = 2688,
>> + .height = 1520,
>> + .max_fps = 30,
>> + .exp_def = 0x0600,
>> + .hts_def = 0x0a80,
>> + .vts_def = 0x0612,
>> + .reg_list = ov4689_2688x1520_regs,
>> + },
>> +};
>> +
>> +#define OV4689_LINK_FREQ_500MHZ 500000000
>
> Please use the plain number --- see also comments in probe.
>

Ack.

>> +static const s64 link_freq_menu_items[] = { OV4689_LINK_FREQ_500MHZ };
>> +
>> +static const char *const ov4689_test_pattern_menu[] = {
>> + "Disabled",
>> + "Vertical Color Bar Type 1",
>> + "Vertical Color Bar Type 2",
>> + "Vertical Color Bar Type 3",
>> + "Vertical Color Bar Type 4"
>> +};
>> +
>> +/* Write registers up to 4 at a time */
>> +static int ov4689_write_reg(struct i2c_client *client, u16 reg, u32 len,
>> + u32 val)
>> +{
>> + u32 buf_i, val_i;
>> + __be32 val_be;
>> + u8 *val_p;
>> + u8 buf[6];
>> +
>> + if (len > 4)
>> + return -EINVAL;
>> +
>> + buf[0] = reg >> 8;
>> + buf[1] = reg & 0xff;
>> +
>> + val_be = cpu_to_be32(val);
>> + val_p = (u8 *)&val_be;
>> + buf_i = 2;
>> + val_i = 4 - len;
>> +
>> + while (val_i < 4)
>> + buf[buf_i++] = val_p[val_i++];
>> +
>> + if (i2c_master_send(client, buf, len + 2) != len + 2)
>> + return -EIO;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_write_array(struct i2c_client *client,
>> + const struct regval *regs)
>> +{
>> + int ret = 0;
>> + u32 i;
>> +
>> + for (i = 0; ret == 0 && regs[i].addr != REG_NULL; i++)
>> + ret = ov4689_write_reg(client, regs[i].addr,
>> + OV4689_REG_VALUE_08BIT, regs[i].val);
>> +
>> + return ret;
>> +}
>> +
>> +/* Read registers up to 4 at a time */
>> +static int ov4689_read_reg(struct i2c_client *client, u16 reg, unsigned int len,
>> + u32 *val)
>> +{
>> + __be16 reg_addr_be = cpu_to_be16(reg);
>> + struct i2c_msg msgs[2];
>> + __be32 data_be = 0;
>> + u8 *data_be_p;
>> + int ret;
>> +
>> + if (len > 4 || !len)
>> + return -EINVAL;
>> +
>> + data_be_p = (u8 *)&data_be;
>> + /* Write register address */
>> + msgs[0].addr = client->addr;
>> + msgs[0].flags = 0;
>> + msgs[0].len = 2;
>> + msgs[0].buf = (u8 *)&reg_addr_be;
>> +
>> + /* Read data from register */
>> + msgs[1].addr = client->addr;
>> + msgs[1].flags = I2C_M_RD;
>> + msgs[1].len = len;
>> + msgs[1].buf = &data_be_p[4 - len];
>> +
>> + ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
>> + if (ret != ARRAY_SIZE(msgs))
>> + return -EIO;
>> +
>> + *val = be32_to_cpu(data_be);
>> +
>> + return 0;
>> +}
>> +
>> +static void ov4689_fill_fmt(const struct ov4689_mode *mode,
>> + struct v4l2_mbus_framefmt *fmt)
>> +{
>> + fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
>> + fmt->width = mode->width;
>> + fmt->height = mode->height;
>> + fmt->field = V4L2_FIELD_NONE;
>> +}
>> +
>> +static int ov4689_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + /* only one mode supported for now */
>> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_get_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *fmt)
>> +{
>> + struct v4l2_mbus_framefmt *mbus_fmt = &fmt->format;
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + /* only one mode supported for now */
>> + ov4689_fill_fmt(ov4689->cur_mode, mbus_fmt);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + if (code->index != 0)
>> + return -EINVAL;
>> + code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enum_frame_sizes(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> + if (fse->index >= ARRAY_SIZE(supported_modes))
>> + return -EINVAL;
>> +
>> + if (fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
>> + return -EINVAL;
>> +
>> + fse->min_width = supported_modes[fse->index].width;
>> + fse->max_width = supported_modes[fse->index].width;
>> + fse->max_height = supported_modes[fse->index].height;
>> + fse->min_height = supported_modes[fse->index].height;
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
>> +{
>> + u32 val;
>> +
>> + if (pattern)
>> + val = (pattern - 1) | OV4689_TEST_PATTERN_ENABLE;
>> + else
>> + val = OV4689_TEST_PATTERN_DISABLE;
>> +
>> + return ov4689_write_reg(ov4689->client, OV4689_REG_TEST_PATTERN,
>> + OV4689_REG_VALUE_08BIT, val);
>> +}
>> +
>> +static int ov4689_get_selection(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + if (sel->which != V4L2_SUBDEV_FORMAT_ACTIVE)
>> + return -EINVAL;
>> +
>> + switch (sel->target) {
>> + case V4L2_SEL_TGT_CROP_BOUNDS:
>> + sel->r.top = 0;
>> + sel->r.left = 0;
>> + sel->r.width = 2720;
>> + sel->r.height = 1536;
>> + return 0;
>> + case V4L2_SEL_TGT_CROP:
>> + case V4L2_SEL_TGT_CROP_DEFAULT:
>> + sel->r.top = 8;
>> + sel->r.left = 16;
>> + sel->r.width = 2688;
>> + sel->r.height = 1520;
>> + return 0;
>> + }
>> + return -EINVAL;
>> +}
>> +
>> +static int ov4689_s_stream(struct v4l2_subdev *sd, int on)
>> +{
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> + struct i2c_client *client = ov4689->client;
>> + int ret = 0;
>> +
>> + mutex_lock(&ov4689->mutex);
>> +
>> + on = !!on;
>> + if (on == ov4689->streaming)
>> + goto unlock_and_return;
>> +
>> + if (on) {
>> + ret = pm_runtime_resume_and_get(&client->dev);
>> + if (ret < 0)
>> + goto unlock_and_return;
>> +
>> + ret = __v4l2_ctrl_handler_setup(&ov4689->ctrl_handler);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> +
>> + ret = ov4689_write_array(ov4689->client,
>> + ov4689->cur_mode->reg_list);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> +
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
>> + OV4689_REG_VALUE_08BIT,
>> + OV4689_MODE_STREAMING);
>> + if (ret) {
>> + pm_runtime_put(&client->dev);
>> + goto unlock_and_return;
>> + }
>> + } else {
>> + ov4689_write_reg(ov4689->client, OV4689_REG_CTRL_MODE,
>> + OV4689_REG_VALUE_08BIT,
>> + OV4689_MODE_SW_STANDBY);
>> + pm_runtime_put(&client->dev);
>> + }
>> +
>> + ov4689->streaming = on;
>> +
>> +unlock_and_return:
>> + mutex_unlock(&ov4689->mutex);
>> +
>> + return ret;
>> +}
>> +
>> +/* Calculate the delay in us by clock rate and clock cycles */
>> +static inline u32 ov4689_cal_delay(u32 cycles)
>> +{
>> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
>
> Please use the actual rate instead.
>

Do you mean clk_get_rate(ov4689->xvclk), right? What if we have an ACPI
system and xvclk is NULL here? Please explain.

>> +}
>> +
>> +static int __ov4689_power_on(struct ov4689 *ov4689)
>> +{
>> + struct device *dev = &ov4689->client->dev;
>> + u32 delay_us;
>> + int ret;
>> +
>> + ret = clk_prepare_enable(ov4689->xvclk);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to enable xvclk\n");
>> + return ret;
>> + }
>> +
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
>> +
>> + ret = regulator_bulk_enable(OV4689_NUM_SUPPLIES, ov4689->supplies);
>> + if (ret < 0) {
>> + dev_err(dev, "Failed to enable regulators\n");
>> + goto disable_clk;
>> + }
>> +
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 0);
>> + usleep_range(500, 1000);
>> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 0);
>> +
>> + /* 8192 cycles prior to first SCCB transaction */
>> + delay_us = ov4689_cal_delay(8192);
>> + usleep_range(delay_us, delay_us * 2);
>> +
>> + return 0;
>> +
>> +disable_clk:
>> + clk_disable_unprepare(ov4689->xvclk);
>> +
>> + return ret;
>> +}
>> +
>> +static void __ov4689_power_off(struct ov4689 *ov4689)
>> +{
>> + gpiod_set_value_cansleep(ov4689->pwdn_gpio, 1);
>> + clk_disable_unprepare(ov4689->xvclk);
>> + gpiod_set_value_cansleep(ov4689->reset_gpio, 1);
>> + regulator_bulk_disable(OV4689_NUM_SUPPLIES, ov4689->supplies);
>> +}
>
> Please merge these two and the wrappers below.
>

Ack.

>> +
>> +static int __maybe_unused ov4689_runtime_resume(struct device *dev)
>> +{
>> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + return __ov4689_power_on(ov4689);
>> +}
>> +
>> +static int __maybe_unused ov4689_runtime_suspend(struct device *dev)
>> +{
>> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + __ov4689_power_off(ov4689);
>> +
>> + return 0;
>> +}
>> +
>> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
>> +static int ov4689_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
>> +{
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> + struct v4l2_mbus_framefmt *try_fmt;
>> +
>> + mutex_lock(&ov4689->mutex);
>> +
>> + try_fmt = v4l2_subdev_get_try_format(sd, fh->state, 0);
>> + /* Initialize try_fmt */
>> + ov4689_fill_fmt(&supported_modes[0], try_fmt);
>> +
>> + mutex_unlock(&ov4689->mutex);
>> +
>> + return 0;
>> +}
>> +#endif
>> +
>> +static const struct dev_pm_ops ov4689_pm_ops = {
>> + SET_RUNTIME_PM_OPS(ov4689_runtime_suspend, ov4689_runtime_resume, NULL)
>> +};
>> +
>> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
>> +static const struct v4l2_subdev_internal_ops ov4689_internal_ops = {
>> + .open = ov4689_open,
>> +};
>> +#endif
>> +
>> +static const struct v4l2_subdev_video_ops ov4689_video_ops = {
>> + .s_stream = ov4689_s_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_pad_ops ov4689_pad_ops = {
>> + .enum_mbus_code = ov4689_enum_mbus_code,
>> + .enum_frame_size = ov4689_enum_frame_sizes,
>> + .get_fmt = ov4689_get_fmt,
>> + .set_fmt = ov4689_set_fmt,
>> + .get_selection = ov4689_get_selection,
>> +};
>> +
>> +static const struct v4l2_subdev_ops ov4689_subdev_ops = {
>> + .video = &ov4689_video_ops,
>> + .pad = &ov4689_pad_ops,
>> +};
>> +
>> +static int ov4689_set_ctrl(struct v4l2_ctrl *ctrl)
>> +{
>> + struct ov4689 *ov4689 =
>> + container_of(ctrl->handler, struct ov4689, ctrl_handler);
>> + struct i2c_client *client = ov4689->client;
>> + s64 max_expo;
>> + int ret;
>> +
>> + /* Propagate change of current control to all related controls */
>> + switch (ctrl->id) {
>> + case V4L2_CID_VBLANK:
>> + /* Update max exposure while meeting expected vblanking */
>> + max_expo = ov4689->cur_mode->height + ctrl->val - 4;
>> + __v4l2_ctrl_modify_range(ov4689->exposure,
>> + ov4689->exposure->minimum, max_expo,
>> + ov4689->exposure->step,
>> + ov4689->exposure->default_value);
>> + break;
>> + }
>> +
>> + if (!pm_runtime_get_if_in_use(&client->dev))
>> + return 0;
>> +
>> + switch (ctrl->id) {
>> + case V4L2_CID_EXPOSURE:
>> + /* 4 least significant bits of expsoure are fractional part */
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_EXPOSURE,
>> + OV4689_REG_VALUE_24BIT, ctrl->val << 4);
>> + break;
>> + case V4L2_CID_ANALOGUE_GAIN:
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_H,
>> + OV4689_REG_VALUE_08BIT,
>> + (ctrl->val >> OV4689_GAIN_H_SHIFT) &
>> + OV4689_GAIN_H_MASK);
>> + ret |= ov4689_write_reg(ov4689->client, OV4689_REG_GAIN_L,
>
> ret = ret ?: ...;
>

Ack

>> + OV4689_REG_VALUE_08BIT,
>> + ctrl->val & OV4689_GAIN_L_MASK);
>> + break;
>> + case V4L2_CID_VBLANK:
>> + ret = ov4689_write_reg(ov4689->client, OV4689_REG_VTS,
>> + OV4689_REG_VALUE_16BIT,
>> + ctrl->val + ov4689->cur_mode->height);
>> + break;
>> + case V4L2_CID_TEST_PATTERN:
>> + ret = ov4689_enable_test_pattern(ov4689, ctrl->val);
>> + break;
>> + default:
>> + dev_warn(&client->dev, "%s Unhandled id:0x%x, val:0x%x\n",
>> + __func__, ctrl->id, ctrl->val);
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + pm_runtime_put(&client->dev);
>> +
>> + return ret;
>> +}
>> +
>> +static const struct v4l2_ctrl_ops ov4689_ctrl_ops = {
>> + .s_ctrl = ov4689_set_ctrl,
>> +};
>> +
>> +static int ov4689_initialize_controls(struct ov4689 *ov4689)
>> +{
>> + struct i2c_client *client = v4l2_get_subdevdata(&ov4689->subdev);
>> + struct v4l2_fwnode_device_properties props;
>> + struct v4l2_ctrl_handler *handler;
>> + const struct ov4689_mode *mode;
>> + s64 exposure_max, vblank_def;
>> + struct v4l2_ctrl *ctrl;
>> + u32 h_blank, pixel_rate;
>> + int ret;
>> +
>> + handler = &ov4689->ctrl_handler;
>> + mode = ov4689->cur_mode;
>> + ret = v4l2_ctrl_handler_init(handler, 10);
>> + if (ret)
>> + return ret;
>> + handler->lock = &ov4689->mutex;
>> +
>> + ctrl = v4l2_ctrl_new_int_menu(handler, NULL, V4L2_CID_LINK_FREQ, 0, 0,
>> + link_freq_menu_items);
>> + if (ctrl)
>> + ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + pixel_rate = (link_freq_menu_items[0] * 2 * OV4689_LANES) /
>> + OV4689_BITS_PER_SAMPLE;
>> + v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 0, pixel_rate, 1,
>> + pixel_rate);
>> +
>> + h_blank = mode->hts_def - mode->width;
>> + ov4689->hblank = v4l2_ctrl_new_std(handler, NULL, V4L2_CID_HBLANK,
>> + h_blank, h_blank, 1, h_blank);
>> + if (ov4689->hblank)
>> + ov4689->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>> +
>> + vblank_def = mode->vts_def - mode->height;
>> + ov4689->vblank =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_VBLANK,
>> + vblank_def, OV4689_VTS_MAX - mode->height, 1,
>> + vblank_def);
>> +
>> + exposure_max = mode->vts_def - 4;
>> + ov4689->exposure =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops, V4L2_CID_EXPOSURE,
>> + OV4689_EXPOSURE_MIN, exposure_max,
>> + OV4689_EXPOSURE_STEP, mode->exp_def);
>> +
>> + ov4689->anal_gain =
>> + v4l2_ctrl_new_std(handler, &ov4689_ctrl_ops,
>> + V4L2_CID_ANALOGUE_GAIN, OV4689_GAIN_MIN,
>> + OV4689_GAIN_MAX, OV4689_GAIN_STEP,
>> + OV4689_GAIN_DEFAULT);
>> +
>> + ov4689->test_pattern =
>> + v4l2_ctrl_new_std_menu_items(handler, &ov4689_ctrl_ops,
>> + V4L2_CID_TEST_PATTERN,
>> + ARRAY_SIZE(ov4689_test_pattern_menu) - 1,
>> + 0, 0, ov4689_test_pattern_menu);
>> +
>> + if (handler->error) {
>> + ret = handler->error;
>> + dev_err(&ov4689->client->dev, "Failed to init controls(%d)\n",
>> + ret);
>> + goto err_free_handler;
>> + }
>> +
>> + ret = v4l2_fwnode_device_parse(&client->dev, &props);
>> + if (ret)
>> + goto err_free_handler;
>> +
>> + ret = v4l2_ctrl_new_fwnode_properties(handler, &ov4689_ctrl_ops,
>> + &props);
>> + if (ret)
>> + goto err_free_handler;
>> +
>> + ov4689->subdev.ctrl_handler = handler;
>> +
>> + return 0;
>> +
>> +err_free_handler:
>> + v4l2_ctrl_handler_free(handler);
>> +
>> + return ret;
>> +}
>> +
>> +static int ov4689_check_sensor_id(struct ov4689 *ov4689,
>> + struct i2c_client *client)
>> +{
>> + struct device *dev = &ov4689->client->dev;
>> + u32 id = 0;
>> + int ret;
>> +
>> + ret = ov4689_read_reg(client, OV4689_REG_CHIP_ID,
>> + OV4689_REG_VALUE_16BIT, &id);
>> + if (id != CHIP_ID) {
>> + dev_err(dev, "Unexpected sensor id(%06x), ret(%d)\n", id, ret);
>> + return -ENODEV;
>> + }
>> +
>> + dev_info(dev, "Detected OV%06x sensor\n", CHIP_ID);
>> +
>> + return 0;
>> +}
>> +
>> +static int ov4689_configure_regulators(struct ov4689 *ov4689)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < OV4689_NUM_SUPPLIES; i++)
>> + ov4689->supplies[i].supply = ov4689_supply_names[i];
>> +
>> + return devm_regulator_bulk_get(&ov4689->client->dev,
>> + OV4689_NUM_SUPPLIES, ov4689->supplies);
>> +}
>> +
>> +static int ov4689_check_hwcfg(struct device *dev)
>> +{
>> + struct fwnode_handle *fwnode = dev_fwnode(dev);
>> + struct v4l2_fwnode_endpoint bus_cfg = {
>> + .bus_type = V4L2_MBUS_CSI2_DPHY,
>> + };
>> + struct fwnode_handle *endpoint;
>> + unsigned int i;
>> + int ret;
>> +
>> + endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
>> + if (!endpoint)
>> + return -EPROBE_DEFER;
>> +
>> + ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
>> + fwnode_handle_put(endpoint);
>> + if (ret)
>> + return ret;
>> +
>> + if (bus_cfg.bus.mipi_csi2.num_data_lanes != 4) {
>> + dev_err(dev, "only a 4-lane CSI2 config is supported");
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> + if (!bus_cfg.nr_of_link_frequencies) {
>> + dev_err(dev, "no link frequencies defined\n");
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> + for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
>> + if (bus_cfg.link_frequencies[i] == OV4689_LINK_FREQ_500MHZ)
>
> Please instead compare with array entries.
>

Ack

>> + break;
>> +
>> + if (i == bus_cfg.nr_of_link_frequencies) {
>> + dev_err(dev, "supported link freq %ull not found\n",
>> + OV4689_LINK_FREQ_500MHZ);
>> + ret = -EINVAL;
>> + goto out_free_bus_cfg;
>> + }
>> +
>> +out_free_bus_cfg:
>> + v4l2_fwnode_endpoint_free(&bus_cfg);
>> +
>> + return ret;
>> +}
>> +
>> +static int ov4689_probe(struct i2c_client *client,
>> + const struct i2c_device_id *id)
>> +{
>> + struct device *dev = &client->dev;
>> + struct v4l2_subdev *sd;
>> + struct ov4689 *ov4689;
>> + int ret;
>> +
>> + ret = ov4689_check_hwcfg(dev);
>> + if (ret)
>> + return ret;
>> +
>> + ov4689 = devm_kzalloc(dev, sizeof(*ov4689), GFP_KERNEL);
>> + if (!ov4689)
>> + return -ENOMEM;
>> +
>> + ov4689->client = client;
>> + ov4689->cur_mode = &supported_modes[0];
>> +
>> + ov4689->xvclk = devm_clk_get(dev, "xvclk");
>> + if (IS_ERR(ov4689->xvclk)) {
>> + dev_err(dev, "Failed to get xvclk\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = clk_set_rate(ov4689->xvclk, OV4689_XVCLK_FREQ);
>
> Please see handling clocks in:
>
> <URL:https://hverkuil.home.xs4all.nl/spec/driver-api/camera-sensor.html>
>

Ack.

>> + if (ret < 0) {
>> + dev_err(dev, "Failed to set xvclk rate (24MHz)\n");
>> + return ret;
>> + }
>> + if (clk_get_rate(ov4689->xvclk) != OV4689_XVCLK_FREQ)
>> + dev_warn(dev, "xvclk mismatched, modes are based on 24MHz\n");
>> +
>> + ov4689->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
>> + if (IS_ERR(ov4689->reset_gpio)) {
>> + dev_err(dev, "Failed to get reset-gpios\n");
>> + return -EINVAL;
>> + }
>> +
>> + ov4689->pwdn_gpio = devm_gpiod_get(dev, "pwdn", GPIOD_OUT_LOW);
>> + if (IS_ERR(ov4689->pwdn_gpio)) {
>> + dev_err(dev, "Failed to get pwdn-gpios\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = ov4689_configure_regulators(ov4689);
>> + if (ret) {
>> + dev_err(dev, "Failed to get power regulators\n");
>> + return ret;
>> + }
>> +
>> + mutex_init(&ov4689->mutex);
>> +
>> + sd = &ov4689->subdev;
>> + v4l2_i2c_subdev_init(sd, client, &ov4689_subdev_ops);
>> + ret = ov4689_initialize_controls(ov4689);
>> + if (ret)
>> + goto err_destroy_mutex;
>> +
>> + ret = __ov4689_power_on(ov4689);
>> + if (ret)
>> + goto err_free_handler;
>> +
>> + ret = ov4689_check_sensor_id(ov4689, client);
>> + if (ret)
>> + goto err_power_off;
>> +
>> +#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
>> + sd->internal_ops = &ov4689_internal_ops;
>> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> +#endif
>> +#if defined(CONFIG_MEDIA_CONTROLLER)
>> + ov4689->pad.flags = MEDIA_PAD_FL_SOURCE;
>> + sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
>> + ret = media_entity_pads_init(&sd->entity, 1, &ov4689->pad);
>> + if (ret < 0)
>> + goto err_power_off;
>> +#endif
>> +
>> + ret = v4l2_async_register_subdev_sensor(sd);
>> + if (ret) {
>> + dev_err(dev, "v4l2 async register subdev failed\n");
>> + goto err_clean_entity;
>> + }
>> +
>> + pm_runtime_set_active(dev);
>> + pm_runtime_enable(dev);
>> + pm_runtime_idle(dev);
>> +
>> + return 0;
>> +
>> +err_clean_entity:
>> +#if defined(CONFIG_MEDIA_CONTROLLER)
>> + media_entity_cleanup(&sd->entity);
>> +#endif
>> +err_power_off:
>> + __ov4689_power_off(ov4689);
>> +err_free_handler:
>> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
>> +err_destroy_mutex:
>> + mutex_destroy(&ov4689->mutex);
>> +
>> + return ret;
>> +}
>> +
>> +static int ov4689_remove(struct i2c_client *client)
>> +{
>> + struct v4l2_subdev *sd = i2c_get_clientdata(client);
>> + struct ov4689 *ov4689 = to_ov4689(sd);
>> +
>> + v4l2_async_unregister_subdev(sd);
>> +#if defined(CONFIG_MEDIA_CONTROLLER)
>
> No need for #if here, please drop.
>

Ack

>> + media_entity_cleanup(&sd->entity);
>> +#endif
>> + v4l2_ctrl_handler_free(&ov4689->ctrl_handler);
>> + mutex_destroy(&ov4689->mutex);
>> +
>> + pm_runtime_disable(&client->dev);
>> + if (!pm_runtime_status_suspended(&client->dev))
>> + __ov4689_power_off(ov4689);
>> + pm_runtime_set_suspended(&client->dev);
>> +
>> + return 0;
>> +}
>> +
>> +static const struct i2c_device_id ov4689_id[] = {
>> + { "ov4689", 0 },
>> + {},
>> +};
>> +MODULE_DEVICE_TABLE(i2c, ov4689_id);
>> +
>> +static const struct of_device_id ov4689_of_match[] = {
>> + { .compatible = "ovti,ov4689" },
>> + {},
>> +};
>> +MODULE_DEVICE_TABLE(of, ov4689_of_match);
>> +
>> +static struct i2c_driver ov4689_i2c_driver = {
>> + .driver = {
>> + .name = "ov4689",
>> + .pm = &ov4689_pm_ops,
>> + .of_match_table = of_match_ptr(ov4689_of_match),
>> + },
>> + .probe = ov4689_probe,
>> + .remove = ov4689_remove,
>> + .id_table = ov4689_id,
>> +};
>> +
>> +module_i2c_driver(ov4689_i2c_driver);
>> +
>> +MODULE_DESCRIPTION("OmniVision ov4689 sensor driver");
>> +MODULE_LICENSE("GPL");
>> --
>> 2.37.3
>>


--
Best regards,
Mikhail Rudenko

2022-09-22 21:20:00

by Sakari Ailus

[permalink] [raw]
Subject: Re: [PATCH v2 2/2] media: i2c: add support for ov4689

Hi Mikhail,

On Thu, Sep 22, 2022 at 06:23:05PM +0300, Mikhail Rudenko wrote:
>
> Hi Sakari,
>
> and thanks for reviewing this!

You're welcome!

>
> Please see my comments below:
>
> On 2022-09-22 at 09:53 GMT, Sakari Ailus <[email protected]> wrote:
...
> >> +static inline u32 ov4689_cal_delay(u32 cycles)
> >> +{
> >> + return DIV_ROUND_UP(cycles, OV4689_XVCLK_FREQ / 1000 / 1000);
> >
> > Please use the actual rate instead.
> >
>
> Do you mean clk_get_rate(ov4689->xvclk), right? What if we have an ACPI

Yes.

> system and xvclk is NULL here? Please explain.

There are a few ways this could work on ACPI based systems but generally
you'd have "clock-frequency" property to indicate the frequency even if
clocks themselves wouldn't be available (when controlled via ACPI)).

--
Kind regards,

Sakari Ailus

2022-09-26 14:45:59

by Mikhail Rudenko

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add Omnivision OV4689 image sensor driver


Hi Dave,

On 2022-09-22 at 11:43 +01, Dave Stevenson <[email protected]> wrote:

> Hi Mikhail & Sakari
>
> On Thu, 22 Sept 2022 at 10:55, Sakari Ailus
> <[email protected]> wrote:
>>
>> Hi Mikhail,
>>
>> On Sun, Sep 11, 2022 at 11:01:33PM +0300, Mikhail Rudenko wrote:
>> > Hello,
>> >
>> > this series implements support for Omnivision OV4689 image
>> > sensor. The Omnivision OV4689 is a high performance, 1/3-inch, 4
>> > megapixel image sensor. Ihis chip supports high frame rate speeds up
>> > to 90 fps at 2688x1520 resolution. It is programmable through an I2C
>> > interface, and sensor output is sent via 1/2/4 lane MIPI CSI-2
>> > connection.
>> >
>> > The driver is based on Rockchip BSP kernel [1]. It implements 4-lane CSI-2
>> > and single 2688x1520 @ 30 fps mode. The driver was tested on Rockchip
>> > 3399-based FriendlyElec NanoPi M4 board with MCAM400 camera module.
>> >
>> > While porting the driver, I stumbled upon two issues:
>> >
>> > (1) In the original driver, horizontal total size (HTS) was set to a
>> > value (2584) lower then the frame width (2688), resulting in negative
>> > hblank. In this driver, I increased HTS to 2688, but fps dropped from
>> > 29.88 to 28.73. What is the preferred way to handle this?
>>
>> If horizontal total size is less than the frame width, something is
>> certainly wrong there. You can't have negative horizontal blanking. Neither
>> it can be zero.
>
> Something certainly seems odd.
>
> To continue my thoughts from earlier in this patch set, Omnivision's
> Product Brief [1] states:
> The 1/3-inch OV4689 can capture full-resolution 4-megapixel high
> definition (HD) video at 90 frames per second (fps), 1080p HD at 120
> fps, and binned 720p HD at 180 fps
>
> The datasheet section 2.1 states:
> The OV4689 color image sensor is a high performance, 4 megapixel RAW
> image sensor that delivers 2688x1520 at 90 fps using OmniBSI-2™ pixel
> technology.
>
> So 4MP 90fps or 1080p120 should be achievable somehow.
>
> 2688x1520 @ 90fps is 367.7MPix/s, and that tallies quite nicely with
> table 2-9 listing the DAC PLL speed limitation of 360-378MHz. Exactly
> how that is then converted into PCLK or SCLK is unclear.
> Ideally you'd be able to contact an Omnivision FAE to confirm, but
> that means you need to be buying modules directly from them or
> otherwise have a business relationship.
> I do note that there is an NVidia Tegra driver for OV4689 at [2]. I
> wonder if analysis of that would reveal anything.
>
> I have just been looking at the ov9282 driver and the timings don't
> tally there either - configure it for 60fps and you get 30fps. The
> TIMING_HTS register appears to be in units of 2 pixels. The default is
> 0x2d8 (728 decimal) on a 1280x720 mode, but consider it as units of 2
> pixels and HTS of 1456 (1280 active and hblank of 176) does match up.
> It works in the general case too.
>
> Looking at the OV4689 datasheet again, the default for TIMING_HTS
> (0x380c/d) is 0x5f8 (1528 decimal) when HOUTPUT_SIZE (0x3808/9) is
> 0x1200 (4608 decimal). Whilst there are no guarantees that default
> register settings will stream in any sensible form, it does imply
> TIMING_HTS is not in units of 1 pixel, and potentially 4 pixels.
> From the Rockchip BSP driver it plainly does stream at 30 (or 29.88)
> fps with TIMING_HTS < HOUTPUT_SIZE, so a quick test of reducing it
> further would be worthwhile. What does the default of 0x2d8 give you
> as a frame rate, and are the images correct?
>

I've done some experimentation with timing registers and here is what I
found. First of all, there are two tables with default values of timing
registers (table 4-2 and table 6-9) in publicly available datasheet, and
they (values) are completely different. I'll just leave relevant parts
here for reference:

Defaults (table 4-2):
H_OUT_SIZE: 4608 (0x1200)
V_OUT_SIZE: 3456 (0x0d80)
HTS: 1528 (0x05f8)
VTS: 3492 (0x0da4)

Defaults (table 6-9):
H_OUT_SIZE: 2688 (0xa80)
V_OUT_SIZE: 1520 (0x5f0)
HTS: 860 (0x35c)
VTS: 1552 (0x610)

Driver v2:
H_OUT_SIZE: 2688
V_OUT_SIZE: 1520
HTS(0x380c/0d): 2688 (0x0a80)
VTS(0x380e/0f): 1554 (0x0612)

Then I tried decreasing hts/vts and seeing what happens. The lowest
possible values before frame dropping started were hts=886 and vts=1552,
and the frame rate was 87.27 fps. Multiplying these three numbers yields
pixel rate of 120.0025e6.

So it looks like you are right, and the unit of HTS register is at least
4 pixels. Hence, in order to allow libcamera do correct timing
calculations, we should report pixel rate of 4*120e6 == 480e6, and
HBLANK of (4*HTS - H_OUT_SIZE). For 30 fps and VTS of 1554, this yields
HTS=2574 and HBLANK = (4*2574 - 2688) = 7608.

In fact, the above is what I'm going to implement in v3. Comments,
anyone?

> Just some thoughts.
> Dave
>
> [1] https://www.ovt.com/wp-content/uploads/2022/01/OV4689-PB-v1.7-WEB.pdf
> [2] https://github.com/bogsen/STLinux-Kernel/blob/master/drivers/media/platform/tegra/ov4689.c
>

--
Best regards,
Mikhail Rudenko