LinuxLists.cc - [PATCH RFC 0/4] media: v4l2-ctrls: add controls for complex lens controller devices

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael

Thanks for the patch.

I've got a personal interest here as I'd love to be able to control a
couple of CCTV lenses that I have. Those have standard motors with
potentiometers for position feedback, not stepper motors, but
otherwise have the same limitations of slow movement.

On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
libcamera-devel <[email protected]> wrote:
>
> From: Michael Riesch <[email protected]>
>
> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
> the status of the zoom lens group and the focus lens group, respectively.
> The returned data structure contains the current position of the lens group
> as well as movement indication flags.
>
> Signed-off-by: Michael Riesch <[email protected]>
> ---
> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
> include/media/v4l2-ctrls.h | 2 +
> include/uapi/linux/v4l2-controls.h | 13 ++++++
> include/uapi/linux/videodev2.h | 2 +
> 6 files changed, 81 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> index daa4f40869f8..3a270bc63f1a 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
> to the camera, negative values towards infinity. This is a
> write-only control.
>
> +``V4L2_CID_FOCUS_STATUS (struct)``
> + The current status of the focus lens group. This is a read-only control.
> + The returned data structure contains the current position and movement
> + indication flags. The unit of the current position is undefined. Positive
> + values move the focus closer to the camera, negative values towards
> + infinity. The possible flags are described in the table below.

The units are undefined, but positive and negative mean something with
respect to some unspecified focal distance represented by 0. That
seems a little odd.

I was going to suggest that it seems to make sense to follow the same
units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
[1] it's the same text.
I suspect there was a little too much copy/paste from
V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
brings the focus closer, and decreasing moves it towards infinity.

If we did specify that it was the same units as
V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
potentially it's redundant and could be deprecated.

[1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html

> +
> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> +
> + * - ``V4L2_LENS_STATUS_IDLE``
> + - Focus lens group is at rest.
> + * - ``V4L2_LENS_STATUS_BUSY``
> + - Focus lens group is moving.
> + * - ``V4L2_LENS_STATUS_REACHED``
> + - Focus lens group has reached its target position.
> + * - ``V4L2_LENS_STATUS_FAILED``
> + - Focus lens group has failed to reach its target position. The driver
> + will not transition from this state until another action is performed
> + by an application.

When would the lens state transition from V4L2_LENS_STATUS_REACHED to
V4L2_LENS_STATUS_IDLE?
If it's reached the position then it is at rest, and being at rest is
the definition of V4L2_LENS_STATUS_IDLE.
Likewise the lens always has a target position based on the control
value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
moving.
Is there a need to have 2 states?

If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
you leave the determination of state to the application?

> ``V4L2_CID_FOCUS_AUTO (boolean)``
> Enables continuous automatic focus adjustments. The effect of manual
> focus adjustments while this feature is enabled is undefined,
> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
> movement. A negative value moves the zoom lens group towards the
> wide-angle direction. The zoom speed unit is driver-specific.
>
> +``V4L2_CID_ZOOM_STATUS (struct)``
> + The current status of the zoom lens group. This is a read-only control.
> + The returned data structure contains the current position and movement
> + indication flags. The unit of the current position is driver-specific and
> + its value should be a positive integer. The possible flags are described
> + in the table below.
> +
> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> +
> + * - ``V4L2_LENS_STATUS_IDLE``
> + - Zoom lens group is at rest.
> + * - ``V4L2_LENS_STATUS_BUSY``
> + - Zoom lens group is moving.
> + * - ``V4L2_LENS_STATUS_REACHED``
> + - Zoom lens group has reached its target position.
> + * - ``V4L2_LENS_STATUS_FAILED``
> + - Zoom lens group has failed to reach its target position. The driver will
> + not transition from this state until another action is performed by an
> + application.
> +
> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
> This control sets the camera's aperture to the specified value. The
> unit is undefined. Larger values open the iris wider, smaller values
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> index 29169170880a..f6ad30f311c5 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
> pr_cont("HEVC_DECODE_PARAMS");
> break;
> + case V4L2_CTRL_TYPE_LENS_STATUS:
> + pr_cont("LENS_STATUS");
> + break;
> default:
> pr_cont("unknown type %d", ctrl->type);
> break;
> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
> return -EINVAL;
> break;
>
> + case V4L2_CTRL_TYPE_LENS_STATUS:
> + break;
> +
> default:
> return -EINVAL;
> }
> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_AREA:
> elem_size = sizeof(struct v4l2_area);
> break;
> + case V4L2_CTRL_TYPE_LENS_STATUS:
> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
> + break;
> default:
> if (type < V4L2_CTRL_COMPOUND_TYPES)
> elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 564fedee2c88..9b26a3aa9e9c 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";

Is there a need for the comma in the text strings?

Cheers
Dave

>
> /* FM Radio Modulator controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
> break;
> + case V4L2_CID_FOCUS_STATUS:
> + case V4L2_CID_ZOOM_STATUS:
> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
> + break;
> case V4L2_CID_FLASH_STROBE_STATUS:
> case V4L2_CID_AUTO_FOCUS_STATUS:
> case V4L2_CID_FLASH_READY:
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index e59d9a234631..f7273ffc20c9 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -52,6 +52,7 @@ struct video_device;
> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
> * @p_area: Pointer to an area.
> + * @p_lens_status: Pointer to a lens status structure.
> * @p: Pointer to a compound value.
> * @p_const: Pointer to a constant compound value.
> */
> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
> struct v4l2_area *p_area;
> + struct v4l2_ctrl_lens_status *p_lens_status;
> void *p;
> const void *p_const;
> };
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 5e80daa4ffe0..8b037467ba9a 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
>
> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
>
> +struct v4l2_ctrl_lens_status {
> + __u32 flags;
> + __s32 current_position;
> +};
> +
> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
> +
> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
> +
> /* FM Modulator class control IDs */
>
> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 17a9b975177a..256c21c68720 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
> +
> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
> };
>
> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>
> --
> 2.30.2
>

2023-04-11 17:39:38

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Dave,

On 4/6/23 17:16, Dave Stevenson wrote:
> Hi Michael
>
> Thanks for the patch.
>
> I've got a personal interest here as I'd love to be able to control a
> couple of CCTV lenses that I have. Those have standard motors with
> potentiometers for position feedback, not stepper motors, but
> otherwise have the same limitations of slow movement.

That's great to hear :-) Thank you for your feedback!

> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
> libcamera-devel <[email protected]> wrote:
>>
>> From: Michael Riesch <[email protected]>
>>
>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
>> the status of the zoom lens group and the focus lens group, respectively.
>> The returned data structure contains the current position of the lens group
>> as well as movement indication flags.
>>
>> Signed-off-by: Michael Riesch <[email protected]>
>> ---
>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
>> include/media/v4l2-ctrls.h | 2 +
>> include/uapi/linux/v4l2-controls.h | 13 ++++++
>> include/uapi/linux/videodev2.h | 2 +
>> 6 files changed, 81 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>> index daa4f40869f8..3a270bc63f1a 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
>> to the camera, negative values towards infinity. This is a
>> write-only control.
>>
>> +``V4L2_CID_FOCUS_STATUS (struct)``
>> + The current status of the focus lens group. This is a read-only control.
>> + The returned data structure contains the current position and movement
>> + indication flags. The unit of the current position is undefined. Positive
>> + values move the focus closer to the camera, negative values towards
>> + infinity. The possible flags are described in the table below.
>
> The units are undefined, but positive and negative mean something with
> respect to some unspecified focal distance represented by 0. That
> seems a little odd.
>
> I was going to suggest that it seems to make sense to follow the same
> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
> [1] it's the same text.
> I suspect there was a little too much copy/paste from
> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
> brings the focus closer, and decreasing moves it towards infinity.
>
> If we did specify that it was the same units as
> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
> potentially it's redundant and could be deprecated.
>
> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html

I think we agree that the _STATUS controls should use the same unit as
the corresponding _ABSOLUTE control, whatever that may be.

Then, the question is whether the description of FOCUS_ABSOLUTE needs to
be revised. I would agree that the intention probably was: "Larger
values move the focus closer to the camera, smaller values move the
focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE).

It would be interesting to know whether zero and negative values are
(intentionally?) included. Since they are not explicitly excluded, my
driver exposes the position of the lens in motor steps to the user
space. If the values were (supposed to be) restricted to positive values
like ZOOM_ABSOLUTE, this would not be allowed of course.

As to the relation to _RELATIVE, I think it should be clear that all
controls should act on the same scale and I don't see any problems here.
However, feel free to point out what I am missing :-)

>> +
>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>> +
>> +.. flat-table::
>> + :header-rows: 0
>> + :stub-columns: 0
>> +
>> + * - ``V4L2_LENS_STATUS_IDLE``
>> + - Focus lens group is at rest.
>> + * - ``V4L2_LENS_STATUS_BUSY``
>> + - Focus lens group is moving.
>> + * - ``V4L2_LENS_STATUS_REACHED``
>> + - Focus lens group has reached its target position.
>> + * - ``V4L2_LENS_STATUS_FAILED``
>> + - Focus lens group has failed to reach its target position. The driver
>> + will not transition from this state until another action is performed
>> + by an application.
>
> When would the lens state transition from V4L2_LENS_STATUS_REACHED to
> V4L2_LENS_STATUS_IDLE?
> If it's reached the position then it is at rest, and being at rest is
> the definition of V4L2_LENS_STATUS_IDLE.
> Likewise the lens always has a target position based on the control
> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
> moving.
> Is there a need to have 2 states?

Good point, I need to reconsider that (and possibly remove one of those
states).

> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
> you leave the determination of state to the application?

I am afraid I don't quite follow.

But FWIW the application should read out the flags to find out whether
the lens is moving and, if this is not the case, whether the last move
command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The
failed state is important as there might be the possibility that
depending on the zoom lens position certain focus positions are not
available (and vice versa).

>> ``V4L2_CID_FOCUS_AUTO (boolean)``
>> Enables continuous automatic focus adjustments. The effect of manual
>> focus adjustments while this feature is enabled is undefined,
>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
>> movement. A negative value moves the zoom lens group towards the
>> wide-angle direction. The zoom speed unit is driver-specific.
>>
>> +``V4L2_CID_ZOOM_STATUS (struct)``
>> + The current status of the zoom lens group. This is a read-only control.
>> + The returned data structure contains the current position and movement
>> + indication flags. The unit of the current position is driver-specific and
>> + its value should be a positive integer. The possible flags are described
>> + in the table below.
>> +
>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>> +
>> +.. flat-table::
>> + :header-rows: 0
>> + :stub-columns: 0
>> +
>> + * - ``V4L2_LENS_STATUS_IDLE``
>> + - Zoom lens group is at rest.
>> + * - ``V4L2_LENS_STATUS_BUSY``
>> + - Zoom lens group is moving.
>> + * - ``V4L2_LENS_STATUS_REACHED``
>> + - Zoom lens group has reached its target position.
>> + * - ``V4L2_LENS_STATUS_FAILED``
>> + - Zoom lens group has failed to reach its target position. The driver will
>> + not transition from this state until another action is performed by an
>> + application.
>> +
>> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
>> This control sets the camera's aperture to the specified value. The
>> unit is undefined. Larger values open the iris wider, smaller values
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>> index 29169170880a..f6ad30f311c5 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
>> pr_cont("HEVC_DECODE_PARAMS");
>> break;
>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>> + pr_cont("LENS_STATUS");
>> + break;
>> default:
>> pr_cont("unknown type %d", ctrl->type);
>> break;
>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
>> return -EINVAL;
>> break;
>>
>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>> + break;
>> +
>> default:
>> return -EINVAL;
>> }
>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>> case V4L2_CTRL_TYPE_AREA:
>> elem_size = sizeof(struct v4l2_area);
>> break;
>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
>> + break;
>> default:
>> if (type < V4L2_CTRL_COMPOUND_TYPES)
>> elem_size = sizeof(s32);
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> index 564fedee2c88..9b26a3aa9e9c 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";
>
> Is there a need for the comma in the text strings?

Not sure, actually. Some other strings used commas. Monkey see, monkey do.

Best regards,
Michael

>
> Cheers
> Dave
>
>>
>> /* FM Radio Modulator controls */
>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
>> break;
>> + case V4L2_CID_FOCUS_STATUS:
>> + case V4L2_CID_ZOOM_STATUS:
>> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
>> + break;
>> case V4L2_CID_FLASH_STROBE_STATUS:
>> case V4L2_CID_AUTO_FOCUS_STATUS:
>> case V4L2_CID_FLASH_READY:
>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>> index e59d9a234631..f7273ffc20c9 100644
>> --- a/include/media/v4l2-ctrls.h
>> +++ b/include/media/v4l2-ctrls.h
>> @@ -52,6 +52,7 @@ struct video_device;
>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
>> * @p_area: Pointer to an area.
>> + * @p_lens_status: Pointer to a lens status structure.
>> * @p: Pointer to a compound value.
>> * @p_const: Pointer to a constant compound value.
>> */
>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
>> struct v4l2_area *p_area;
>> + struct v4l2_ctrl_lens_status *p_lens_status;
>> void *p;
>> const void *p_const;
>> };
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index 5e80daa4ffe0..8b037467ba9a 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
>>
>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
>>
>> +struct v4l2_ctrl_lens_status {
>> + __u32 flags;
>> + __s32 current_position;
>> +};
>> +
>> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
>> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
>> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
>> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
>> +
>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
>> +
>> /* FM Modulator class control IDs */
>>
>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 17a9b975177a..256c21c68720 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
>> +
>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
>> };
>>
>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>>
>> --
>> 2.30.2
>>

2023-04-11 18:21:12

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael

On Tue, 11 Apr 2023 at 18:33, Michael Riesch
<[email protected]> wrote:
>
> Hi Dave,
>
> On 4/6/23 17:16, Dave Stevenson wrote:
> > Hi Michael
> >
> > Thanks for the patch.
> >
> > I've got a personal interest here as I'd love to be able to control a
> > couple of CCTV lenses that I have. Those have standard motors with
> > potentiometers for position feedback, not stepper motors, but
> > otherwise have the same limitations of slow movement.
>
> That's great to hear :-) Thank you for your feedback!
>
> > On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
> > libcamera-devel <[email protected]> wrote:
> >>
> >> From: Michael Riesch <[email protected]>
> >>
> >> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
> >> the status of the zoom lens group and the focus lens group, respectively.
> >> The returned data structure contains the current position of the lens group
> >> as well as movement indication flags.
> >>
> >> Signed-off-by: Michael Riesch <[email protected]>
> >> ---
> >> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
> >> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
> >> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
> >> include/media/v4l2-ctrls.h | 2 +
> >> include/uapi/linux/v4l2-controls.h | 13 ++++++
> >> include/uapi/linux/videodev2.h | 2 +
> >> 6 files changed, 81 insertions(+)
> >>
> >> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >> index daa4f40869f8..3a270bc63f1a 100644
> >> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
> >> to the camera, negative values towards infinity. This is a
> >> write-only control.
> >>
> >> +``V4L2_CID_FOCUS_STATUS (struct)``
> >> + The current status of the focus lens group. This is a read-only control.
> >> + The returned data structure contains the current position and movement
> >> + indication flags. The unit of the current position is undefined. Positive
> >> + values move the focus closer to the camera, negative values towards
> >> + infinity. The possible flags are described in the table below.
> >
> > The units are undefined, but positive and negative mean something with
> > respect to some unspecified focal distance represented by 0. That
> > seems a little odd.
> >
> > I was going to suggest that it seems to make sense to follow the same
> > units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
> > [1] it's the same text.
> > I suspect there was a little too much copy/paste from
> > V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
> > brings the focus closer, and decreasing moves it towards infinity.
> >
> > If we did specify that it was the same units as
> > V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
> > V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
> > to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
> > potentially it's redundant and could be deprecated.
> >
> > [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html
>
> I think we agree that the _STATUS controls should use the same unit as
> the corresponding _ABSOLUTE control, whatever that may be.
>
> Then, the question is whether the description of FOCUS_ABSOLUTE needs to
> be revised. I would agree that the intention probably was: "Larger
> values move the focus closer to the camera, smaller values move the
> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE).
>
> It would be interesting to know whether zero and negative values are
> (intentionally?) included. Since they are not explicitly excluded, my
> driver exposes the position of the lens in motor steps to the user
> space. If the values were (supposed to be) restricted to positive values
> like ZOOM_ABSOLUTE, this would not be allowed of course.
>
> As to the relation to _RELATIVE, I think it should be clear that all
> controls should act on the same scale and I don't see any problems here.
> However, feel free to point out what I am missing :-)
>
> >> +
> >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> >> +
> >> +.. flat-table::
> >> + :header-rows: 0
> >> + :stub-columns: 0
> >> +
> >> + * - ``V4L2_LENS_STATUS_IDLE``
> >> + - Focus lens group is at rest.
> >> + * - ``V4L2_LENS_STATUS_BUSY``
> >> + - Focus lens group is moving.
> >> + * - ``V4L2_LENS_STATUS_REACHED``
> >> + - Focus lens group has reached its target position.
> >> + * - ``V4L2_LENS_STATUS_FAILED``
> >> + - Focus lens group has failed to reach its target position. The driver
> >> + will not transition from this state until another action is performed
> >> + by an application.
> >
> > When would the lens state transition from V4L2_LENS_STATUS_REACHED to
> > V4L2_LENS_STATUS_IDLE?
> > If it's reached the position then it is at rest, and being at rest is
> > the definition of V4L2_LENS_STATUS_IDLE.
> > Likewise the lens always has a target position based on the control
> > value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
> > moving.
> > Is there a need to have 2 states?
>
> Good point, I need to reconsider that (and possibly remove one of those
> states).
>
> > If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
> > you leave the determination of state to the application?
>
> I am afraid I don't quite follow.

If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and
V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the
application knows that the lens has reached the requested position. If
the two controls return different values then the lens is still
moving.
What new information does adding an additional status flag give you?

(Thinking aloud) I guess you have got my case where your readback is
via an ADC so that the position may fluctuate slightly due to
conversion noise. The control loop for applying power to the motors
will presumably stop at some point and stop trying to adjust the
position, so potentially it could be the state returned from that
control loop. However the noisy ADC position could also be solved by
the position being returned by the control loop instead of giving the
raw value.

> But FWIW the application should read out the flags to find out whether
> the lens is moving and, if this is not the case, whether the last move
> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The
> failed state is important as there might be the possibility that
> depending on the zoom lens position certain focus positions are not
> available (and vice versa).

How would you configure those kinds of restrictions in the kernel
driver? Is the kernel driver the best place to do it (so many things
are being kicked back to userspace these days)?
If it is to be done in the kernel, then shouldn't eg changing the zoom
position alter the ranges advertised for the relevant focus controls?
If the ranges aren't updated, where should that out-of-range lens
movement leave the lens?

Dave

> >> ``V4L2_CID_FOCUS_AUTO (boolean)``
> >> Enables continuous automatic focus adjustments. The effect of manual
> >> focus adjustments while this feature is enabled is undefined,
> >> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
> >> movement. A negative value moves the zoom lens group towards the
> >> wide-angle direction. The zoom speed unit is driver-specific.
> >>
> >> +``V4L2_CID_ZOOM_STATUS (struct)``
> >> + The current status of the zoom lens group. This is a read-only control.
> >> + The returned data structure contains the current position and movement
> >> + indication flags. The unit of the current position is driver-specific and
> >> + its value should be a positive integer. The possible flags are described
> >> + in the table below.
> >> +
> >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> >> +
> >> +.. flat-table::
> >> + :header-rows: 0
> >> + :stub-columns: 0
> >> +
> >> + * - ``V4L2_LENS_STATUS_IDLE``
> >> + - Zoom lens group is at rest.
> >> + * - ``V4L2_LENS_STATUS_BUSY``
> >> + - Zoom lens group is moving.
> >> + * - ``V4L2_LENS_STATUS_REACHED``
> >> + - Zoom lens group has reached its target position.
> >> + * - ``V4L2_LENS_STATUS_FAILED``
> >> + - Zoom lens group has failed to reach its target position. The driver will
> >> + not transition from this state until another action is performed by an
> >> + application.
> >> +
> >> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
> >> This control sets the camera's aperture to the specified value. The
> >> unit is undefined. Larger values open the iris wider, smaller values
> >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >> index 29169170880a..f6ad30f311c5 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
> >> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
> >> pr_cont("HEVC_DECODE_PARAMS");
> >> break;
> >> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >> + pr_cont("LENS_STATUS");
> >> + break;
> >> default:
> >> pr_cont("unknown type %d", ctrl->type);
> >> break;
> >> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
> >> return -EINVAL;
> >> break;
> >>
> >> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >> + break;
> >> +
> >> default:
> >> return -EINVAL;
> >> }
> >> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> >> case V4L2_CTRL_TYPE_AREA:
> >> elem_size = sizeof(struct v4l2_area);
> >> break;
> >> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
> >> + break;
> >> default:
> >> if (type < V4L2_CTRL_COMPOUND_TYPES)
> >> elem_size = sizeof(s32);
> >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >> index 564fedee2c88..9b26a3aa9e9c 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
> >> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
> >> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
> >> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
> >> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
> >> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";
> >
> > Is there a need for the comma in the text strings?
>
> Not sure, actually. Some other strings used commas. Monkey see, monkey do.
>
> Best regards,
> Michael
>
> >
> > Cheers
> > Dave
> >
> >>
> >> /* FM Radio Modulator controls */
> >> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> >> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> >> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
> >> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
> >> break;
> >> + case V4L2_CID_FOCUS_STATUS:
> >> + case V4L2_CID_ZOOM_STATUS:
> >> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
> >> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
> >> + break;
> >> case V4L2_CID_FLASH_STROBE_STATUS:
> >> case V4L2_CID_AUTO_FOCUS_STATUS:
> >> case V4L2_CID_FLASH_READY:
> >> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> >> index e59d9a234631..f7273ffc20c9 100644
> >> --- a/include/media/v4l2-ctrls.h
> >> +++ b/include/media/v4l2-ctrls.h
> >> @@ -52,6 +52,7 @@ struct video_device;
> >> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
> >> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
> >> * @p_area: Pointer to an area.
> >> + * @p_lens_status: Pointer to a lens status structure.
> >> * @p: Pointer to a compound value.
> >> * @p_const: Pointer to a constant compound value.
> >> */
> >> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
> >> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
> >> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
> >> struct v4l2_area *p_area;
> >> + struct v4l2_ctrl_lens_status *p_lens_status;
> >> void *p;
> >> const void *p_const;
> >> };
> >> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> >> index 5e80daa4ffe0..8b037467ba9a 100644
> >> --- a/include/uapi/linux/v4l2-controls.h
> >> +++ b/include/uapi/linux/v4l2-controls.h
> >> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
> >>
> >> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
> >>
> >> +struct v4l2_ctrl_lens_status {
> >> + __u32 flags;
> >> + __s32 current_position;
> >> +};
> >> +
> >> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
> >> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
> >> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
> >> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
> >> +
> >> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
> >> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
> >> +
> >> /* FM Modulator class control IDs */
> >>
> >> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
> >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >> index 17a9b975177a..256c21c68720 100644
> >> --- a/include/uapi/linux/videodev2.h
> >> +++ b/include/uapi/linux/videodev2.h
> >> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
> >> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
> >> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
> >> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
> >> +
> >> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
> >> };
> >>
> >> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> >>
> >> --
> >> 2.30.2
> >>

2023-04-12 08:06:54

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Dave,

On 4/11/23 20:15, Dave Stevenson wrote:
> Hi Michael
>
> On Tue, 11 Apr 2023 at 18:33, Michael Riesch
> <[email protected]> wrote:
>>
>> Hi Dave,
>>
>> On 4/6/23 17:16, Dave Stevenson wrote:
>>> Hi Michael
>>>
>>> Thanks for the patch.
>>>
>>> I've got a personal interest here as I'd love to be able to control a
>>> couple of CCTV lenses that I have. Those have standard motors with
>>> potentiometers for position feedback, not stepper motors, but
>>> otherwise have the same limitations of slow movement.
>>
>> That's great to hear :-) Thank you for your feedback!
>>
>>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
>>> libcamera-devel <[email protected]> wrote:
>>>>
>>>> From: Michael Riesch <[email protected]>
>>>>
>>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
>>>> the status of the zoom lens group and the focus lens group, respectively.
>>>> The returned data structure contains the current position of the lens group
>>>> as well as movement indication flags.
>>>>
>>>> Signed-off-by: Michael Riesch <[email protected]>
>>>> ---
>>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
>>>> include/media/v4l2-ctrls.h | 2 +
>>>> include/uapi/linux/v4l2-controls.h | 13 ++++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 6 files changed, 81 insertions(+)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>> index daa4f40869f8..3a270bc63f1a 100644
>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
>>>> to the camera, negative values towards infinity. This is a
>>>> write-only control.
>>>>
>>>> +``V4L2_CID_FOCUS_STATUS (struct)``
>>>> + The current status of the focus lens group. This is a read-only control.
>>>> + The returned data structure contains the current position and movement
>>>> + indication flags. The unit of the current position is undefined. Positive
>>>> + values move the focus closer to the camera, negative values towards
>>>> + infinity. The possible flags are described in the table below.
>>>
>>> The units are undefined, but positive and negative mean something with
>>> respect to some unspecified focal distance represented by 0. That
>>> seems a little odd.
>>>
>>> I was going to suggest that it seems to make sense to follow the same
>>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
>>> [1] it's the same text.
>>> I suspect there was a little too much copy/paste from
>>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
>>> brings the focus closer, and decreasing moves it towards infinity.
>>>
>>> If we did specify that it was the same units as
>>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
>>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
>>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
>>> potentially it's redundant and could be deprecated.
>>>
>>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html
>>
>> I think we agree that the _STATUS controls should use the same unit as
>> the corresponding _ABSOLUTE control, whatever that may be.
>>
>> Then, the question is whether the description of FOCUS_ABSOLUTE needs to
>> be revised. I would agree that the intention probably was: "Larger
>> values move the focus closer to the camera, smaller values move the
>> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE).
>>
>> It would be interesting to know whether zero and negative values are
>> (intentionally?) included. Since they are not explicitly excluded, my
>> driver exposes the position of the lens in motor steps to the user
>> space. If the values were (supposed to be) restricted to positive values
>> like ZOOM_ABSOLUTE, this would not be allowed of course.
>>
>> As to the relation to _RELATIVE, I think it should be clear that all
>> controls should act on the same scale and I don't see any problems here.
>> However, feel free to point out what I am missing :-)
>>
>>>> +
>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>>>> +
>>>> +.. flat-table::
>>>> + :header-rows: 0
>>>> + :stub-columns: 0
>>>> +
>>>> + * - ``V4L2_LENS_STATUS_IDLE``
>>>> + - Focus lens group is at rest.
>>>> + * - ``V4L2_LENS_STATUS_BUSY``
>>>> + - Focus lens group is moving.
>>>> + * - ``V4L2_LENS_STATUS_REACHED``
>>>> + - Focus lens group has reached its target position.
>>>> + * - ``V4L2_LENS_STATUS_FAILED``
>>>> + - Focus lens group has failed to reach its target position. The driver
>>>> + will not transition from this state until another action is performed
>>>> + by an application.
>>>
>>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to
>>> V4L2_LENS_STATUS_IDLE?
>>> If it's reached the position then it is at rest, and being at rest is
>>> the definition of V4L2_LENS_STATUS_IDLE.
>>> Likewise the lens always has a target position based on the control
>>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
>>> moving.
>>> Is there a need to have 2 states?
>>
>> Good point, I need to reconsider that (and possibly remove one of those
>> states).
>>
>>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
>>> you leave the determination of state to the application?
>>
>> I am afraid I don't quite follow.
>
> If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and
> V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the
> application knows that the lens has reached the requested position. If
> the two controls return different values then the lens is still
> moving.
> What new information does adding an additional status flag give you?

I can think of two situations:

- Hardware error: the lens stops at 40 (for whatever reason). Since 40
!= 42 the lens is still moving, although a hardware error has occured
that may need recovery or at least reporting.

- Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current),
then what happens if the application performs a
V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39,
V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from
the application's point of view.

Don't get me wrong, a simple control that reports the current position
would be way easier. But I feel that we may regret this in the future.
Also, I would not use two separate controls "current" and "flags/status"
in order to avoid non-atomic access patterns.

> (Thinking aloud) I guess you have got my case where your readback is
> via an ADC so that the position may fluctuate slightly due to
> conversion noise. The control loop for applying power to the motors
> will presumably stop at some point and stop trying to adjust the
> position, so potentially it could be the state returned from that
> control loop. However the noisy ADC position could also be solved by
> the position being returned by the control loop instead of giving the
> raw value.

With stepper motors the readback is not that noisy, but in any case the
flags field allows an underlying controller to signal certain conditions
without relying on the exact value of the current position. I think that
is a plus, especially for noisy readback.

>> But FWIW the application should read out the flags to find out whether
>> the lens is moving and, if this is not the case, whether the last move
>> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The
>> failed state is important as there might be the possibility that
>> depending on the zoom lens position certain focus positions are not
>> available (and vice versa).
>
> How would you configure those kinds of restrictions in the kernel
> driver? Is the kernel driver the best place to do it (so many things
> are being kicked back to userspace these days)?

In our case we have a hardware controller that needs to handle the
restrictions anyway in order to avoid mechanical damage. The kernel
driver is only responsible for reading out the controller status and
transforming it into the V4L2 control.

I am not sure how any restrictions can be reliably handled in software,
hence I don't have a strong opinion on whether this is done in kernel or
user space.

> If it is to be done in the kernel, then shouldn't eg changing the zoom
> position alter the ranges advertised for the relevant focus controls?

While it should be possible to update the minimum and maximum of e.g.,
FOCUS_ABSOLUTE, I am not sure whether the interface is designed for
frequent re-reading of the range.

> If the ranges aren't updated, where should that out-of-range lens
> movement leave the lens?

This is up to the hardware controller, but I would guess it typically
stops one step before disaster. Wherever that may be, the error
condition and the current position can be read out via this new STATUS
control.

Does this sound good so far?

Best regards,
Michael

>
> Dave
>
>>>> ``V4L2_CID_FOCUS_AUTO (boolean)``
>>>> Enables continuous automatic focus adjustments. The effect of manual
>>>> focus adjustments while this feature is enabled is undefined,
>>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
>>>> movement. A negative value moves the zoom lens group towards the
>>>> wide-angle direction. The zoom speed unit is driver-specific.
>>>>
>>>> +``V4L2_CID_ZOOM_STATUS (struct)``
>>>> + The current status of the zoom lens group. This is a read-only control.
>>>> + The returned data structure contains the current position and movement
>>>> + indication flags. The unit of the current position is driver-specific and
>>>> + its value should be a positive integer. The possible flags are described
>>>> + in the table below.
>>>> +
>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>>>> +
>>>> +.. flat-table::
>>>> + :header-rows: 0
>>>> + :stub-columns: 0
>>>> +
>>>> + * - ``V4L2_LENS_STATUS_IDLE``
>>>> + - Zoom lens group is at rest.
>>>> + * - ``V4L2_LENS_STATUS_BUSY``
>>>> + - Zoom lens group is moving.
>>>> + * - ``V4L2_LENS_STATUS_REACHED``
>>>> + - Zoom lens group has reached its target position.
>>>> + * - ``V4L2_LENS_STATUS_FAILED``
>>>> + - Zoom lens group has failed to reach its target position. The driver will
>>>> + not transition from this state until another action is performed by an
>>>> + application.
>>>> +
>>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
>>>> This control sets the camera's aperture to the specified value. The
>>>> unit is undefined. Larger values open the iris wider, smaller values
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>> index 29169170880a..f6ad30f311c5 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
>>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
>>>> pr_cont("HEVC_DECODE_PARAMS");
>>>> break;
>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>> + pr_cont("LENS_STATUS");
>>>> + break;
>>>> default:
>>>> pr_cont("unknown type %d", ctrl->type);
>>>> break;
>>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
>>>> return -EINVAL;
>>>> break;
>>>>
>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>> + break;
>>>> +
>>>> default:
>>>> return -EINVAL;
>>>> }
>>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>>>> case V4L2_CTRL_TYPE_AREA:
>>>> elem_size = sizeof(struct v4l2_area);
>>>> break;
>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
>>>> + break;
>>>> default:
>>>> if (type < V4L2_CTRL_COMPOUND_TYPES)
>>>> elem_size = sizeof(s32);
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>> index 564fedee2c88..9b26a3aa9e9c 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
>>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
>>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
>>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
>>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
>>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";
>>>
>>> Is there a need for the comma in the text strings?
>>
>> Not sure, actually. Some other strings used commas. Monkey see, monkey do.
>>
>> Best regards,
>> Michael
>>
>>>
>>> Cheers
>>> Dave
>>>
>>>>
>>>> /* FM Radio Modulator controls */
>>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
>>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
>>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
>>>> break;
>>>> + case V4L2_CID_FOCUS_STATUS:
>>>> + case V4L2_CID_ZOOM_STATUS:
>>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
>>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
>>>> + break;
>>>> case V4L2_CID_FLASH_STROBE_STATUS:
>>>> case V4L2_CID_AUTO_FOCUS_STATUS:
>>>> case V4L2_CID_FLASH_READY:
>>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>>>> index e59d9a234631..f7273ffc20c9 100644
>>>> --- a/include/media/v4l2-ctrls.h
>>>> +++ b/include/media/v4l2-ctrls.h
>>>> @@ -52,6 +52,7 @@ struct video_device;
>>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
>>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
>>>> * @p_area: Pointer to an area.
>>>> + * @p_lens_status: Pointer to a lens status structure.
>>>> * @p: Pointer to a compound value.
>>>> * @p_const: Pointer to a constant compound value.
>>>> */
>>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
>>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
>>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
>>>> struct v4l2_area *p_area;
>>>> + struct v4l2_ctrl_lens_status *p_lens_status;
>>>> void *p;
>>>> const void *p_const;
>>>> };
>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>>> index 5e80daa4ffe0..8b037467ba9a 100644
>>>> --- a/include/uapi/linux/v4l2-controls.h
>>>> +++ b/include/uapi/linux/v4l2-controls.h
>>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
>>>>
>>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
>>>>
>>>> +struct v4l2_ctrl_lens_status {
>>>> + __u32 flags;
>>>> + __s32 current_position;
>>>> +};
>>>> +
>>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
>>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
>>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
>>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
>>>> +
>>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
>>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
>>>> +
>>>> /* FM Modulator class control IDs */
>>>>
>>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index 17a9b975177a..256c21c68720 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
>>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
>>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
>>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
>>>> +
>>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
>>>> };
>>>>
>>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>>>>
>>>> --
>>>> 2.30.2
>>>>

2023-04-12 11:53:23

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael,

On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote:
> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current),
> then what happens if the application performs a
> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39,
> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from
> the application's point of view.

Would there be a reason to implement both of these controls in a single
driver? AFAIU, the relative one should be used if there absolute value
isn't known to the driver.

--
Kind regards,

Sakari Ailus

2023-04-12 12:25:22

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Sakari,

On 4/12/23 13:50, Sakari Ailus wrote:
> Hi Michael,
>
> On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote:
>> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current),
>> then what happens if the application performs a
>> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39,
>> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from
>> the application's point of view.
>
> Would there be a reason to implement both of these controls in a single
> driver? AFAIU, the relative one should be used if there absolute value
> isn't known to the driver.

Probably not, but on the other hand there is nothing the prevents a
driver developer from doing so, right? Point is that should there be a
driver which does implement both controls, we are in trouble AFAIU.

Best regards,
Michael

2023-04-12 14:07:45

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael

On Wed, 12 Apr 2023 at 09:00, Michael Riesch
<[email protected]> wrote:
>
> Hi Dave,
>
> On 4/11/23 20:15, Dave Stevenson wrote:
> > Hi Michael
> >
> > On Tue, 11 Apr 2023 at 18:33, Michael Riesch
> > <[email protected]> wrote:
> >>
> >> Hi Dave,
> >>
> >> On 4/6/23 17:16, Dave Stevenson wrote:
> >>> Hi Michael
> >>>
> >>> Thanks for the patch.
> >>>
> >>> I've got a personal interest here as I'd love to be able to control a
> >>> couple of CCTV lenses that I have. Those have standard motors with
> >>> potentiometers for position feedback, not stepper motors, but
> >>> otherwise have the same limitations of slow movement.
> >>
> >> That's great to hear :-) Thank you for your feedback!
> >>
> >>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
> >>> libcamera-devel <[email protected]> wrote:
> >>>>
> >>>> From: Michael Riesch <[email protected]>
> >>>>
> >>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
> >>>> the status of the zoom lens group and the focus lens group, respectively.
> >>>> The returned data structure contains the current position of the lens group
> >>>> as well as movement indication flags.
> >>>>
> >>>> Signed-off-by: Michael Riesch <[email protected]>
> >>>> ---
> >>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
> >>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
> >>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
> >>>> include/media/v4l2-ctrls.h | 2 +
> >>>> include/uapi/linux/v4l2-controls.h | 13 ++++++
> >>>> include/uapi/linux/videodev2.h | 2 +
> >>>> 6 files changed, 81 insertions(+)
> >>>>
> >>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >>>> index daa4f40869f8..3a270bc63f1a 100644
> >>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> >>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
> >>>> to the camera, negative values towards infinity. This is a
> >>>> write-only control.
> >>>>
> >>>> +``V4L2_CID_FOCUS_STATUS (struct)``
> >>>> + The current status of the focus lens group. This is a read-only control.
> >>>> + The returned data structure contains the current position and movement
> >>>> + indication flags. The unit of the current position is undefined. Positive
> >>>> + values move the focus closer to the camera, negative values towards
> >>>> + infinity. The possible flags are described in the table below.
> >>>
> >>> The units are undefined, but positive and negative mean something with
> >>> respect to some unspecified focal distance represented by 0. That
> >>> seems a little odd.
> >>>
> >>> I was going to suggest that it seems to make sense to follow the same
> >>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
> >>> [1] it's the same text.
> >>> I suspect there was a little too much copy/paste from
> >>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
> >>> brings the focus closer, and decreasing moves it towards infinity.
> >>>
> >>> If we did specify that it was the same units as
> >>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
> >>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
> >>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
> >>> potentially it's redundant and could be deprecated.
> >>>
> >>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html
> >>
> >> I think we agree that the _STATUS controls should use the same unit as
> >> the corresponding _ABSOLUTE control, whatever that may be.
> >>
> >> Then, the question is whether the description of FOCUS_ABSOLUTE needs to
> >> be revised. I would agree that the intention probably was: "Larger
> >> values move the focus closer to the camera, smaller values move the
> >> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE).
> >>
> >> It would be interesting to know whether zero and negative values are
> >> (intentionally?) included. Since they are not explicitly excluded, my
> >> driver exposes the position of the lens in motor steps to the user
> >> space. If the values were (supposed to be) restricted to positive values
> >> like ZOOM_ABSOLUTE, this would not be allowed of course.
> >>
> >> As to the relation to _RELATIVE, I think it should be clear that all
> >> controls should act on the same scale and I don't see any problems here.
> >> However, feel free to point out what I am missing :-)
> >>
> >>>> +
> >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> >>>> +
> >>>> +.. flat-table::
> >>>> + :header-rows: 0
> >>>> + :stub-columns: 0
> >>>> +
> >>>> + * - ``V4L2_LENS_STATUS_IDLE``
> >>>> + - Focus lens group is at rest.
> >>>> + * - ``V4L2_LENS_STATUS_BUSY``
> >>>> + - Focus lens group is moving.
> >>>> + * - ``V4L2_LENS_STATUS_REACHED``
> >>>> + - Focus lens group has reached its target position.
> >>>> + * - ``V4L2_LENS_STATUS_FAILED``
> >>>> + - Focus lens group has failed to reach its target position. The driver
> >>>> + will not transition from this state until another action is performed
> >>>> + by an application.
> >>>
> >>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to
> >>> V4L2_LENS_STATUS_IDLE?
> >>> If it's reached the position then it is at rest, and being at rest is
> >>> the definition of V4L2_LENS_STATUS_IDLE.
> >>> Likewise the lens always has a target position based on the control
> >>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
> >>> moving.
> >>> Is there a need to have 2 states?
> >>
> >> Good point, I need to reconsider that (and possibly remove one of those
> >> states).
> >>
> >>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
> >>> you leave the determination of state to the application?
> >>
> >> I am afraid I don't quite follow.
> >
> > If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and
> > V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the
> > application knows that the lens has reached the requested position. If
> > the two controls return different values then the lens is still
> > moving.
> > What new information does adding an additional status flag give you?
>
> I can think of two situations:
>
> - Hardware error: the lens stops at 40 (for whatever reason). Since 40
> != 42 the lens is still moving, although a hardware error has occured
> that may need recovery or at least reporting.

Are you thinking of the lens drive having totally failed, or just that
the requested position is out of the achievable range?

Looking at the flash controls[1], hardware failures are handled by
V4L2_CID_FLASH_FAULT as a separate control.

[1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-flash.html

> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current),
> then what happens if the application performs a
> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39,
> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from
> the application's point of view.

V4L2_CID_FOCUS_RELATIVE really ought to die IMHO, or at least work in
co-operation with V4L2_CID_FOCUS_ABSOLUTE such that the absolute
current position can always be retrieved.

Using _RELATIVE to set a position that is outside of the range defined
by _ABSOLUTE is meaningless. Any call to S_CTRL on _RELATIVE almost
has to be followed by a call to G_CTRL _ABSOLUTE to know the final
result, so why not just make a S_CTRL _ABSOLUTE call in the first
place.

I guess _RELATIVE may have uses in the case where there is no
read-back of the position at all, but in that case I don't see a way
that your new controls could be implemented.

> Don't get me wrong, a simple control that reports the current position
> would be way easier. But I feel that we may regret this in the future.
> Also, I would not use two separate controls "current" and "flags/status"
> in order to avoid non-atomic access patterns.

I have no fixed views, but overcomplicating things is generally bad news.

> > (Thinking aloud) I guess you have got my case where your readback is
> > via an ADC so that the position may fluctuate slightly due to
> > conversion noise. The control loop for applying power to the motors
> > will presumably stop at some point and stop trying to adjust the
> > position, so potentially it could be the state returned from that
> > control loop. However the noisy ADC position could also be solved by
> > the position being returned by the control loop instead of giving the
> > raw value.
>
> With stepper motors the readback is not that noisy, but in any case the
> flags field allows an underlying controller to signal certain conditions
> without relying on the exact value of the current position. I think that
> is a plus, especially for noisy readback.
>
> >> But FWIW the application should read out the flags to find out whether
> >> the lens is moving and, if this is not the case, whether the last move
> >> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The
> >> failed state is important as there might be the possibility that
> >> depending on the zoom lens position certain focus positions are not
> >> available (and vice versa).
> >
> > How would you configure those kinds of restrictions in the kernel
> > driver? Is the kernel driver the best place to do it (so many things
> > are being kicked back to userspace these days)?
>
> In our case we have a hardware controller that needs to handle the
> restrictions anyway in order to avoid mechanical damage. The kernel
> driver is only responsible for reading out the controller status and
> transforming it into the V4L2 control.

How does userspace know what the valid ranges are? Do you end up with
the same restrictions having to be coded in both your controller and
userspace, or are you relying on the driver reporting back this error
status with userspace not knowing by how much it got it wrong by?

> I am not sure how any restrictions can be reliably handled in software,
> hence I don't have a strong opinion on whether this is done in kernel or
> user space.
>
> > If it is to be done in the kernel, then shouldn't eg changing the zoom
> > position alter the ranges advertised for the relevant focus controls?
>
> While it should be possible to update the minimum and maximum of e.g.,
> FOCUS_ABSOLUTE, I am not sure whether the interface is designed for
> frequent re-reading of the range.

There's nothing in there that stops you, and several hooks to help you.
Updating V4L2_CID_VBLANK on most raw sensors will update the range on
V4L2_CID_EXPOSURE, as the exposure period can't be longer than the
frame period. It would be similar that each change to ZOOM_ABSOLUTE
would result in a range adjustment to FOCUS_ABSOLUTE.
Userspace can also use VIDIOC_SUBSCRIBE_EVENT to subscribe to any
updates in controls, so it gets told automatically of any updates.

> > If the ranges aren't updated, where should that out-of-range lens
> > movement leave the lens?
>
> This is up to the hardware controller, but I would guess it typically
> stops one step before disaster. Wherever that may be, the error
> condition and the current position can be read out via this new STATUS
> control.
>
> Does this sound good so far?

Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
Laurent), and I'm just expressing my views based on the lenses I've
encountered.
All of my lenses have a single drive for focus, a single drive for
zoom, and where there are multiple elements they are all connected
mechanically. Your setup sounds far more complex and is likely to need
a more extensive driver, but it'd be nice to not unnecessarily
overcomplicate the interface.

Dave

> Best regards,
> Michael
>
> >
> > Dave
> >
> >>>> ``V4L2_CID_FOCUS_AUTO (boolean)``
> >>>> Enables continuous automatic focus adjustments. The effect of manual
> >>>> focus adjustments while this feature is enabled is undefined,
> >>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
> >>>> movement. A negative value moves the zoom lens group towards the
> >>>> wide-angle direction. The zoom speed unit is driver-specific.
> >>>>
> >>>> +``V4L2_CID_ZOOM_STATUS (struct)``
> >>>> + The current status of the zoom lens group. This is a read-only control.
> >>>> + The returned data structure contains the current position and movement
> >>>> + indication flags. The unit of the current position is driver-specific and
> >>>> + its value should be a positive integer. The possible flags are described
> >>>> + in the table below.
> >>>> +
> >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
> >>>> +
> >>>> +.. flat-table::
> >>>> + :header-rows: 0
> >>>> + :stub-columns: 0
> >>>> +
> >>>> + * - ``V4L2_LENS_STATUS_IDLE``
> >>>> + - Zoom lens group is at rest.
> >>>> + * - ``V4L2_LENS_STATUS_BUSY``
> >>>> + - Zoom lens group is moving.
> >>>> + * - ``V4L2_LENS_STATUS_REACHED``
> >>>> + - Zoom lens group has reached its target position.
> >>>> + * - ``V4L2_LENS_STATUS_FAILED``
> >>>> + - Zoom lens group has failed to reach its target position. The driver will
> >>>> + not transition from this state until another action is performed by an
> >>>> + application.
> >>>> +
> >>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
> >>>> This control sets the camera's aperture to the specified value. The
> >>>> unit is undefined. Larger values open the iris wider, smaller values
> >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >>>> index 29169170880a..f6ad30f311c5 100644
> >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
> >>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
> >>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
> >>>> pr_cont("HEVC_DECODE_PARAMS");
> >>>> break;
> >>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >>>> + pr_cont("LENS_STATUS");
> >>>> + break;
> >>>> default:
> >>>> pr_cont("unknown type %d", ctrl->type);
> >>>> break;
> >>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
> >>>> return -EINVAL;
> >>>> break;
> >>>>
> >>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >>>> + break;
> >>>> +
> >>>> default:
> >>>> return -EINVAL;
> >>>> }
> >>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> >>>> case V4L2_CTRL_TYPE_AREA:
> >>>> elem_size = sizeof(struct v4l2_area);
> >>>> break;
> >>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
> >>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
> >>>> + break;
> >>>> default:
> >>>> if (type < V4L2_CTRL_COMPOUND_TYPES)
> >>>> elem_size = sizeof(s32);
> >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>>> index 564fedee2c88..9b26a3aa9e9c 100644
> >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
> >>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
> >>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
> >>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
> >>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
> >>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";
> >>>
> >>> Is there a need for the comma in the text strings?
> >>
> >> Not sure, actually. Some other strings used commas. Monkey see, monkey do.
> >>
> >> Best regards,
> >> Michael
> >>
> >>>
> >>> Cheers
> >>> Dave
> >>>
> >>>>
> >>>> /* FM Radio Modulator controls */
> >>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> >>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> >>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
> >>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
> >>>> break;
> >>>> + case V4L2_CID_FOCUS_STATUS:
> >>>> + case V4L2_CID_ZOOM_STATUS:
> >>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
> >>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
> >>>> + break;
> >>>> case V4L2_CID_FLASH_STROBE_STATUS:
> >>>> case V4L2_CID_AUTO_FOCUS_STATUS:
> >>>> case V4L2_CID_FLASH_READY:
> >>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> >>>> index e59d9a234631..f7273ffc20c9 100644
> >>>> --- a/include/media/v4l2-ctrls.h
> >>>> +++ b/include/media/v4l2-ctrls.h
> >>>> @@ -52,6 +52,7 @@ struct video_device;
> >>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
> >>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
> >>>> * @p_area: Pointer to an area.
> >>>> + * @p_lens_status: Pointer to a lens status structure.
> >>>> * @p: Pointer to a compound value.
> >>>> * @p_const: Pointer to a constant compound value.
> >>>> */
> >>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
> >>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
> >>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
> >>>> struct v4l2_area *p_area;
> >>>> + struct v4l2_ctrl_lens_status *p_lens_status;
> >>>> void *p;
> >>>> const void *p_const;
> >>>> };
> >>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> >>>> index 5e80daa4ffe0..8b037467ba9a 100644
> >>>> --- a/include/uapi/linux/v4l2-controls.h
> >>>> +++ b/include/uapi/linux/v4l2-controls.h
> >>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
> >>>>
> >>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
> >>>>
> >>>> +struct v4l2_ctrl_lens_status {
> >>>> + __u32 flags;
> >>>> + __s32 current_position;
> >>>> +};
> >>>> +
> >>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
> >>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
> >>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
> >>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
> >>>> +
> >>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
> >>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
> >>>> +
> >>>> /* FM Modulator class control IDs */
> >>>>
> >>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
> >>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >>>> index 17a9b975177a..256c21c68720 100644
> >>>> --- a/include/uapi/linux/videodev2.h
> >>>> +++ b/include/uapi/linux/videodev2.h
> >>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
> >>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
> >>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
> >>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
> >>>> +
> >>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
> >>>> };
> >>>>
> >>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> >>>>
> >>>> --
> >>>> 2.30.2
> >>>>

2023-04-12 15:17:54

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Dave, Michael,

On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote:
> > > If the ranges aren't updated, where should that out-of-range lens
> > > movement leave the lens?
> >
> > This is up to the hardware controller, but I would guess it typically
> > stops one step before disaster. Wherever that may be, the error
> > condition and the current position can be read out via this new STATUS
> > control.
> >
> > Does this sound good so far?
>
> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
> Laurent), and I'm just expressing my views based on the lenses I've
> encountered.
> All of my lenses have a single drive for focus, a single drive for
> zoom, and where there are multiple elements they are all connected
> mechanically. Your setup sounds far more complex and is likely to need
> a more extensive driver, but it'd be nice to not unnecessarily
> overcomplicate the interface.

Could we also have a driver that uses these new controls?

The controls themselves appear reasonable to me as well. I guess there are
changes to be made based on the discussion?

--
Regards,

Sakari Ailus

2023-04-17 08:39:17

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Dave,

On 4/12/23 15:55, Dave Stevenson wrote:
> Hi Michael
>
> On Wed, 12 Apr 2023 at 09:00, Michael Riesch
> <[email protected]> wrote:
>>
>> Hi Dave,
>>
>> On 4/11/23 20:15, Dave Stevenson wrote:
>>> Hi Michael
>>>
>>> On Tue, 11 Apr 2023 at 18:33, Michael Riesch
>>> <[email protected]> wrote:
>>>>
>>>> Hi Dave,
>>>>
>>>> On 4/6/23 17:16, Dave Stevenson wrote:
>>>>> Hi Michael
>>>>>
>>>>> Thanks for the patch.
>>>>>
>>>>> I've got a personal interest here as I'd love to be able to control a
>>>>> couple of CCTV lenses that I have. Those have standard motors with
>>>>> potentiometers for position feedback, not stepper motors, but
>>>>> otherwise have the same limitations of slow movement.
>>>>
>>>> That's great to hear :-) Thank you for your feedback!
>>>>
>>>>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via
>>>>> libcamera-devel <[email protected]> wrote:
>>>>>>
>>>>>> From: Michael Riesch <[email protected]>
>>>>>>
>>>>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report
>>>>>> the status of the zoom lens group and the focus lens group, respectively.
>>>>>> The returned data structure contains the current position of the lens group
>>>>>> as well as movement indication flags.
>>>>>>
>>>>>> Signed-off-by: Michael Riesch <[email protected]>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++
>>>>>> include/media/v4l2-ctrls.h | 2 +
>>>>>> include/uapi/linux/v4l2-controls.h | 13 ++++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 6 files changed, 81 insertions(+)
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>>>> index daa4f40869f8..3a270bc63f1a 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
>>>>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering -
>>>>>> to the camera, negative values towards infinity. This is a
>>>>>> write-only control.
>>>>>>
>>>>>> +``V4L2_CID_FOCUS_STATUS (struct)``
>>>>>> + The current status of the focus lens group. This is a read-only control.
>>>>>> + The returned data structure contains the current position and movement
>>>>>> + indication flags. The unit of the current position is undefined. Positive
>>>>>> + values move the focus closer to the camera, negative values towards
>>>>>> + infinity. The possible flags are described in the table below.
>>>>>
>>>>> The units are undefined, but positive and negative mean something with
>>>>> respect to some unspecified focal distance represented by 0. That
>>>>> seems a little odd.
>>>>>
>>>>> I was going to suggest that it seems to make sense to follow the same
>>>>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in
>>>>> [1] it's the same text.
>>>>> I suspect there was a little too much copy/paste from
>>>>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value
>>>>> brings the focus closer, and decreasing moves it towards infinity.
>>>>>
>>>>> If we did specify that it was the same units as
>>>>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with
>>>>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears
>>>>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so
>>>>> potentially it's redundant and could be deprecated.
>>>>>
>>>>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html
>>>>
>>>> I think we agree that the _STATUS controls should use the same unit as
>>>> the corresponding _ABSOLUTE control, whatever that may be.
>>>>
>>>> Then, the question is whether the description of FOCUS_ABSOLUTE needs to
>>>> be revised. I would agree that the intention probably was: "Larger
>>>> values move the focus closer to the camera, smaller values move the
>>>> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE).
>>>>
>>>> It would be interesting to know whether zero and negative values are
>>>> (intentionally?) included. Since they are not explicitly excluded, my
>>>> driver exposes the position of the lens in motor steps to the user
>>>> space. If the values were (supposed to be) restricted to positive values
>>>> like ZOOM_ABSOLUTE, this would not be allowed of course.
>>>>
>>>> As to the relation to _RELATIVE, I think it should be clear that all
>>>> controls should act on the same scale and I don't see any problems here.
>>>> However, feel free to point out what I am missing :-)
>>>>
>>>>>> +
>>>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>>>>>> +
>>>>>> +.. flat-table::
>>>>>> + :header-rows: 0
>>>>>> + :stub-columns: 0
>>>>>> +
>>>>>> + * - ``V4L2_LENS_STATUS_IDLE``
>>>>>> + - Focus lens group is at rest.
>>>>>> + * - ``V4L2_LENS_STATUS_BUSY``
>>>>>> + - Focus lens group is moving.
>>>>>> + * - ``V4L2_LENS_STATUS_REACHED``
>>>>>> + - Focus lens group has reached its target position.
>>>>>> + * - ``V4L2_LENS_STATUS_FAILED``
>>>>>> + - Focus lens group has failed to reach its target position. The driver
>>>>>> + will not transition from this state until another action is performed
>>>>>> + by an application.
>>>>>
>>>>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to
>>>>> V4L2_LENS_STATUS_IDLE?
>>>>> If it's reached the position then it is at rest, and being at rest is
>>>>> the definition of V4L2_LENS_STATUS_IDLE.
>>>>> Likewise the lens always has a target position based on the control
>>>>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not
>>>>> moving.
>>>>> Is there a need to have 2 states?
>>>>
>>>> Good point, I need to reconsider that (and possibly remove one of those
>>>> states).
>>>>
>>>>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do
>>>>> you leave the determination of state to the application?
>>>>
>>>> I am afraid I don't quite follow.
>>>
>>> If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and
>>> V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the
>>> application knows that the lens has reached the requested position. If
>>> the two controls return different values then the lens is still
>>> moving.
>>> What new information does adding an additional status flag give you?
>>
>> I can think of two situations:
>>
>> - Hardware error: the lens stops at 40 (for whatever reason). Since 40
>> != 42 the lens is still moving, although a hardware error has occured
>> that may need recovery or at least reporting.
>
> Are you thinking of the lens drive having totally failed, or just that
> the requested position is out of the achievable range?

I would say the latter. At least one of our optics (with one focus lens
and two zoom lenses) exhibits nontrivial constraints for the range of
the individual lenses.

> Looking at the flash controls[1], hardware failures are handled by
> V4L2_CID_FLASH_FAULT as a separate control.
>
> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-flash.html
>
>> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current),
>> then what happens if the application performs a
>> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39,
>> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from
>> the application's point of view.
>
> V4L2_CID_FOCUS_RELATIVE really ought to die IMHO, or at least work in
> co-operation with V4L2_CID_FOCUS_ABSOLUTE such that the absolute
> current position can always be retrieved.
>
> Using _RELATIVE to set a position that is outside of the range defined
> by _ABSOLUTE is meaningless. Any call to S_CTRL on _RELATIVE almost
> has to be followed by a call to G_CTRL _ABSOLUTE to know the final
> result, so why not just make a S_CTRL _ABSOLUTE call in the first
> place.
>
> I guess _RELATIVE may have uses in the case where there is no
> read-back of the position at all, but in that case I don't see a way
> that your new controls could be implemented.

I am quite neutral to this. We don't plan to use the _RELATIVE controls
and if you want to get rid of them there won't be any objections from
our side.

If the drivers should only implement either ABSOLUTE or RELATIVE then
situation I described above is not an issue, of course.

>> Don't get me wrong, a simple control that reports the current position
>> would be way easier. But I feel that we may regret this in the future.
>> Also, I would not use two separate controls "current" and "flags/status"
>> in order to avoid non-atomic access patterns.
>
> I have no fixed views, but overcomplicating things is generally bad news.
>
>>> (Thinking aloud) I guess you have got my case where your readback is
>>> via an ADC so that the position may fluctuate slightly due to
>>> conversion noise. The control loop for applying power to the motors
>>> will presumably stop at some point and stop trying to adjust the
>>> position, so potentially it could be the state returned from that
>>> control loop. However the noisy ADC position could also be solved by
>>> the position being returned by the control loop instead of giving the
>>> raw value.
>>
>> With stepper motors the readback is not that noisy, but in any case the
>> flags field allows an underlying controller to signal certain conditions
>> without relying on the exact value of the current position. I think that
>> is a plus, especially for noisy readback.
>>
>>>> But FWIW the application should read out the flags to find out whether
>>>> the lens is moving and, if this is not the case, whether the last move
>>>> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The
>>>> failed state is important as there might be the possibility that
>>>> depending on the zoom lens position certain focus positions are not
>>>> available (and vice versa).
>>>
>>> How would you configure those kinds of restrictions in the kernel
>>> driver? Is the kernel driver the best place to do it (so many things
>>> are being kicked back to userspace these days)?
>>
>> In our case we have a hardware controller that needs to handle the
>> restrictions anyway in order to avoid mechanical damage. The kernel
>> driver is only responsible for reading out the controller status and
>> transforming it into the V4L2 control.
>
> How does userspace know what the valid ranges are? Do you end up with
> the same restrictions having to be coded in both your controller and
> userspace, or are you relying on the driver reporting back this error
> status with userspace not knowing by how much it got it wrong by?

Maybe I should clarify a bit the use cases here. Most of the time the
ZOOM_ABSOLUTE and FOCUS_ABSOLUTE controls will be used to operate the
optics. Here, AFAIK there should not be any constraints (or at least,
they are handled by the lens controller hardware) and user space can
pick any value between the minimum and maximum of the respective control.

The situation is more complicated for the calibration. Here, individual
lenses can be moved and may be subject to certain constraints. Before
the calibration, the limits may be unknown or estimated. Then, one
approach could be to set the target position of a lens to INT_MAX and
read back the status. The error status plus the current position could
then yield the actual limit.

Note that such calibration techniques are very specialized use cases,
but with the proposed API it should be possible to cover several
different approaches.

>> I am not sure how any restrictions can be reliably handled in software,
>> hence I don't have a strong opinion on whether this is done in kernel or
>> user space.
>>
>>> If it is to be done in the kernel, then shouldn't eg changing the zoom
>>> position alter the ranges advertised for the relevant focus controls?
>>
>> While it should be possible to update the minimum and maximum of e.g.,
>> FOCUS_ABSOLUTE, I am not sure whether the interface is designed for
>> frequent re-reading of the range.
>
> There's nothing in there that stops you, and several hooks to help you.
> Updating V4L2_CID_VBLANK on most raw sensors will update the range on
> V4L2_CID_EXPOSURE, as the exposure period can't be longer than the
> frame period. It would be similar that each change to ZOOM_ABSOLUTE
> would result in a range adjustment to FOCUS_ABSOLUTE.
> Userspace can also use VIDIOC_SUBSCRIBE_EVENT to subscribe to any
> updates in controls, so it gets told automatically of any updates.

OK, maybe we'll give this some more thought!

Thanks a lot for your comments!

Best regards,
Michael

>>> If the ranges aren't updated, where should that out-of-range lens
>>> movement leave the lens?
>>
>> This is up to the hardware controller, but I would guess it typically
>> stops one step before disaster. Wherever that may be, the error
>> condition and the current position can be read out via this new STATUS
>> control.
>>
>> Does this sound good so far?
>
> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
> Laurent), and I'm just expressing my views based on the lenses I've
> encountered.
> All of my lenses have a single drive for focus, a single drive for
> zoom, and where there are multiple elements they are all connected
> mechanically. Your setup sounds far more complex and is likely to need
> a more extensive driver, but it'd be nice to not unnecessarily
> overcomplicate the interface.
>
> Dave
>
>> Best regards,
>> Michael
>>
>>>
>>> Dave
>>>
>>>>>> ``V4L2_CID_FOCUS_AUTO (boolean)``
>>>>>> Enables continuous automatic focus adjustments. The effect of manual
>>>>>> focus adjustments while this feature is enabled is undefined,
>>>>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range -
>>>>>> movement. A negative value moves the zoom lens group towards the
>>>>>> wide-angle direction. The zoom speed unit is driver-specific.
>>>>>>
>>>>>> +``V4L2_CID_ZOOM_STATUS (struct)``
>>>>>> + The current status of the zoom lens group. This is a read-only control.
>>>>>> + The returned data structure contains the current position and movement
>>>>>> + indication flags. The unit of the current position is driver-specific and
>>>>>> + its value should be a positive integer. The possible flags are described
>>>>>> + in the table below.
>>>>>> +
>>>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}|
>>>>>> +
>>>>>> +.. flat-table::
>>>>>> + :header-rows: 0
>>>>>> + :stub-columns: 0
>>>>>> +
>>>>>> + * - ``V4L2_LENS_STATUS_IDLE``
>>>>>> + - Zoom lens group is at rest.
>>>>>> + * - ``V4L2_LENS_STATUS_BUSY``
>>>>>> + - Zoom lens group is moving.
>>>>>> + * - ``V4L2_LENS_STATUS_REACHED``
>>>>>> + - Zoom lens group has reached its target position.
>>>>>> + * - ``V4L2_LENS_STATUS_FAILED``
>>>>>> + - Zoom lens group has failed to reach its target position. The driver will
>>>>>> + not transition from this state until another action is performed by an
>>>>>> + application.
>>>>>> +
>>>>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)``
>>>>>> This control sets the camera's aperture to the specified value. The
>>>>>> unit is undefined. Larger values open the iris wider, smaller values
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>>>> index 29169170880a..f6ad30f311c5 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>>>>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
>>>>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
>>>>>> pr_cont("HEVC_DECODE_PARAMS");
>>>>>> break;
>>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>>>> + pr_cont("LENS_STATUS");
>>>>>> + break;
>>>>>> default:
>>>>>> pr_cont("unknown type %d", ctrl->type);
>>>>>> break;
>>>>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
>>>>>> return -EINVAL;
>>>>>> break;
>>>>>>
>>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>>>> + break;
>>>>>> +
>>>>>> default:
>>>>>> return -EINVAL;
>>>>>> }
>>>>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>>>>>> case V4L2_CTRL_TYPE_AREA:
>>>>>> elem_size = sizeof(struct v4l2_area);
>>>>>> break;
>>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS:
>>>>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status);
>>>>>> + break;
>>>>>> default:
>>>>>> if (type < V4L2_CTRL_COMPOUND_TYPES)
>>>>>> elem_size = sizeof(s32);
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>>>> index 564fedee2c88..9b26a3aa9e9c 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>>>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id)
>>>>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation";
>>>>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation";
>>>>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode";
>>>>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status";
>>>>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status";
>>>>>
>>>>> Is there a need for the comma in the text strings?
>>>>
>>>> Not sure, actually. Some other strings used commas. Monkey see, monkey do.
>>>>
>>>> Best regards,
>>>> Michael
>>>>
>>>>>
>>>>> Cheers
>>>>> Dave
>>>>>
>>>>>>
>>>>>> /* FM Radio Modulator controls */
>>>>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
>>>>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>>>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
>>>>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
>>>>>> break;
>>>>>> + case V4L2_CID_FOCUS_STATUS:
>>>>>> + case V4L2_CID_ZOOM_STATUS:
>>>>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS;
>>>>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE;
>>>>>> + break;
>>>>>> case V4L2_CID_FLASH_STROBE_STATUS:
>>>>>> case V4L2_CID_AUTO_FOCUS_STATUS:
>>>>>> case V4L2_CID_FLASH_READY:
>>>>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>>>>>> index e59d9a234631..f7273ffc20c9 100644
>>>>>> --- a/include/media/v4l2-ctrls.h
>>>>>> +++ b/include/media/v4l2-ctrls.h
>>>>>> @@ -52,6 +52,7 @@ struct video_device;
>>>>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure.
>>>>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure.
>>>>>> * @p_area: Pointer to an area.
>>>>>> + * @p_lens_status: Pointer to a lens status structure.
>>>>>> * @p: Pointer to a compound value.
>>>>>> * @p_const: Pointer to a constant compound value.
>>>>>> */
>>>>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr {
>>>>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
>>>>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
>>>>>> struct v4l2_area *p_area;
>>>>>> + struct v4l2_ctrl_lens_status *p_lens_status;
>>>>>> void *p;
>>>>>> const void *p_const;
>>>>>> };
>>>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>>>>> index 5e80daa4ffe0..8b037467ba9a 100644
>>>>>> --- a/include/uapi/linux/v4l2-controls.h
>>>>>> +++ b/include/uapi/linux/v4l2-controls.h
>>>>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range {
>>>>>>
>>>>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)
>>>>>>
>>>>>> +struct v4l2_ctrl_lens_status {
>>>>>> + __u32 flags;
>>>>>> + __s32 current_position;
>>>>>> +};
>>>>>> +
>>>>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0)
>>>>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0)
>>>>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1)
>>>>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2)
>>>>>> +
>>>>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37)
>>>>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38)
>>>>>> +
>>>>>> /* FM Modulator class control IDs */
>>>>>>
>>>>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900)
>>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>>> index 17a9b975177a..256c21c68720 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type {
>>>>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
>>>>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
>>>>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
>>>>>> +
>>>>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300,
>>>>>> };
>>>>>>
>>>>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>>>>>>
>>>>>> --
>>>>>> 2.30.2
>>>>>>

2023-04-17 12:41:01

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Sakari,

On 4/12/23 17:12, Sakari Ailus wrote:
> Hi Dave, Michael,
>
> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote:
>>>> If the ranges aren't updated, where should that out-of-range lens
>>>> movement leave the lens?
>>>
>>> This is up to the hardware controller, but I would guess it typically
>>> stops one step before disaster. Wherever that may be, the error
>>> condition and the current position can be read out via this new STATUS
>>> control.
>>>
>>> Does this sound good so far?
>>
>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
>> Laurent), and I'm just expressing my views based on the lenses I've
>> encountered.
>> All of my lenses have a single drive for focus, a single drive for
>> zoom, and where there are multiple elements they are all connected
>> mechanically. Your setup sounds far more complex and is likely to need
>> a more extensive driver, but it'd be nice to not unnecessarily
>> overcomplicate the interface.
>
> Could we also have a driver that uses these new controls?

If you are referring to the driver for our custom lens controller, then
I have to say that it is under development and simply not ready for
release yet. Also, the decision has not yet been made whether or not
this will be an open-source driver.

A different approach could be the adaptation of the vimc-lens driver,
which currently only supports FOCUS_ABSOLUTE. But this would raise
several implementation questions and at least for me this would be a
nontrivial task.

Is it required to have a driver for this interface (in the sense that
the patches cannot be accepted otherwise)?

> The controls themselves appear reasonable to me as well. I guess there are
> changes to be made based on the discussion?

I'd summarize that whether or not the status controls are compound
controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question.

As a potential follow-up question I recently asked myself if the struct
v4l2_ctrl_lens_status should contain trailing reserved bytes for future
extension (no idea, though, what this could be).

Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)"
for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add
further controls when they are needed. Here, we lose atomicity but maybe
this can be ignored. One could assume that all relevant controls are
read out with a single ioctl which provides at least some level of
atomicity.

Any comments and/or recommendations to this open question would be much
appreciated.

Other review comments will be incorporated in the next iteration of this
series as well, but they are quite straightforward.

Best regards,
Michael

2023-04-19 09:03:53

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael,

On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote:
> Hi Sakari,
>
> On 4/12/23 17:12, Sakari Ailus wrote:
> > Hi Dave, Michael,
> >
> > On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote:
> >>>> If the ranges aren't updated, where should that out-of-range lens
> >>>> movement leave the lens?
> >>>
> >>> This is up to the hardware controller, but I would guess it typically
> >>> stops one step before disaster. Wherever that may be, the error
> >>> condition and the current position can be read out via this new STATUS
> >>> control.
> >>>
> >>> Does this sound good so far?
> >>
> >> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
> >> Laurent), and I'm just expressing my views based on the lenses I've
> >> encountered.
> >> All of my lenses have a single drive for focus, a single drive for
> >> zoom, and where there are multiple elements they are all connected
> >> mechanically. Your setup sounds far more complex and is likely to need
> >> a more extensive driver, but it'd be nice to not unnecessarily
> >> overcomplicate the interface.
> >
> > Could we also have a driver that uses these new controls?
>
> If you are referring to the driver for our custom lens controller, then
> I have to say that it is under development and simply not ready for
> release yet. Also, the decision has not yet been made whether or not
> this will be an open-source driver.
>
> A different approach could be the adaptation of the vimc-lens driver,
> which currently only supports FOCUS_ABSOLUTE. But this would raise
> several implementation questions and at least for me this would be a
> nontrivial task.
>
> Is it required to have a driver for this interface (in the sense that
> the patches cannot be accepted otherwise)?

That has been traditionally required, and a virtual driver isn't usually
considered enough. There are at least two reasons for this. The first one
being that if the driver isn't reviewable and targetting upstream it may be
difficult to figure out whether the interface changes are the right ones
for that driver. This is perhaps a lesser concern here. Secondly, there is
also unwillingness to add interface elements that might never be supported
by the kernel itself --- this is effectively just dead code.

Also cc Hans and Laurent.

>
> > The controls themselves appear reasonable to me as well. I guess there are
> > changes to be made based on the discussion?
>
> I'd summarize that whether or not the status controls are compound
> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question.
>
> As a potential follow-up question I recently asked myself if the struct
> v4l2_ctrl_lens_status should contain trailing reserved bytes for future
> extension (no idea, though, what this could be).
>
> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)"
> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add
> further controls when they are needed. Here, we lose atomicity but maybe
> this can be ignored. One could assume that all relevant controls are
> read out with a single ioctl which provides at least some level of
> atomicity.

There might be something that could be done in the control framework to
address this. But it's not something that can be expected to happen soon.

I'd perhaps keep them separate, not to make it a compound control just for
the access reason. But I certainly don't have a strong opinion about it.

>
> Any comments and/or recommendations to this open question would be much
> appreciated.
>
> Other review comments will be incorporated in the next iteration of this
> series as well, but they are quite straightforward.

--
Kind regards,

Sakari Ailus

2023-04-19 11:33:48

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Sakari,

On 4/19/23 11:01, Sakari Ailus wrote:
> Hi Michael,
>
> On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote:
>> Hi Sakari,
>>
>> On 4/12/23 17:12, Sakari Ailus wrote:
>>> Hi Dave, Michael,
>>>
>>> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote:
>>>>>> If the ranges aren't updated, where should that out-of-range lens
>>>>>> movement leave the lens?
>>>>>
>>>>> This is up to the hardware controller, but I would guess it typically
>>>>> stops one step before disaster. Wherever that may be, the error
>>>>> condition and the current position can be read out via this new STATUS
>>>>> control.
>>>>>
>>>>> Does this sound good so far?
>>>>
>>>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
>>>> Laurent), and I'm just expressing my views based on the lenses I've
>>>> encountered.
>>>> All of my lenses have a single drive for focus, a single drive for
>>>> zoom, and where there are multiple elements they are all connected
>>>> mechanically. Your setup sounds far more complex and is likely to need
>>>> a more extensive driver, but it'd be nice to not unnecessarily
>>>> overcomplicate the interface.
>>>
>>> Could we also have a driver that uses these new controls?
>>
>> If you are referring to the driver for our custom lens controller, then
>> I have to say that it is under development and simply not ready for
>> release yet. Also, the decision has not yet been made whether or not
>> this will be an open-source driver.
>>
>> A different approach could be the adaptation of the vimc-lens driver,
>> which currently only supports FOCUS_ABSOLUTE. But this would raise
>> several implementation questions and at least for me this would be a
>> nontrivial task.
>>
>> Is it required to have a driver for this interface (in the sense that
>> the patches cannot be accepted otherwise)?
>
> That has been traditionally required, and a virtual driver isn't usually
> considered enough. There are at least two reasons for this. The first one
> being that if the driver isn't reviewable and targetting upstream it may be
> difficult to figure out whether the interface changes are the right ones
> for that driver. This is perhaps a lesser concern here. Secondly, there is
> also unwillingness to add interface elements that might never be supported
> by the kernel itself --- this is effectively just dead code.
>
> Also cc Hans and Laurent.

I understand your concerns. Cc: Alexander and Dieter

We aim to be an open-source friendly company. If you are OK with us
submitting a driver that targets very custom hardware that is only
available in integrated form in our products (and not, for instance,
available for sale as a standalone device), then we are prepared to
submit the driver sources for consideration for inclusion in mainline
Linux. Would this be acceptable?

As I already stated above, it will take us some time to prepare
everything in a form that is suitable for submission. Now should I
submit the next iteration(s) of the series at hand as RFC or as regular
patch series?

>>> The controls themselves appear reasonable to me as well. I guess there are
>>> changes to be made based on the discussion?
>>
>> I'd summarize that whether or not the status controls are compound
>> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question.
>>
>> As a potential follow-up question I recently asked myself if the struct
>> v4l2_ctrl_lens_status should contain trailing reserved bytes for future
>> extension (no idea, though, what this could be).
>>
>> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)"
>> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add
>> further controls when they are needed. Here, we lose atomicity but maybe
>> this can be ignored. One could assume that all relevant controls are
>> read out with a single ioctl which provides at least some level of
>> atomicity.
>
> There might be something that could be done in the control framework to
> address this. But it's not something that can be expected to happen soon.
>
> I'd perhaps keep them separate, not to make it a compound control just for
> the access reason. But I certainly don't have a strong opinion about it.

After some further considerations, and following Dave's and your
comments, I'll keep them separate.

Discussion to be continued with v2.

Best regards,
Michael

>
>>
>> Any comments and/or recommendations to this open question would be much
>> appreciated.
>>
>> Other review comments will be incorporated in the next iteration of this
>> series as well, but they are quite straightforward.
>

2023-04-19 12:53:04

[permalink] [raw]

Subject: Re: [libcamera-devel] [PATCH RFC 1/4] media: v4l2-ctrls: add lens group status controls for zoom and focus

Hi Michael

On Wed, 19 Apr 2023 at 12:25, Michael Riesch
<[email protected]> wrote:
>
> Hi Sakari,
>
> On 4/19/23 11:01, Sakari Ailus wrote:
> > Hi Michael,
> >
> > On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote:
> >> Hi Sakari,
> >>
> >> On 4/12/23 17:12, Sakari Ailus wrote:
> >>> Hi Dave, Michael,
> >>>
> >>> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote:
> >>>>>> If the ranges aren't updated, where should that out-of-range lens
> >>>>>> movement leave the lens?
> >>>>>
> >>>>> This is up to the hardware controller, but I would guess it typically
> >>>>> stops one step before disaster. Wherever that may be, the error
> >>>>> condition and the current position can be read out via this new STATUS
> >>>>> control.
> >>>>>
> >>>>> Does this sound good so far?
> >>>>
> >>>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or
> >>>> Laurent), and I'm just expressing my views based on the lenses I've
> >>>> encountered.
> >>>> All of my lenses have a single drive for focus, a single drive for
> >>>> zoom, and where there are multiple elements they are all connected
> >>>> mechanically. Your setup sounds far more complex and is likely to need
> >>>> a more extensive driver, but it'd be nice to not unnecessarily
> >>>> overcomplicate the interface.
> >>>
> >>> Could we also have a driver that uses these new controls?
> >>
> >> If you are referring to the driver for our custom lens controller, then
> >> I have to say that it is under development and simply not ready for
> >> release yet. Also, the decision has not yet been made whether or not
> >> this will be an open-source driver.
> >>
> >> A different approach could be the adaptation of the vimc-lens driver,
> >> which currently only supports FOCUS_ABSOLUTE. But this would raise
> >> several implementation questions and at least for me this would be a
> >> nontrivial task.
> >>
> >> Is it required to have a driver for this interface (in the sense that
> >> the patches cannot be accepted otherwise)?
> >
> > That has been traditionally required, and a virtual driver isn't usually
> > considered enough. There are at least two reasons for this. The first one
> > being that if the driver isn't reviewable and targetting upstream it may be
> > difficult to figure out whether the interface changes are the right ones
> > for that driver. This is perhaps a lesser concern here. Secondly, there is
> > also unwillingness to add interface elements that might never be supported
> > by the kernel itself --- this is effectively just dead code.
> >
> > Also cc Hans and Laurent.
>
> I understand your concerns. Cc: Alexander and Dieter
>
> We aim to be an open-source friendly company. If you are OK with us
> submitting a driver that targets very custom hardware that is only
> available in integrated form in our products (and not, for instance,
> available for sale as a standalone device), then we are prepared to
> submit the driver sources for consideration for inclusion in mainline
> Linux. Would this be acceptable?

My plan with the motor drive is far simpler with a Pi RP2040
microcontroller on I2C running an ADC for the potentiometers, PWM for
motor control, and a PID loop driving it. The MCU code will be open
source.
It is a spare time project rather than work, so I can't guarantee
timescales, but I'll see if I can find some time to progress it.

(It could all be driven from the kernel with ADC and PWM, but I don't
see such a driver being accepted. Offloading it to an MCU seems to be
the easier option).

> As I already stated above, it will take us some time to prepare
> everything in a form that is suitable for submission. Now should I
> submit the next iteration(s) of the series at hand as RFC or as regular
> patch series?
>
> >>> The controls themselves appear reasonable to me as well. I guess there are
> >>> changes to be made based on the discussion?
> >>
> >> I'd summarize that whether or not the status controls are compound
> >> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question.
> >>
> >> As a potential follow-up question I recently asked myself if the struct
> >> v4l2_ctrl_lens_status should contain trailing reserved bytes for future
> >> extension (no idea, though, what this could be).
> >>
> >> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)"
> >> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add
> >> further controls when they are needed. Here, we lose atomicity but maybe
> >> this can be ignored. One could assume that all relevant controls are
> >> read out with a single ioctl which provides at least some level of
> >> atomicity.

VIDIOC_G_EXT_CTRLS should allow you to read multiple controls in one ioctl call.
It would be multiple calls to your g_volatile_ctrl handler, so
potentially multiple I2C calls to your lens controller.
There is the option of cluster controls, but I think that is largely
only applicable for setting controls rather than reading them.

Dave

> > There might be something that could be done in the control framework to
> > address this. But it's not something that can be expected to happen soon.
> >
> > I'd perhaps keep them separate, not to make it a compound control just for
> > the access reason. But I certainly don't have a strong opinion about it.
>
> After some further considerations, and following Dave's and your
> comments, I'll keep them separate.
>
> Discussion to be continued with v2.
>
> Best regards,
> Michael
>
> >
> >>
> >> Any comments and/or recommendations to this open question would be much
> >> appreciated.
> >>
> >> Other review comments will be incorporated in the next iteration of this
> >> series as well, but they are quite straightforward.
> >

2023-04-24 20:03:27