Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Louis Kuo <louis.kuo@mediatek.com>
To:     <sakari.ailus@linux.intel.com>,
        <laurent.pinchart@ideasonboard.com>, <mchehab@kernel.org>,
        <matthias.bgg@gmail.com>, <hverkuil-cisco@xs4all.nl>,
        <arnd@arndb.de>, <louis.kuo@mediatek.com>,
        <sergey.senozhatsky@gmail.com>, <helen.koike@collabora.com>,
        <niklas.soderlund+renesas@ragnatech.se>, <yepeilin.cs@gmail.com>
CC:     <frederic.chen@mediatek.com>, <linux-media@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>,
        <linux-arm-kernel@lists.infradead.org>,
        <linux-mediatek@lists.infradead.org>
Subject: [RFC PATCH V0 0/4] media: some framework interface extension for new feature of Mediatek Camsys driver 
Date:   Wed, 5 May 2021 16:06:22 +0800
Message-ID: <20210505080626.15432-1-louis.kuo@mediatek.com>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Hello,

This is the first version of the RFC patch series extending V4L2 and media
framework to support some advanced camera function, for example, to change
the sensor when ISP is still streaming. A typical scenario is the wide-angle
sensor and telephoto sensor switching in camera application. When the user
is using the zooming UI, the application needs to switch the sensor from
wide-angle sensor to telephoto sensor smoothly.

To finish the function, we may need to modify the links of a pipeline and
the format of pad and video device per request. Currently, the link,
pad and video device format and selection settings are not involved in
media request's design. Therefore, we try to extend the related interface
to support the request-based operations. In the early version, we added
request fd to the parameters of MEDIA_IOC_SETUP_LINK,
VIDIOC_S_FMT, VIDIOC_SUBDEV_S_SELECTION, VIDIOC_SUBDEV_S_FMT.
The driver uses media_request_get_by_fd() to retrieve the media request
and save the pending change in it, so that we can apply the pending change
in req_queue() callback then.

Here is an example:

int mtk_cam_vidioc_s_selection(struct file *file, void *fh,
				struct v4l2_selection *s)
{
	struct mtk_cam_device *cam = video_drvdata(file);
	struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
	struct mtk_cam_request_stream_data *stream_data;
	struct mtk_cam_request *cam_req;
	struct media_request *req;
	s32 fd;

	fd = s->request_fd;
	if (fd < 0)
		return -EINVAL;

	req = media_request_get_by_fd(&cam->media_dev, fd);

	/* .... */
 
	cam_req = to_mtk_cam_req(req);
	stream_data = &cam_req->stream_data[node->uid.pipe_id];
	stream_data->vdev_selection_update |= (1 << node->desc.id);
	stream_data->vdev_selection[node->desc.id] = *s;

	/* .... */

	media_request_put(req);

	return 0;
}

I posted interface change as RFC to discuss first and would like some
review comments.

Thank you very much.

  media: v4l2-core: extend the v4l2 format to support request
  media: subdev: support which in v4l2_subdev_frame_interval
  media: v4l2-ctrl: Add ISP Camsys user control
  media: pixfmt: Add ISP Camsys formats

 drivers/media/mc/mc-device.c         |   7 +-
 drivers/media/v4l2-core/v4l2-ioctl.c | 153 ++++++++++++++++++++++++++-
 include/media/media-entity.h         |   3 +
 include/uapi/linux/media.h           |   3 +-
 include/uapi/linux/v4l2-controls.h   |   4 +
 include/uapi/linux/v4l2-subdev.h     |   8 +-
 include/uapi/linux/videodev2.h       | 109 ++++++++++++++++++-
 7 files changed, 275 insertions(+), 12 deletions(-)