Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 1/4] v4l2-ctrl: Make display delay and display enable std
 controls
To:     Hans Verkuil <hverkuil-cisco@xs4all.nl>,
        Stanimir Varbanov <stanimir.varbanov@linaro.org>,
        linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org
Cc:     Kyungmin Park <kyungmin.park@samsung.com>,
        Kamil Debski <kamil@wypas.org>,
        Ezequiel Garcia <ezequiel@collabora.com>,
        Nicolas Dufresne <nicolas.dufresne@collabora.com>,
        Marek Szyprowski <m.szyprowski@samsung.com>
References: <20201109173541.10016-1-stanimir.varbanov@linaro.org>
 <20201109173541.10016-2-stanimir.varbanov@linaro.org>
 <e0a49eb2-f4d0-0532-52e0-5bc58ce85ad9@xs4all.nl>
From:   Stanimir Varbanov <stanimir.varbanov@linaro.org>
Message-ID: <f775e5cf-3361-e7ed-bd85-6d0ad06db48c@linaro.org>
Date:   Fri, 4 Dec 2020 15:13:13 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <e0a49eb2-f4d0-0532-52e0-5bc58ce85ad9@xs4all.nl>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 12/2/20 4:09 PM, Hans Verkuil wrote:
> On 09/11/2020 18:35, Stanimir Varbanov wrote:
>> Make display delay and display delay enable MFC controls standard v4l
>> controls. This will allow reuse of the controls for other decoder
>> drivers. Also the new proposed controls are now codec agnostic because
>> they could be used for any codec.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  .../userspace-api/media/v4l/ext-ctrls-codec.rst   | 15 +++++++++++++++
>>  drivers/media/v4l2-core/v4l2-ctrls.c              |  4 ++++
>>  include/uapi/linux/v4l2-controls.h                |  2 ++
>>  3 files changed, 21 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index ce728c757eaf..82c9cda40270 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -679,6 +679,21 @@ enum v4l2_mpeg_video_frame_skip_mode -
>>      otherwise the decoder expects a single frame in per buffer.
>>      Applicable to the decoder, all codecs.
>>  
>> +``V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY_ENABLE (boolean)``
> 
> I'd use _DEC_ instead of _DECODER_.

OK.

> 
>> +    If the display delay is enabled then the decoder is forced to return
>> +    a CAPTURE buffer (decoded frame) after processing a certain number
>> +    of OUTPUT buffers. The delay can be set through
>> +    ``V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY``. This
>> +    feature can be used for example for generating thumbnails of videos.
>> +    Applicable to the decoder.
> 
> Hmm. Is this: "after processing the first 'display delay' number of OUTPUT buffers."
> Or is this: "every 'display delay' number of OUTPUT buffers."
> 
> I.e., is it a one-shot thing or a periodical thing?

It is periodical.

> 
> If it is a one-shot thing, then this should probably be a button type, not
> a boolean.
> 
>> +
>> +``V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY (integer)``
>> +    Display delay value for decoder. The decoder is forced to
>> +    return a decoded frame after the set 'display delay' number of
>> +    frames. If this number is low it may result in frames returned out
>> +    of display order, in addition the hardware may still be using the
>> +    returned buffer as a reference picture for subsequent frames.
> 
> Can this be 0? And if so, what does that mean?

Yes, it can be 0 and I'm using this to change decoder decode order in
Venus driver:

V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY ctl->val = 0
V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY_ENABLE ctl->val = true

Then the decoder will produce output buffers in decode-order instead of
default display-order.

> 
>> +
>>  ``V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE (boolean)``
>>      Enable writing sample aspect ratio in the Video Usability
>>      Information. Applicable to the H264 encoder.
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
>> index bd7f330c941c..4a21802e026b 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>> @@ -874,6 +874,8 @@ const char *v4l2_ctrl_get_name(u32 id)
>>  	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:			return "Sequence Header Mode";
>>  	case V4L2_CID_MPEG_VIDEO_MAX_REF_PIC:			return "Max Number of Reference Pics";
>>  	case V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE:		return "Frame Skip Mode";
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY:		return "Display Delay";
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY_ENABLE:	return "Display Delay Enable";
>>  	case V4L2_CID_MPEG_VIDEO_H263_I_FRAME_QP:		return "H263 I-Frame QP Value";
>>  	case V4L2_CID_MPEG_VIDEO_H263_P_FRAME_QP:		return "H263 P-Frame QP Value";
>>  	case V4L2_CID_MPEG_VIDEO_H263_B_FRAME_QP:		return "H263 B-Frame QP Value";
>> @@ -1221,6 +1223,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>  	case V4L2_CID_FLASH_READY:
>>  	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
>>  	case V4L2_CID_MPEG_VIDEO_DECODER_SLICE_INTERFACE:
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY_ENABLE:
>>  	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
>>  	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
>>  	case V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM:
>> @@ -1256,6 +1259,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>  		break;
>>  	case V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE:
>>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY:
>>  		*type = V4L2_CTRL_TYPE_INTEGER;
>>  		break;
>>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index 7035f4fb182c..d6b19f8d0022 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -773,6 +773,8 @@ enum v4l2_mpeg_video_frame_skip_mode {
>>  	V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT	= 1,
>>  	V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT	= 2,
>>  };
>> +#define V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY		(V4L2_CID_MPEG_BASE + 647)
>> +#define V4L2_CID_MPEG_VIDEO_DECODER_DISPLAY_DELAY_ENABLE	(V4L2_CID_MPEG_BASE + 648)
> 
> This will need to be rebased once this PR is merged:
> https://patchwork.linuxtv.org/project/linux-media/patch/d68da172-b251-000f-653d-38a8a4c7b715@xs4all.nl/
> 
>>  
>>  /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */
>>  #define V4L2_CID_MPEG_CX2341X_BASE				(V4L2_CTRL_CLASS_MPEG | 0x1000)
>>
> 
> Regards,
> 
> 	Hans
> 

-- 
regards,
Stan