by Nicolas Dufresne

[permalink] [raw]

Subject: Re: [PATCH v17 8/8] media: verisilicon: Support deleting buffers on capture queue

Hi,

Le mercredi 24 janvier 2024 à 16:35 +0100, Benjamin Gaignard a écrit :
> Le 24/01/2024 à 13:52, Hans Verkuil a écrit :
> > On 19/01/2024 10:49, Benjamin Gaignard wrote:
> > > Allow to delete buffers on capture queue because it the one which
> > > own the decoded buffers. After a dynamic resolution change lot of
> > > them could remain allocated but won't be used anymore so deleting
> > > them save memory.
> > > Do not add this feature on output queue because the buffers are
> > > smaller, fewer and always recycled even after a dynamic resolution
> > > change.
> > >
> > > Signed-off-by: Benjamin Gaignard <[email protected]>
> > > ---
> > > drivers/media/platform/verisilicon/hantro_drv.c | 1 +
> > > drivers/media/platform/verisilicon/hantro_v4l2.c | 1 +
> > > 2 files changed, 2 insertions(+)
> > >
> > > diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
> > > index db3df6cc4513..f6b0a676a740 100644
> > > --- a/drivers/media/platform/verisilicon/hantro_drv.c
> > > +++ b/drivers/media/platform/verisilicon/hantro_drv.c
> > > @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
> > > dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > > dst_vq->lock = &ctx->dev->vpu_mutex;
> > > dst_vq->dev = ctx->dev->v4l2_dev.dev;
> > > + src_vq->supports_delete_bufs = true;
> > As I mentioned, I remain unconvinced by this. It is just making the API inconsistent
> > since if you support delete_bufs, then why support it for one queue only and not both?
>
> Because the both queues don't handle the same type of data.
> For example for a stateless decoder, for me, it makes sense to allow delete decoded frames
> if they won't be used anymore but that won't makes sense for bitstream buffers.

I personally think that for stateless and stateful decoder bitstream queue this
can be useful. We currently guess the size we need, and there is no way to
allocate bigger ones without the driver forgetting about reference frames.

In stateful, some drivers allow to split the bitstream (I tested wave5 notably),
but I was told this is not always the case. A bit of a gray zone in that API,
with lack of capabilities to describe it. On stateless, the entire bitstream
slice must be in one buffer.

Though, for the asymmetry, most stateful decoders won't allow delete bufs on
capture, because the buffers are registered in the firmware ahead of time. wave5
can't even implement create_bufs on capture. We had an argument about having
create_bufs on only one queue for that specific driver, as we wanted something
upstream, we flex to removing create bufs completely. I think the all or nothing
rule on per queue create/delete_bufs is not aligned with the reality.

Nicolas
>
> >
> > >
> > > return vb2_queue_init(dst_vq);
> > > }
> > > diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
> > > index 941fa23c211a..34eab90e8a42 100644
> > > --- a/drivers/media/platform/verisilicon/hantro_v4l2.c
> > > +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
> > > @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
> > > .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
> > > .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
> > > .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
> > > + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs,
> > > .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
> > >
> > > .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > In my view setting vidioc_delete_bufs should enable this feature, and if
> > for some strange reason only one queue support it, then make a wrapper
> > callback that returns an error when used with the wrong queue.
> >
> > Also note that patch 6/8 never checks for q->supports_delete_bufs in
> > vb2_core_delete_bufs(), which is wrong!
>
> I will fix that in next version.
> Regards,
> Benjamin
>
> >
> > Regards,
> >
> > Hans
> >
>

2024-01-26 10:13:02

by Hans Verkuil

[permalink] [raw]

Subject: Re: [PATCH v17 8/8] media: verisilicon: Support deleting buffers on capture queue

On 25/01/2024 17:27, Nicolas Dufresne wrote:
> Hi,
>
> Le mercredi 24 janvier 2024 à 16:35 +0100, Benjamin Gaignard a écrit :
>> Le 24/01/2024 à 13:52, Hans Verkuil a écrit :
>>> On 19/01/2024 10:49, Benjamin Gaignard wrote:
>>>> Allow to delete buffers on capture queue because it the one which
>>>> own the decoded buffers. After a dynamic resolution change lot of
>>>> them could remain allocated but won't be used anymore so deleting
>>>> them save memory.
>>>> Do not add this feature on output queue because the buffers are
>>>> smaller, fewer and always recycled even after a dynamic resolution
>>>> change.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <[email protected]>
>>>> ---
>>>> drivers/media/platform/verisilicon/hantro_drv.c | 1 +
>>>> drivers/media/platform/verisilicon/hantro_v4l2.c | 1 +
>>>> 2 files changed, 2 insertions(+)
>>>>
>>>> diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
>>>> index db3df6cc4513..f6b0a676a740 100644
>>>> --- a/drivers/media/platform/verisilicon/hantro_drv.c
>>>> +++ b/drivers/media/platform/verisilicon/hantro_drv.c
>>>> @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
>>>> dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>>> dst_vq->lock = &ctx->dev->vpu_mutex;
>>>> dst_vq->dev = ctx->dev->v4l2_dev.dev;
>>>> + src_vq->supports_delete_bufs = true;
>>> As I mentioned, I remain unconvinced by this. It is just making the API inconsistent
>>> since if you support delete_bufs, then why support it for one queue only and not both?
>>
>> Because the both queues don't handle the same type of data.
>> For example for a stateless decoder, for me, it makes sense to allow delete decoded frames
>> if they won't be used anymore but that won't makes sense for bitstream buffers.
>
> I personally think that for stateless and stateful decoder bitstream queue this
> can be useful. We currently guess the size we need, and there is no way to
> allocate bigger ones without the driver forgetting about reference frames.
>
> In stateful, some drivers allow to split the bitstream (I tested wave5 notably),
> but I was told this is not always the case. A bit of a gray zone in that API,
> with lack of capabilities to describe it. On stateless, the entire bitstream
> slice must be in one buffer.
>
> Though, for the asymmetry, most stateful decoders won't allow delete bufs on
> capture, because the buffers are registered in the firmware ahead of time. wave5
> can't even implement create_bufs on capture. We had an argument about having
> create_bufs on only one queue for that specific driver, as we wanted something
> upstream, we flex to removing create bufs completely. I think the all or nothing
> rule on per queue create/delete_bufs is not aligned with the reality.

I think the default should be that it supports DELETE_BUFS for both queues, but
it should still be possible to only have it on one queue.

When v18 is posted I want to play around with that, because I am not certain
what the easiest way is to implement this.

Another thing that needs to be added is a check that DELETE_BUFS is only enabled
if CREATE_BUFS is also present, it makes no sense otherwise.

Finally I want to take another look at the work required to make a CREATE_BUFS
replacement since that ioctl is horrible. Whether that will become part of this
series or done as a follow-up I am not sure about, but this series should definitely
make it possible to cleanly integrate it.

Regards,

Hans

>
> Nicolas
>>
>>>
>>>>
>>>> return vb2_queue_init(dst_vq);
>>>> }
>>>> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
>>>> index 941fa23c211a..34eab90e8a42 100644
>>>> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c
>>>> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
>>>> @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
>>>> .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
>>>> .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
>>>> .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
>>>> + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs,
>>>> .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
>>>>
>>>> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>>> In my view setting vidioc_delete_bufs should enable this feature, and if
>>> for some strange reason only one queue support it, then make a wrapper
>>> callback that returns an error when used with the wrong queue.
>>>
>>> Also note that patch 6/8 never checks for q->supports_delete_bufs in
>>> vb2_core_delete_bufs(), which is wrong!
>>
>> I will fix that in next version.
>> Regards,
>> Benjamin
>>
>>>
>>> Regards,
>>>
>>> Hans
>>>
>>
>