by Chen-Yu Tsai

On Fri, Apr 12, 2019 at 12:57 AM Maxime Ripard
<[email protected]> wrote:
>
> Hi,
>
> On Fri, Apr 05, 2019 at 12:27:48PM -0400, Nicolas Dufresne wrote:
> > Le vendredi 05 avril 2019 à 17:15 +0200, Maxime Ripard a écrit :
> > > Hi Nicolas,
> > >
> > > On Thu, Apr 04, 2019 at 11:41:13AM -0400, Nicolas Dufresne wrote:
> > > > > > > + * - __u16
> > > > > > > + - ``pic_width_in_mbs_minus1``
> > > > > > > + -
> > > > > > > + * - __u16
> > > > > > > + - ``pic_height_in_map_units_minus1``
> > > > > > > + -
> > > > > >
> > > > > > We recently had some reflection with Alex that this is redundant with
> > > > > > the width and height in the OUTPUT format. It may also apply to some
> > > > > > other fields in these structs. I feel like they should be removed and
> > > > > > passed via corresponding generic V4L2 properties - format, selection,
> > > > > > etc.
> > > > > >
> > > > > > The same problem is also present in the MPEG2 controls. In fact, there
> > > > > > was a patch already which used some fields from the controls to
> > > > > > calculate the destination buffer strides, rather than bytesperline in
> > > > > > the format.
> > > > > >
> > > > > > Since we're in staging, it could be done with a follow-up patch, though.
> > > > >
> > > > > Just my two cents. I played with some codecs a while back. IIRC some
> > > > > specify a "codec" size in addition to the actual picture size, like
> > > > > when the encoder does padding to fit the requirements of the codec
> > > > > (spec). Is this needed anywhere?
> > > >
> > > > With state-less encoders, the headers, which contains the crop
> > > > information is created by userspace and for state less decoder, the
> > > > headers that contains this information is parsed by userspace. So I
> > > > believe that in theory, the accelerator does not strictly need to be
> > > > aware of the cropped dimensions.
> > > >
> > > > Another thing, is that there is not guarantied matches between e.g.
> > > > depth of the chrome/luma and the final image buffers. Some hardware may
> > > > have bandwidth limitation or internal converter and could possibly
> > > > decode 10bit data into 8bit buffers.
> > > >
> > > > A third reason why I would not try and encode this header information
> > > > is that there can be multiple PPS/SPS at the same time, and I think
> > > > it's confusing if the relevant information to differentiate them is
> > > > removed.
> > >
> > > Sorry if that sounds a bit dumb, but it's not really clear to me if
> > > you're arguing for the removal of the data as Tomasz suggests, or if
> > > you want to keep them.
> > >
> > > The first paragrah seems to advocate for the former, but the two
> > > others for the latter.
> >
> > I think the data should stay. As I said, there can be multiple SPS/PPS,
> > while there is only one format. That being said, how does SPS/PPS
> > activation works ? How do you tell the driver about all the SPS/PPS and
> > which one is being activated ?
>
> The current way of dealing with this is that the _SLICE_PARAMS control
> actually takes an array of SPS's. And same thing for PPS.
>
> There's no difference between activated and deactivated ones
> though. What is the use case for this?

Uhm, there is only one format, but so there is only one active
SPS/PPS, isn't it?

An alternative would be to just make the OUTPUT format a function of
the currently set controls, so that it would always match the active
PPS/SPS, without the ability for the application to change it.

I just want to avoid the case when the OUTPUT format can be set
randomly to something different than in the stream and also drivers
reading randomly the format-related information sometimes from the
controls and sometimes from the format state.

Best regards,
Tomasz

2019-04-16 11:56:36

by Nicolas Dufresne

[permalink] [raw]

Subject: Re: [PATCH RESEND v7 1/2] media: uapi: Add H264 low-level decoder API compound controls.

Le mardi 16 avril 2019 à 16:16 +0900, Tomasz Figa a écrit :
> On Fri, Apr 12, 2019 at 12:57 AM Maxime Ripard
> <[email protected]> wrote:
> > Hi,
> >
> > On Fri, Apr 05, 2019 at 12:27:48PM -0400, Nicolas Dufresne wrote:
> > > Le vendredi 05 avril 2019 à 17:15 +0200, Maxime Ripard a écrit :
> > > > Hi Nicolas,
> > > >
> > > > On Thu, Apr 04, 2019 at 11:41:13AM -0400, Nicolas Dufresne wrote:
> > > > > > > > + * - __u16
> > > > > > > > + - ``pic_width_in_mbs_minus1``
> > > > > > > > + -
> > > > > > > > + * - __u16
> > > > > > > > + - ``pic_height_in_map_units_minus1``
> > > > > > > > + -
> > > > > > >
> > > > > > > We recently had some reflection with Alex that this is redundant with
> > > > > > > the width and height in the OUTPUT format. It may also apply to some
> > > > > > > other fields in these structs. I feel like they should be removed and
> > > > > > > passed via corresponding generic V4L2 properties - format, selection,
> > > > > > > etc.
> > > > > > >
> > > > > > > The same problem is also present in the MPEG2 controls. In fact, there
> > > > > > > was a patch already which used some fields from the controls to
> > > > > > > calculate the destination buffer strides, rather than bytesperline in
> > > > > > > the format.
> > > > > > >
> > > > > > > Since we're in staging, it could be done with a follow-up patch, though.
> > > > > >
> > > > > > Just my two cents. I played with some codecs a while back. IIRC some
> > > > > > specify a "codec" size in addition to the actual picture size, like
> > > > > > when the encoder does padding to fit the requirements of the codec
> > > > > > (spec). Is this needed anywhere?
> > > > >
> > > > > With state-less encoders, the headers, which contains the crop
> > > > > information is created by userspace and for state less decoder, the
> > > > > headers that contains this information is parsed by userspace. So I
> > > > > believe that in theory, the accelerator does not strictly need to be
> > > > > aware of the cropped dimensions.
> > > > >
> > > > > Another thing, is that there is not guarantied matches between e.g.
> > > > > depth of the chrome/luma and the final image buffers. Some hardware may
> > > > > have bandwidth limitation or internal converter and could possibly
> > > > > decode 10bit data into 8bit buffers.
> > > > >
> > > > > A third reason why I would not try and encode this header information
> > > > > is that there can be multiple PPS/SPS at the same time, and I think
> > > > > it's confusing if the relevant information to differentiate them is
> > > > > removed.
> > > >
> > > > Sorry if that sounds a bit dumb, but it's not really clear to me if
> > > > you're arguing for the removal of the data as Tomasz suggests, or if
> > > > you want to keep them.
> > > >
> > > > The first paragrah seems to advocate for the former, but the two
> > > > others for the latter.
> > >
> > > I think the data should stay. As I said, there can be multiple SPS/PPS,
> > > while there is only one format. That being said, how does SPS/PPS
> > > activation works ? How do you tell the driver about all the SPS/PPS and
> > > which one is being activated ?
> >
> > The current way of dealing with this is that the _SLICE_PARAMS control
> > actually takes an array of SPS's. And same thing for PPS.
> >
> > There's no difference between activated and deactivated ones
> > though. What is the use case for this?
>
> Uhm, there is only one format, but so there is only one active
> SPS/PPS, isn't it?

Yes, and the active one can only change on frame boundary.

> An alternative would be to just make the OUTPUT format a function of
> the currently set controls, so that it would always match the active
> PPS/SPS, without the ability for the application to change it.
>
> I just want to avoid the case when the OUTPUT format can be set
> randomly to something different than in the stream and also drivers
> reading randomly the format-related information sometimes from the
> controls and sometimes from the format state.

Understood.

>
> Best regards,
> Tomasz

Attachments:

signature.asc (201.00 B)
This is a digitally signed message part