Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <42ba6a9516f4359b757d2f94b16c1bb23cc41cb2.camel@ndufresne.ca>
Subject: Re: [EXT] Re: [PATCH] media: amphion: only insert the first
 sequence startcode for vc1l format
From:   Nicolas Dufresne <nicolas@ndufresne.ca>
To:     Ming Qian <ming.qian@nxp.com>,
        "mchehab@kernel.org" <mchehab@kernel.org>,
        "hverkuil-cisco@xs4all.nl" <hverkuil-cisco@xs4all.nl>
Cc:     "shawnguo@kernel.org" <shawnguo@kernel.org>,
        "robh+dt@kernel.org" <robh+dt@kernel.org>,
        "s.hauer@pengutronix.de" <s.hauer@pengutronix.de>,
        "kernel@pengutronix.de" <kernel@pengutronix.de>,
        "festevam@gmail.com" <festevam@gmail.com>,
        dl-linux-imx <linux-imx@nxp.com>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>
Date:   Tue, 05 Jul 2022 08:59:11 -0400
In-Reply-To: <AM6PR04MB634136824EC98EE804FAD0CEE7819@AM6PR04MB6341.eurprd04.prod.outlook.com>
References: <20220628052017.26979-1-ming.qian@nxp.com>
         <6e54af5243d324c5df1c9ec18d4b091fbd52150f.camel@ndufresne.ca>
         <AM6PR04MB6341F4EB028CAE9B61C85157E7819@AM6PR04MB6341.eurprd04.prod.outlook.com>
         <AM6PR04MB634136824EC98EE804FAD0CEE7819@AM6PR04MB6341.eurprd04.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.2 (3.44.2-1.fc36) 
MIME-Version: 1.0
Precedence: bulk

Le mardi 05 juillet 2022 =C3=A0 11:27 +0000, Ming Qian a =C3=A9crit=C2=A0:
> > > From: Ming Qian
> > > Sent: 2022=E5=B9=B47=E6=9C=885=E6=97=A5 10:00
> > > To: Nicolas Dufresne <nicolas@ndufresne.ca>; mchehab@kernel.org;
> > > hverkuil-cisco@xs4all.nl
> > > Cc: shawnguo@kernel.org; robh+dt@kernel.org; s.hauer@pengutronix.de;
> > > kernel@pengutronix.de; festevam@gmail.com; dl-linux-imx
> > > <linux-imx@nxp.com>; linux-media@vger.kernel.org;
> > > linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org
> > > Subject: RE: [EXT] Re: [PATCH] media: amphion: only insert the first
> > > sequence
> > > startcode for vc1l format
> > >=20
> > > > From: Nicolas Dufresne <nicolas@ndufresne.ca>
> > > > Sent: 2022=E5=B9=B47=E6=9C=885=E6=97=A5 0:06
> > > > To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
> > > > hverkuil-cisco@xs4all.nl
> > > > Cc: shawnguo@kernel.org; robh+dt@kernel.org; s.hauer@pengutronix.de=
;
> > > > kernel@pengutronix.de; festevam@gmail.com; dl-linux-imx
> > > > <linux-imx@nxp.com>; linux-media@vger.kernel.org;
> > > > linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org
> > > > Subject: [EXT] Re: [PATCH] media: amphion: only insert the first
> > > > sequence startcode for vc1l format
> > > >=20
> > > > Caution: EXT Email
> > > >=20
> > > > Hi Ming,
> > > >=20
> > > > Le mardi 28 juin 2022 =C3=A0 13:20 +0800, Ming Qian a =C3=A9crit :
> > > > > For some formats, the amphion vpu requires startcode before seque=
nce
> > > > > and frame, such as vc1, vp8.
> > > >=20
> > > > I'm not sure why VP8 is being mentioned here. There is no specified
> > > > sartcode for VP8, and no split of headers either.
> > > >=20
> > >=20
> > > Hi Nicolas,
> > > =C2=A0=C2=A0=C2=A0This patch has nothing to do with vp8, and indeed t=
here is no specified
> > > startcode for VP8.
> > > But amphion vpu requires driver to help insert some custom startcode =
for
> > > vp8
> > > and vc1.
> > > It's custom behavior.
> > >=20
> > Instead of exposing the custom format, you should use data_offset like =
VENUS
> > driver do.  They vp8/9 codec in VENUS have the frame prefixed with an I=
VF
> > header, the data_offset let the userland skip it.
>=20
> Hi Nicolas,
> =C2=A0=C2=A0=C2=A0=C2=A0There is a stream buffer, and driver will copy th=
e coded frame data to the
> stream buffer, driver can decide to insert custom startcode or not, userl=
and
> won't know about it, so I don't think it's necessary to use data_offset t=
o let
> userland skip something.
> =C2=A0=C2=A0=C2=A0=C2=A0Currently , driver will insert startcode for form=
at vp8 and vc1. This is
> transparent to the user.

Can't you save the slow copy by using data_offset then ? I think most of th=
e
confusion comes from this commit message, someone else then you should be a=
ble
to understand what it means.

regards,
Nicolas

>=20
> Ming
>=20
> > > =C2=A0=C2=A0=C2=A0I'm sorry that my description include some confusio=
n
> > >=20
> > > Ming
> > >=20
> > > > >=20
> > > > > But for V4L2_PIX_FMT_VC1_ANNEX_L, only the first sequence startco=
de
> > > > > is needed, the extra startcode will cause decoding error.
> > > > > So after seek, we don't need to insert the sequence startcode.
> > > > >=20
> > > > > In other words, for V4L2_PIX_FMT_VC1_ANNEX_L, the vpu doesn't
> > > > > support dynamic resolution change.
> > > > >=20
> > > > > Fixes: 145e936380edb ("media: amphion: implement malone decoder r=
pc
> > > > > interface")
> > > > > Signed-off-by: Ming Qian <ming.qian@nxp.com>
> > > > > ---
> > > > > =C2=A0drivers/media/platform/amphion/vdec.c       | 1 +
> > > > > =C2=A0drivers/media/platform/amphion/vpu.h        | 1 +
> > > > > =C2=A0drivers/media/platform/amphion/vpu_malone.c | 2 ++
> > > > > =C2=A0drivers/media/platform/amphion/vpu_rpc.h    | 7 ++++++-
> > > > > =C2=A04 files changed, 10 insertions(+), 1 deletion(-)
> > > > >=20
> > > > > diff --git a/drivers/media/platform/amphion/vdec.c
> > > > > b/drivers/media/platform/amphion/vdec.c
> > > > > index 09d4f27970ec..51218a41a5ac 100644
> > > > > --- a/drivers/media/platform/amphion/vdec.c
> > > > > +++ b/drivers/media/platform/amphion/vdec.c
> > > > > @@ -731,6 +731,7 @@ static void vdec_stop_done(struct vpu_inst *i=
nst)
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0vdec->eos_received =3D 0;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0vdec->is_source_changed =3D f=
alse;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0vdec->source_change =3D 0;
> > > > > +     inst->total_input_count =3D 0;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0vpu_inst_unlock(inst);
> > > > > =C2=A0}
> > > > >=20
> > > > > diff --git a/drivers/media/platform/amphion/vpu.h
> > > > > b/drivers/media/platform/amphion/vpu.h
> > > > > index e56b96a7e5d3..f914de6ed81e 100644
> > > > > --- a/drivers/media/platform/amphion/vpu.h
> > > > > +++ b/drivers/media/platform/amphion/vpu.h
> > > > > @@ -258,6 +258,7 @@ struct vpu_inst {
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct vpu_format cap_format;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0u32 min_buffer_cap;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0u32 min_buffer_out;
> > > > > +     u32 total_input_count;
> > > > >=20
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct v4l2_rect crop;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0u32 colorspace;
> > > > > diff --git a/drivers/media/platform/amphion/vpu_malone.c
> > > > > b/drivers/media/platform/amphion/vpu_malone.c
> > > > > index c62b49e85060..f4a488bf9880 100644
> > > > > --- a/drivers/media/platform/amphion/vpu_malone.c
> > > > > +++ b/drivers/media/platform/amphion/vpu_malone.c
> > > > > @@ -1314,6 +1314,8 @@ static int
> > > > vpu_malone_insert_scode_vc1_l_seq(struct malone_scode_t *scode)
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0int size =3D 0;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0u8 rcv_seqhdr[MALONE_VC1_RCV_=
SEQ_HEADER_LEN];
> > > > >=20
> > > > > +     if (scode->inst->total_input_count)
> > > > > +             return 0;
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0scode->need_data =3D 0;
> > > > >=20
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0ret =3D vpu_malone_insert_sco=
de_seq(scode,
> > > > > MALONE_CODEC_ID_VC1_SIMPLE, sizeof(rcv_seqhdr)); diff --git
> > > > > a/drivers/media/platform/amphion/vpu_rpc.h
> > > > > b/drivers/media/platform/amphion/vpu_rpc.h
> > > > > index 25119e5e807e..7eb6f01e6ab5 100644
> > > > > --- a/drivers/media/platform/amphion/vpu_rpc.h
> > > > > +++ b/drivers/media/platform/amphion/vpu_rpc.h
> > > > > @@ -312,11 +312,16 @@ static inline int vpu_iface_input_frame(str=
uct
> > > > vpu_inst *inst,
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0struct vb2_buffer *vb)  {
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0struct vpu_iface_ops *ops =3D=
 vpu_core_get_iface(inst->core);
> > > > > +     int ret;
> > > > >=20
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if (!ops || !ops->input_frame=
)
> > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0return -EINVAL;
> > > > >=20
> > > > > -     return ops->input_frame(inst->core->iface, inst, vb);
> > > > > +     ret =3D ops->input_frame(inst->core->iface, inst, vb);
> > > > > +     if (ret < 0)
> > > > > +             return ret;
> > > > > +     inst->total_input_count++;
> > > > > +     return ret;
> > > > > =C2=A0}
> > > > >=20
> > > > > =C2=A0static inline int vpu_iface_config_memory_resource(struct v=
pu_inst
> > > > > *inst,
>=20