Message-ID: <a77c2fca8bd1e95e1948d3fa98fe106c228221c7.camel@ndufresne.ca>
Subject: Re: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver
From:   Nicolas Dufresne <nicolas@ndufresne.ca>
To:     Ming Qian <ming.qian@nxp.com>,
        "mchehab@kernel.org" <mchehab@kernel.org>,
        "shawnguo@kernel.org" <shawnguo@kernel.org>,
        "robh+dt@kernel.org" <robh+dt@kernel.org>,
        "s.hauer@pengutronix.de" <s.hauer@pengutronix.de>
Cc:     "hverkuil-cisco@xs4all.nl" <hverkuil-cisco@xs4all.nl>,
        "kernel@pengutronix.de" <kernel@pengutronix.de>,
        "festevam@gmail.com" <festevam@gmail.com>,
        dl-linux-imx <linux-imx@nxp.com>,
        Aisheng Dong <aisheng.dong@nxp.com>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>
Date:   Thu, 02 Dec 2021 23:09:59 -0500
In-Reply-To: <DB8PR04MB6346B64E4129D81BA0CF0A1CE7639@DB8PR04MB6346.eurprd04.prod.outlook.com>
References: <cover.1636445575.git.ming.qian@nxp.com>
         <9947131322e034bb6336802e5afb4b6132ca5071.camel@ndufresne.ca>
         <AM6PR04MB6341BF1FB2A839961DBADF4EE7619@AM6PR04MB6341.eurprd04.prod.outlook.com>
         <9b7fd6802c89c5d3bb6a42b44f13a90f6c3caf22.camel@ndufresne.ca>
         <AM6PR04MB6341F599A97253C1BA812467E7629@AM6PR04MB6341.eurprd04.prod.outlook.com>
         <0deb3fa15dd162d7c7b6dbe24821fdf9523dddd3.camel@ndufresne.ca>
         <DB8PR04MB6346B64E4129D81BA0CF0A1CE7639@DB8PR04MB6346.eurprd04.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.42.1 (3.42.1-1.fc35) 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

Le vendredi 26 novembre 2021 à 06:31 +0000, Ming Qian a écrit :
> > -----Original Message-----
> > From: Nicolas Dufresne [mailto:nicolas@ndufresne.ca]
> > Sent: Thursday, November 25, 2021 11:36 PM
> > To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
> > shawnguo@kernel.org; robh+dt@kernel.org; s.hauer@pengutronix.de
> > Cc: hverkuil-cisco@xs4all.nl; kernel@pengutronix.de; festevam@gmail.com;
> > dl-linux-imx <linux-imx@nxp.com>; Aisheng Dong <aisheng.dong@nxp.com>;
> > linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> > devicetree@vger.kernel.org; linux-arm-kernel@lists.infradead.org
> > Subject: Re: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder
> > driver
> > 
> > Caution: EXT Email
> > 
> > Le jeudi 25 novembre 2021 à 05:25 +0000, Ming Qian a écrit :
> > > For test [JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
> > > VPSSPSPPS_A_MainConcept_1, The vpu report an unsupported message to
> > driver, so driver report pollerr to gstreamer.
> > > But this stream can be decoded using the amphion vpu when I test it
> > > using our unit test, I checked the difference, there are many vps, sps
> > > and pps at the beginning, gstreamer will skip the first vpu and two
> > > pps, totally skip 56 bytes. It leds to vpu can't decode And our unit test
> > > won't
> > skip anthing, so the vpu can decode the stream.
> > 
> > This specific test triggers a bug in GStreamer HEVC parser, I'm aware of
> > this
> > one, and it is on my todo to fix (just not as trivial as it looks like, the
> > VCL nal
> > detection code was implemented wrong and that ended up leaking into the
> > rest of the design). This specific test will of course be marked and skipped
> > for CI
> > test that uses GStreamer.
> > 
> > regards,
> > Nicolas
> 
> Hi Nicolas,
> 
>     For the h264 decoder test, If I test with our unit test tool, the result
> is 124/135 tests successfully,
> But only 75/135 tests successfully when test using gstreamer.
>     There are 49 tests with different results, and all of the 49 test streams
> are interlaced stream.
> The amphion vpu will output the interlaced frame directly, it won't merge the
> interlaced two parts into one progressive frame.
>      And the gstreamer tiled unpack function(unpack_NV12_TILED) won't handle
> the interlaced case, so the output frame is abnormal, and led to test fail.
>      I think it should be a hardware limitation of the amphion vpu. And maybe

If this is the case, it needs to be signaled as ALTERNATE (if you have 1 v4l2
buffer per field) or SEQ if they are stored in the same v4l2_buffer but on top
of each other instead of interleaved.

Some bit about this might not we all wired properly, I have only tested
alternate field for capture devices (Xilinx ZinqMP). Please check the signalling
of interlace field in your driver, I'll check the state of GStreamer in this
regard.

> the gstreamer videoconvert can handle it.
> The interlaced list is as below:
> [JVT-AVC_V1] cabac_mot_fld0_full
> [JVT-AVC_V1] cabac_mot_mbaff0_full
> [JVT-AVC_V1] cabac_mot_picaff0_full
> [JVT-AVC_V1] CABREF3_Sand_D
> [JVT-AVC_V1] CAFI1_SVA_C
> [JVT-AVC_V1] CAMA1_Sony_C
> [JVT-AVC_V1] CAMA1_TOSHIBA_B
> [JVT-AVC_V1] CAMA3_Sand_E
> [JVT-AVC_V1] CAMACI3_Sony_C
> [JVT-AVC_V1] CAMANL1_TOSHIBA_B
> [JVT-AVC_V1] CAMANL2_TOSHIBA_B
> [JVT-AVC_V1] CAMANL3_Sand_E
> [JVT-AVC_V1] CAMASL3_Sony_B
> [JVT-AVC_V1] CAMP_MOT_MBAFF_L30
> [JVT-AVC_V1] CAMP_MOT_MBAFF_L31
> [JVT-AVC_V1] CANLMA2_Sony_C
> [JVT-AVC_V1] CANLMA3_Sony_C
> [JVT-AVC_V1] CAPA1_TOSHIBA_B
> [JVT-AVC_V1] CAPAMA3_Sand_F
> [JVT-AVC_V1] cavlc_mot_fld0_full_B
> [JVT-AVC_V1] cavlc_mot_mbaff0_full_B
> [JVT-AVC_V1] cavlc_mot_picaff0_full_B
> [JVT-AVC_V1] CVCANLMA2_Sony_C
> [JVT-AVC_V1] CVFI1_Sony_D
> [JVT-AVC_V1] CVFI1_SVA_C
> [JVT-AVC_V1] CVFI2_Sony_H
> [JVT-AVC_V1] CVFI2_SVA_C
> [JVT-AVC_V1] CVMA1_Sony_D
> [JVT-AVC_V1] CVMA1_TOSHIBA_B
> [JVT-AVC_V1] CVMANL1_TOSHIBA_B
> [JVT-AVC_V1] CVMANL2_TOSHIBA_B
> [JVT-AVC_V1] CVMAPAQP3_Sony_E
> [JVT-AVC_V1] CVMAQP2_Sony_G
> [JVT-AVC_V1] CVMAQP3_Sony_D
> [JVT-AVC_V1] CVMP_MOT_FLD_L30_B
> [JVT-AVC_V1] CVMP_MOT_FRM_L31_B
> [JVT-AVC_V1] CVNLFI1_Sony_C
> [JVT-AVC_V1] CVNLFI2_Sony_H
> [JVT-AVC_V1] CVPA1_TOSHIBA_B
> [JVT-AVC_V1] FI1_Sony_E
> [JVT-AVC_V1] MR6_BT_B
> [JVT-AVC_V1] MR7_BT_B
> [JVT-AVC_V1] MR8_BT_B
> [JVT-AVC_V1] MR9_BT_B
> [JVT-AVC_V1] Sharp_MP_Field_1_B
> [JVT-AVC_V1] Sharp_MP_Field_2_B
> [JVT-AVC_V1] Sharp_MP_Field_3_B
> [JVT-AVC_V1] Sharp_MP_PAFF_1r2
> [JVT-AVC_V1] Sharp_MP_PAFF_2r
>