Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 2 Jun 2020 15:45:04 +0300
From:   Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To:     Helen Koike <helen.koike@collabora.com>
Cc:     kieran.bingham@ideasonboard.com,
        Dafna Hirschfeld <dafna.hirschfeld@collabora.com>,
        Kaaira Gupta <kgupta@es.iitr.ac.in>,
        Shuah Khan <skhan@linuxfoundation.org>,
        Mauro Carvalho Chehab <mchehab@kernel.org>,
        linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
        dafna Hirschfeld <dafna3@gmail.com>
Subject: Re: [PATCH] vimc: debayer: Add support for ARGB format
Message-ID: <20200602124504.GA12043@pendragon.ideasonboard.com>
References: <20200528185717.GA20581@kaaira-HP-Pavilion-Notebook>
 <0ab57863-935d-3ab5-dfea-80a44c63ae18@collabora.com>
 <20200601121626.GA13308@kaaira-HP-Pavilion-Notebook>
 <273a36d8-fc87-f9d4-0cf2-15beddf1661c@collabora.com>
 <f927c8e3-73de-598d-130d-97b5380579e5@collabora.com>
 <3b4c4447-677c-08b9-9366-95a012f8f018@ideasonboard.com>
 <cdcc42bf-b0dc-41b7-5104-eff8aa42feb2@collabora.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <cdcc42bf-b0dc-41b7-5104-eff8aa42feb2@collabora.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hello,

On Tue, Jun 02, 2020 at 08:31:26AM -0300, Helen Koike wrote:
> On 6/2/20 8:24 AM, Kieran Bingham wrote:
> > On 02/06/2020 11:55, Helen Koike wrote:
> >> On 6/2/20 7:52 AM, Dafna Hirschfeld wrote:
> >>> On 01.06.20 14:16, Kaaira Gupta wrote:
> >>>> On Fri, May 29, 2020 at 05:43:57PM +0200, Dafna Hirschfeld wrote:
> >>>>> Hi,
> >>>>> Thanks for the patch
> >>>>>
> >>>>> I don't know how real devices handle ARGB formats,
> >>>>> I wonder if it should be the part of the debayer.
> >>>>
> >>>> Hi! qcam tries to support BA24 as it is one of the formats that vimc
> >>>> lists as its supported formats wih --list-formats. Shouldn't BA24 be
> >>>> possible to capture with vimc?
> >>>
> >>> Hi,
> >>> Just to clarify, when listing the supported formats of a video node, the node lists
> >>> the formats that the video node as an independent media entity support.
> >>> It does not mean that the 'camera' as a whole (that is, the media topology graph) supports
> >>> all the formats that the video node lists. When interacting with a video node or
> >>> a subdevice node, one interacts only with that specific entity.
> >>> In the case of vimc, the RGB video node as an independent entity supports BA24 so the format
> >>> appears in the list of the its supported formats. But since the Debayer does not
> >>> support it, the format can not be generated by the entire vimc topology.
> >>> This is not a bug.

Is here a valid configuration for the vimc pipeline that produces BA24 ?
I agree that not all pipeline configurations need to support every
format, but we shouldn't report a format that can't be produced at all.

This being said, and as discussed before, the de-bayering subdev should
just produce MEDIA_BUS_FMT_RGB888_1X24, and the video node should then
implement the RGB pixel formats. BA24 should likely be one of the
supported formats (or maybe BX24 ?).

> >> This is also my understanding.
> >>
> >> You should have an -EPIPE error when start streaming though, it
> >> shouldn't fail silently.
> > 
> > Yes, we had -EPIPE, and that is what I think we were trying to resolve.
> > 
> > How would userspace be expected to detect what formats to use ? Should
> > the available formats on the capture node depend on the current linking
> > of the media graph?
> 
> This is a good question, I don't recall v4l2 API defining this.

A recent extension to VIDIOC_ENUMFMT allows enumerating pixel formats
for a given media bus code, I think that's the way forward.

> It would be a bit hard to implement in Vimc, specially when we have configfs
> for custom topology, since the capture would need to query all the pipeline.
> But could be implemented.
> 
> > Otherwise, to know what formats are supported - userspace must first
> > 'get a list of formats' then try to 'set' the formats to know what is
> > possible?
> 
> At the moment yes.
> 
> > Or should (given VIMC is quite specialist anyway) userspace 'just know'
> > what is capable all the same?
> > 
> > That's possibly fine, as we can simply remove support for the ARGB
> > formats from the libcamera pipeline handler if it is never expected to
> > be supported.
> 
> With the configfs feature, you could build a topology with sensor->capture,
> and ARGB would be supported.
> 
> > But then as a further question - what formats will we expect VIMC to
> > support? VIVID has a (very) wide range of formats.
> > 
> > Would we ever expect VIMC to be as configurable?
> > Or is the scope limited to what we have today?
> 
> I know it is very limited atm, but I would like to increase the range,
> I'm just with a limited bandwitdh to work on it.
>
> >>>>
> >>>> If yes, which entity should support it, if not debayer? Should there be
> >>>> a separate conversion entity, or should we keep the support in debayer
> >>>> itself for efficiency issues?
> >>>>
> >>>>> On 28.05.20 20:57, Kaaira Gupta wrote:
> >>>>>> Running qcam for pixelformat 0x34324142 showed that vimc debayer does
> >>>>>> not support it. Hence, add the support for Alpha (255).
> >>>>>
> >>>>> I would change the commit log to:
> >>>>>
> >>>>> Add support for V4L2_PIX_FMT_RGB24 format in the debayer
> >>>>> and set the alpha channel to constant 255.
> >>>>>
> >>>>>> Signed-off-by: Kaaira Gupta <kgupta@es.iitr.ac.in>
> >>>>>> ---
> >>>>>>    .../media/test-drivers/vimc/vimc-debayer.c    | 27 ++++++++++++-------
> >>>>>>    1 file changed, 18 insertions(+), 9 deletions(-)
> >>>>>>
> >>>>>> diff --git a/drivers/media/test-drivers/vimc/vimc-debayer.c b/drivers/media/test-drivers/vimc/vimc-debayer.c
> >>>>>> index c3f6fef34f68..f34148717a40 100644
> >>>>>> --- a/drivers/media/test-drivers/vimc/vimc-debayer.c
> >>>>>> +++ b/drivers/media/test-drivers/vimc/vimc-debayer.c
> >>>>>> @@ -62,6 +62,7 @@ static const u32 vimc_deb_src_mbus_codes[] = {
> >>>>>>        MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
> >>>>>>        MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA,
> >>>>>>        MEDIA_BUS_FMT_RGB888_1X32_PADHI,
> >>>>>> +    MEDIA_BUS_FMT_ARGB8888_1X32
> >>>>>>    };
> >>>>>>    static const struct vimc_deb_pix_map vimc_deb_pix_map_list[] = {
> >>>>>> @@ -322,15 +323,23 @@ static void vimc_deb_process_rgb_frame(struct vimc_deb_device *vdeb,
> >>>>>>        unsigned int i, index;
> >>>>>>        vpix = vimc_pix_map_by_code(vdeb->src_code);
> >>>>>> -    index = VIMC_FRAME_INDEX(lin, col, vdeb->sink_fmt.width, 3);
> >>>>>> -    for (i = 0; i < 3; i++) {
> >>>>>> -        switch (vpix->pixelformat) {
> >>>>>> -        case V4L2_PIX_FMT_RGB24:
> >>>>>> -            vdeb->src_frame[index + i] = rgb[i];
> >>>>>> -            break;
> >>>>>> -        case V4L2_PIX_FMT_BGR24:
> >>>>>> -            vdeb->src_frame[index + i] = rgb[2 - i];
> >>>>>> -            break;
> >>>>>> +
> >>>>>> +    if (vpix->pixelformat == V4L2_PIX_FMT_ARGB32) {
> >>>>>> +        index =  VIMC_FRAME_INDEX(lin, col, vdeb->sink_fmt.width, 4);
> >>>>>> +        vdeb->src_frame[index] = 255;
> >>>>>> +        for (i = 0; i < 3; i++)
> >>>>>> +            vdeb->src_frame[index + i + 1] = rgb[i];
> >>>>>> +    } else {
> >>>>>> +        index =  VIMC_FRAME_INDEX(lin, col, vdeb->sink_fmt.width, 3);
> >>>>>> +        for (i = 0; i < 3; i++) {
> >>>>>> +            switch (vpix->pixelformat) {
> >>>>>> +            case V4L2_PIX_FMT_RGB24:
> >>>>>> +                vdeb->src_frame[index + i] = rgb[i];
> >>>>>> +                break;
> >>>>>> +            case V4L2_PIX_FMT_BGR24:
> >>>>>> +                vdeb->src_frame[index + i] = rgb[2 - i];
> >>>>>> +                break;
> >>>>>> +            }
> >>>>>>            }
> >>>>>>        }
> >>>>>>    }

-- 
Regards,

Laurent Pinchart