Received: by 2002:a25:6193:0:0:0:0:0 with SMTP id v141csp886040ybb;
        Wed, 1 Apr 2020 11:26:38 -0700 (PDT)
X-Google-Smtp-Source: APiQypKwt2s4BZWqnqCn6W/q6QrLblW4oUwd+JXRrwCTjlZDEQcGqogpz82W2453HWrMvr/awIbu
X-Received: by 2002:aca:450a:: with SMTP id s10mr3693367oia.25.1585765597975;
        Wed, 01 Apr 2020 11:26:37 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1585765597; cv=none;
        d=google.com; s=arc-20160816;
        b=ZDPH1zMRdOyezdu1CDe1uhtwPwNJvJ+jhZkTZPay00Wybon0TqJTJn2n/hYTJO9lXO
         TmMNSy3IQwWf1ZjuPAR8fq0YAYC3NgraIyR80d3jhZ88756wP+Kb4vdMPKhDwm7Jbjjr
         FA7HYxgSJfZDSCIYhhnIYAIsF3JIfuqZZ3ZHTbRCzxmlPkSao8R5G26G5HmbtduVx+G2
         ocwIg0IZyfNv0ficWG8zvquInWWoSHclYoVYthQhoP17emil8PpNodRb0sIZmTPvwjvj
         9WVKhDAmirdbUb89Duis5lQLBRe6j1ABAxhusI/X0da14z5p+eU6ffBfBWwdPPFYtC26
         uWow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:sender:dkim-signature:content-language
         :content-transfer-encoding:in-reply-to:mime-version:user-agent:date
         :message-id:from:references:cc:to:subject;
        bh=wuMn7wVJU3FgOIzl/GEIXEsEfIt24oALd2phqNyAMuU=;
        b=OQojm8f7hsdQD69nwLR80BalwxUyjMEJbcEurfwH5B/aBi1nhjh9N7A/WBK0dS9ZgE
         1DudikQpN6q5JJFYAg+xRwC94Uai2yP5wBXyyQgCK1ChkWTsh5wpyBkfNPa/Gug0SLfN
         sQRKsMV4LNhLsH9mPIlGrklWjk8LLv95FUcMqELlQGU2CplYPPqoZpI69BzUxpBXHgt5
         FCYf/yZ4rbgTRjLjVAYtQJGQ99f5SCuGGkWvkjFWhWOk0jcYaOE6/5oFaGULn5h+UNdy
         /fjPYwmD0I62LWLqR6ok17DmhCTjORmMUFv2FCqBt0dE/zCmPto6ggkESMwrQHCCV0Q1
         YMSA==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=lXHTsCYC;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67])
        by mx.google.com with ESMTP id h27si1220352oos.21.2020.04.01.11.26.25;
        Wed, 01 Apr 2020 11:26:37 -0700 (PDT)
Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=lXHTsCYC;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1733021AbgDASYK (ORCPT <rfc822;jaegeuk-test@gmail.com>
        + 99 others); Wed, 1 Apr 2020 14:24:10 -0400
Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:6574 "EHLO
        hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1732842AbgDASYI (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 1 Apr 2020 14:24:08 -0400
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA)
        id <B5e84dc3a0001>; Wed, 01 Apr 2020 11:23:54 -0700
Received: from hqmail.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Wed, 01 Apr 2020 11:24:07 -0700
X-PGP-Universal: processed;
        by hqpgpgate101.nvidia.com on Wed, 01 Apr 2020 11:24:07 -0700
Received: from [10.2.164.193] (172.20.13.39) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 1 Apr
 2020 18:24:07 +0000
Subject: Re: [RFC PATCH v5 6/9] media: tegra: Add Tegra210 Video input driver
To:     Laurent Pinchart <laurent.pinchart@ideasonboard.com>
CC:     Hans Verkuil <hverkuil@xs4all.nl>,
        Sakari Ailus <sakari.ailus@iki.fi>, <thierry.reding@gmail.com>,
        <jonathanh@nvidia.com>, <frankc@nvidia.com>,
        <helen.koike@collabora.com>, <digetx@gmail.com>,
        <sboyd@kernel.org>, <linux-media@vger.kernel.org>,
        <devicetree@vger.kernel.org>, <linux-clk@vger.kernel.org>,
        <linux-tegra@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20200325110358.GB853@valkosipuli.retiisi.org.uk>
 <8bc44545-7d1e-0e37-db27-d37784679574@xs4all.nl>
 <20200331103215.GI2394@valkosipuli.retiisi.org.uk>
 <ba37eb84-392c-3767-57f6-d297b0ab79a3@xs4all.nl>
 <20200331111018.GJ2394@valkosipuli.retiisi.org.uk>
 <a1145ee4-2991-a958-1225-090c57fec533@xs4all.nl>
 <20200331115221.GA4767@pendragon.ideasonboard.com>
 <6aa7d86c-3943-d508-ccf6-5ac46546abe9@nvidia.com>
 <3b00a559-992a-2da9-92b1-bee44e137ba2@nvidia.com>
 <1c60491b-1bb2-6291-80a6-c0fa14094077@nvidia.com>
 <20200401165805.GE4876@pendragon.ideasonboard.com>
From:   Sowjanya Komatineni <skomatineni@nvidia.com>
Message-ID: <e3b437c6-76e1-d407-e81d-c05912ffcd0b@nvidia.com>
Date:   Wed, 1 Apr 2020 11:24:05 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <20200401165805.GE4876@pendragon.ideasonboard.com>
X-Originating-IP: [172.20.13.39]
X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To
 HQMAIL107.nvidia.com (172.20.187.13)
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1585765434; bh=wuMn7wVJU3FgOIzl/GEIXEsEfIt24oALd2phqNyAMuU=;
        h=X-PGP-Universal:Subject:To:CC:References:From:Message-ID:Date:
         User-Agent:MIME-Version:In-Reply-To:X-Originating-IP:
         X-ClientProxiedBy:Content-Type:Content-Transfer-Encoding:
         Content-Language;
        b=lXHTsCYCf9kAi+an+1CCXN4D5DsUO8HiWJpzEgXYoxpziUI5/YXGAyI1Ng63tZNCn
         sDs0BJ9TjF3gdM8GN3A+9bqGkrIkA6Z8AiNmuZfGC10Z6nWbFdiVzJnMn1r63JLe3H
         izLR++U3RQSfNX9oj0Hb9pXt+v6ZtuM7+RiRkS1glZ0lBj8qFD5pcDE78s6exDSfBg
         lEGGdM0Yt5F1eYR1AUSBFTNop+am26CRmaRpRqG/BODaiokrYxyD/IVYzDTLjyRXMI
         JZOrPsF+nU/7t7J09CKhH2wutN99cHMXyekMHIMjxRUiq7dUxPk7yzT6skCSAp3RJJ
         yUeiQGyxpgTlA==
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 4/1/20 9:58 AM, Laurent Pinchart wrote:
> External email: Use caution opening links or attachments
>
>
> Hi Sowjanya,
>
> On Wed, Apr 01, 2020 at 09:36:03AM -0700, Sowjanya Komatineni wrote:
>> Hi Sakari/Laurent,
>>
>> Few questions to confirm my understanding on below discussion.
>>
>> 1. Some sensors that you are referring as don't work with single devnode
>> controlling pipeline devices are ISP built-in sensors where setup of
>> pipeline and subdevices happen separately?
> Sensors that include ISPs could indeed require to be exposed as multiple
> subdevs, but I was mostly referring to raw Bayer sensors with hardware
> architectures similar to the SMIA++ and MIPI CCS specifications. Those
> sensors can perform cropping in up to three different locations (analog
> crop, digital crop, output crop), and can also scale in up to three
> different locations (binning, skipping and filter-based scaling).
>
> Furthermore, with the V4L2 support for multiplexed streams that we are
> working on, a sensor that can produce both image data and embedded data
> would also need to be split in multiple subdevs.

Thanks Laurent.

For sensors with meta/embedded data along with image in same frame, 
Tegra VI HW extracts based on programmed embedded data size info.

So in our driver we capture this as separate buffer as embedded data is 
part of frame.

You above comment on multiplexed streams is for sensors using different 
virutal channels for diff streams?


>> 2. With driver supporting single device node control of entire pipeline
>> devices compared to MC-based, limitation is with userspace apps for only
>> these complex camera sensors?
> In those cases, several policy decisions on how to configure the sensor
> (whether to use binning, skipping and/or filter-based scaling for
> instance, or how much cropping and scaling to apply to achieve a certain
> output resolution) will need to be implemented in the kernel, and
> userspace will not have any control on them.
>
>> 3. Does all upstream video capture drivers eventually will be moved to
>> support MC-based?
> I think we'll see a decrease of the video-node-centric drivers in the
> future for embedded systems, especially the ones that include an ISP.
> When a system has an ISP, even if the ISP is implemented as a
> memory-to-memory device separate from the CSI-2 capture side, userspace
> will likely have a need for fine-grained control of the camera sensor.
>
>> 4. Based on libcamera doc looks like it will work with both types of
>> MC-based and single devnode based pipeline setup drivers for normal
>> sensors and limitation is when we use ISP built-in sensor or ISP HW
>> block. Is my understanding correct?
> libcamera supports both, it doesn't put any restriction in that area.
> The pipeline handler (the device-specific code in libcamera that
> configures and control the hardware pipeline) is responsible for
> interfacing with the kernel drivers, and is free to use an MC-centric or
> video-node-centric API depending on what the kernel drivers offer.
>
> The IPA (image processing algorithms) module is also vendor-specific.
> Although it will not interface directly with kernel drivers, it will
> have requirements on how fine-grained control of the sensor is required.
> For systems that have an ISP in the SoC, reaching a high image quality
> level requires fine-grained control of the sensor, or at the very least
> being able to retrieve fine-grained sensor configuration information
> from the kernel. For systems using a camera sensor with an integrated
> ISP and a CSI-2 receiver without any further processing on the SoC side,
> there will be no such fine-grained control of the sensor by the IPA (and
> there could even be no IPA module at all).
>
> --
> Regards,
>
> Laurent Pinchart