From:   "Wu, Wentong" <wentong.wu@intel.com>
To:     Sakari Ailus <sakari.ailus@linux.intel.com>
CC:     "mchehab@kernel.org" <mchehab@kernel.org>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "Pandruvada, Srinivas" <srinivas.pandruvada@intel.com>,
        "pierre-louis.bossart@linux.intel.com" 
        <pierre-louis.bossart@linux.intel.com>,
        "Wang, Zhifeng" <zhifeng.wang@intel.com>,
        "Ye, Xiang" <xiang.ye@intel.com>,
        "Qiu, Tian Shu" <tian.shu.qiu@intel.com>,
        "Cao, Bingbu" <bingbu.cao@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Hans de Goede <hdegoede@redhat.com>
Subject: RE: [PATCH v2 3/3] media: pci: intel: ivsc: Add acquire/release API
 for ivsc
Thread-Topic: [PATCH v2 3/3] media: pci: intel: ivsc: Add acquire/release API
 for ivsc
Thread-Index: AQHZP1Hum/btr+ww/kiynrKipZgIc67OndeAgBVeV6CAACETAIABdr6w
Date:   Wed, 1 Mar 2023 07:26:40 +0000
Message-ID: <DM6PR11MB43167D398FE7F280C54140B28DAD9@DM6PR11MB4316.namprd11.prod.outlook.com>
References: <20230213022347.2480307-1-wentong.wu@intel.com>
 <20230213022347.2480307-4-wentong.wu@intel.com>
 <Y+uxbQi7seGf+adP@kekkonen.localdomain>
 <DM6PR11MB43169FAE93274F64AA7E06F28DAC9@DM6PR11MB4316.namprd11.prod.outlook.com>
 <Y/26BpJ15rpSAypW@kekkonen.localdomain>
In-Reply-To: <Y/26BpJ15rpSAypW@kekkonen.localdomain>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?1lSGbe1RGqK/SHQ0yEs2/rlUn4Av5pdhxFgbnZEzim19SKXYKDspH7MSixqY?=
 =?us-ascii?Q?xWs1HDwfT0AbsnvaAUrSGNoY+K66kvHHtqmt+WhDkx0nfEgtfVB751xIb7hr?=
 =?us-ascii?Q?matoGzyHgaPqngT2XAS1hHbXybxJTp73SBS3wc/u2QG14B/c0GfoNhI+LOA4?=
 =?us-ascii?Q?lvehvRPjOYDkSlIS9wMtdX0xcadCZ5AB5JD9vH+Hau5xDa3dItW5Rf36gFx/?=
 =?us-ascii?Q?8MreNtsjevSj81QNES/lcsYA4Q2Ac4NB7lhUuLp+lIRSXfeNny6/0G+78iGx?=
 =?us-ascii?Q?bJTSomelO25ys/Nj5G0fI0yxyX4+eDaYHB9a60WaN0oSWC6T8ufwrGDDuHZJ?=
 =?us-ascii?Q?Tr3uxEwNXbNQd2Fuw7UyVmYDUNs7lNaY3momdxXiMH0PhSg8PiPBRQlvp15a?=
 =?us-ascii?Q?uQwh3RaIQKVaCrmsCcFu3m78V9goNT8IDaaB8YN/9J4YQQ5lihO5hh2L342D?=
 =?us-ascii?Q?6HlXHMCnpKz9fDd9Pn39gi6K6KePWRlYiDlHXCvzx0L0g7xgPDo6HdXOv5xy?=
 =?us-ascii?Q?w9eR5pcL8/pj2A3uK8zU9z+tT/TetVRZ2BmZhiNnsuUns2cy56F9WVvXB/CY?=
 =?us-ascii?Q?vNBtJqHHWpG2QfxqBj7vmwYCy1gFjhxhawgB39VaA+1cW85r1LCBv2wKp6NL?=
 =?us-ascii?Q?MjQg8T8ZBkVXicLk2k/XOoYaVwT4GIIKmYPftDk5E18+mcQ7lFVQIt12GlwD?=
 =?us-ascii?Q?6ZF95jvt0wwc34aP+HGlfI9r28NU/fQEq4IDTAozyCH33LNrEbIqFiwubirW?=
 =?us-ascii?Q?AeUMYyr1kbZtNpc06kPD6X5kyeSB1AMZ+LoaOrCkKFNlN8tOtvIQZ3dcXSPy?=
 =?us-ascii?Q?BrTn3TDDfxfs/e5p3/QEuZq6P83yegC2M5CvWtGW6nl+771elBzLHj4UJc/q?=
 =?us-ascii?Q?QGUr1AyF+48XLfzxAWfWokYA1SPrWqFdJCazWU8RKXHYWtIcrE7yytpYfuK3?=
 =?us-ascii?Q?C1lm1PQPKqSpJ7UmYkenaM2mIfh77Ras81kSnQFPdf26RsP4f1GDW5DwF66Z?=
 =?us-ascii?Q?b/IM6i34J74hRHFlKL0/wSIPV4Nln5y8gPrnFrXvo4OzI0kH0CS3/TeMmGvw?=
 =?us-ascii?Q?9OPz3OksiHdVlVDilwRhGb7afZKgiQb3+jHzl4NU5PWTdgf/sTJjBpBNKPAg?=
 =?us-ascii?Q?eZp+Yp2+IIYey7GdRd4CrwBjEu+zb25qEb50ap/EeBAMT0LIpNDoeCsp1hjc?=
 =?us-ascii?Q?T8uGgWkNx9MiFHIGOIHbwQqdsJ4Q0j9tJuKcBeaj/5tjLyIvidjBGwVis+kc?=
 =?us-ascii?Q?wAH6pLNErO3JisKUTxgtPmqFfQaC2+veVXOppTDUAWqTHR6er0A7I4WapCH4?=
 =?us-ascii?Q?/J9xOPNNrJ2oyVxhRA+Ex3htvCTb8csoqRisF9qzK6fa5Nb6yLyNb3cvEJ//?=
 =?us-ascii?Q?IomZMW9bDhhTH6V0w/myfBGosANC+J1PbAVNzQk5Xrrh3wPp5XkAB6ac1Ny2?=
 =?us-ascii?Q?jtSJUdl2auXyXhe3kInDbM2n3DIid+XOZXw2qQ08UZ8VACnXUTcDUJlj11Jj?=
 =?us-ascii?Q?a4Bu2YuBX7EI9bddVLC1zVsSW6+JL6vOSf+1lVZnERIZwC1z9IjNH+u2rSGq?=
 =?us-ascii?Q?Nb+Rv2s71HXpUKerB6j84h+2HceCrOVozZpD2gjM?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4316.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 319adfb6-1413-4f4d-a895-08db1a264d1e
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Mar 2023 07:26:40.7678
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dMiQAoYCaCc4KAnSQigG72fYxytO/dARooLpw7LXJaIKUlV/iIfbj/KIrNEzKeTRpVSJNpTLIawPeuW7iOLWeA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> Sent: Tuesday, February 28, 2023 4:24 PM
>=20
> Hi Wentong,
>=20
> On Tue, Feb 28, 2023 at 06:35:41AM +0000, Wu, Wentong wrote:
> > Hi Sakari,
> >
> > few questions as switching to v4l2 sub-dev framework.
> >
> > > -----Original Message-----
> > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Sent: Wednesday, February 15, 2023 12:06 AM
> > >
> > > Hi Wentong,
> > >
> > > Thanks for the patchset.
> > >
> > > On Mon, Feb 13, 2023 at 10:23:47AM +0800, Wentong Wu wrote:
> > > > IVSC directly connects to camera sensor on source side, and on
> > > > output side it not only connects ISH via I2C, but also exposes
> > > > MIPI CSI-2 interface to output camera sensor data. IVSC can use
> > > > the camera sensor data to do AI algorithm, and send the results to
> > > > ISH. On the other end, IVSC can share camera sensor to host by
> > > > routing the raw camera sensor data to the exposed MIPI CSI-2
> > > > interface. But they can not work at the same time, so software APIs=
 are
> defined to sync the ownership.
> > > >
> > > > This commit defines the interfaces between IVSC and camera sensor
> > > > driver in include/linux/ivsc.h. The camera driver controls
> > > > ownership of the CSI-2 link and sensor with the acquire/release
> > > > APIs. When acquiring camera, lane number and link freq are also
> > > > required by IVSC frame router.
> > >
> > > The more I learn about this system, the more I'm inclined to think
> > > this functionality should be exposed as a V4L2 sub-device. IVSC
> > > doesn't really do anything to the data (as long as it directs it
> > > towards the CSI-2 receiver in the SoC), but it is definitely part of =
the image
> pipeline.
> > >
> > > I suppose the intended use cases assume a single instance of IVSC
> > > (as well as
> > > MEI) but there can, and often are, be multiple camera sensors in the
> > > system. The decision whether to request pass-through from IVCS can't
> > > be done in the camera sensor driver, and should not be visible to
> > > the camera sensor driver. Exposing IVSC as a V4L2 sub-device makes
> > > this trivial to address, as the IVSC driver's V4L2 sub-device video s=
_stream()
> operation gets called before streaming is started.
> > >
> > > The information whether IVSC is found between the camera sensor and
> > > the host's CSI-2 receiver (IPU in this case) should come from system
> > > firmware and accessed most probably by what is called cio2-bridge at =
the
> moment.
> > >
> > > The privacy status can be a V4L2 control.
> >
> > This should be a control or event? If control, how user-space handle
> > privacy stuff?
>=20
> Changing control events generates events for the user space.
>=20
> <URL:https://hverkuil.home.xs4all.nl/spec/userspace-api/v4l/dev-event.htm=
l>

Ok, V4L2_EVENT_CTRL event which reports 'struct v4l2_event_ctrl' data to us=
er space

>=20
> >
> > For the required link freq and lane number, is v4l2 control the
> > correct way to configure them? If yes, seems there is no CID value for
> > them so that we should custom some CID value(link freqm, lane number,
> > and
> > privacy) for ivsc in linux/v4l2-controls.h, is this acceptable?
>=20
> You should obtain these using the V4L2 fwnode interface. Please see e.g.
> drivers/media/platform/renesas/rzg2l-cru/rzg2l-csi2.c how that driver reg=
isters
> a V4L2 async sub-device and a V4L2 async notifier.

Ok, something like v4l2_ctrl_find(source->ctrl_handler, V4L2_CID_LINK_FREQ)=
; is to get sensor's link
frequency, and the code like 'v4l2_subdev_call(source, pad, get_mbus_config=
, source, &mbus_config);
num_of_lanes =3D mbus_config.bus.mipi_csi2.num_data_lanes;' is to get senso=
r's lane number.

BR,
Wentong
>=20
> --
> Kind regards,
>=20
> Sakari Ailus