From:   "Wu, Wentong" <wentong.wu@intel.com>
To:     Sakari Ailus <sakari.ailus@linux.intel.com>
CC:     "mchehab@kernel.org" <mchehab@kernel.org>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "Pandruvada, Srinivas" <srinivas.pandruvada@intel.com>,
        "pierre-louis.bossart@linux.intel.com" 
        <pierre-louis.bossart@linux.intel.com>,
        "Wang, Zhifeng" <zhifeng.wang@intel.com>,
        "Ye, Xiang" <xiang.ye@intel.com>,
        "Qiu, Tian Shu" <tian.shu.qiu@intel.com>,
        "Cao, Bingbu" <bingbu.cao@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Hans de Goede <hdegoede@redhat.com>
Subject: RE: [PATCH v2 3/3] media: pci: intel: ivsc: Add acquire/release API
 for ivsc
Thread-Topic: [PATCH v2 3/3] media: pci: intel: ivsc: Add acquire/release API
 for ivsc
Thread-Index: AQHZP1Hum/btr+ww/kiynrKipZgIc67OndeAgBVeV6A=
Date:   Tue, 28 Feb 2023 06:35:41 +0000
Message-ID: <DM6PR11MB43169FAE93274F64AA7E06F28DAC9@DM6PR11MB4316.namprd11.prod.outlook.com>
References: <20230213022347.2480307-1-wentong.wu@intel.com>
 <20230213022347.2480307-4-wentong.wu@intel.com>
 <Y+uxbQi7seGf+adP@kekkonen.localdomain>
In-Reply-To: <Y+uxbQi7seGf+adP@kekkonen.localdomain>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?l9f0W3Bam1CWw5q5I6RE5liW+eEoTEDeKNrqSHp2GbpY2yCVLKqghwHpkpZa?=
 =?us-ascii?Q?d3PGLfNLysdIXJCOrz10e0KosG+H+PczcNwqLdg41nSbvKao+U2bObzhcCg1?=
 =?us-ascii?Q?H2zhGOYCGI42OuxNVDoav9O9v+Mprj9N6ueriKPQLjUhogjFkoVifUFXJXxc?=
 =?us-ascii?Q?X9Kz09/E/zUSNa4f2vrgq8+vRW9rPMK789L3g4MWRCzytfyAd4DIYKVgcSxd?=
 =?us-ascii?Q?+2to1mkpCZvQKAaaya4nYik3c3DIWLK8XGn6uZRomlslslG3iaW0LXKxtQh9?=
 =?us-ascii?Q?7o5Slu0grdFFJjwGdX/TB3TPkC6FVyfXJszAnw7aK6pa/VvjPTubiB/sj0SE?=
 =?us-ascii?Q?84Po9HYoq8QDSo9TTx+Crs1jCeGiRIQP6n7JUNPUjMAzav433mRjGYS4/C2x?=
 =?us-ascii?Q?geZDIKHEA1vae5VYspkyd1hPncB6zhXZlUXb1MB0z7n/ieIUNLGNvi/XG4p4?=
 =?us-ascii?Q?36fes62PBL7PgjmPcerlR/YB+9lMLL5028k4XFwUtpsJZETZpDM/4Ll651xE?=
 =?us-ascii?Q?dZ5xt72bkmdsa+N62PjZ33W76SrtkGyIhUj4uLmsSr4orqGCJYaDIV59s4uZ?=
 =?us-ascii?Q?W9NTlMbfoVyyTcQ+lLvW4/HoOZzsi+sKhiy3cxDELZUw+xYPvyZXPrAqZHkd?=
 =?us-ascii?Q?Gs8xhSkfq9IAubAWVMgt72nWa8ImPYzeJL/5F0gw3g3LvFSqokkLnbvc38XA?=
 =?us-ascii?Q?DWeAUtGAiT8g9d8FVgeIFxPX0+XT4hBmy9BydUnishzFVT9q1uu04XX3NFHt?=
 =?us-ascii?Q?8vculjgKAWO2GjTi3c4cw/8IGCSemnUuhc+WKlqGoBW6kwJ4GWMcgtKOuD8e?=
 =?us-ascii?Q?FYcCQzi1hvDTpaLCtSZRX9IOCF61/Zy3LwYOeW17ZiaaeYHjn1i/9PIqdx97?=
 =?us-ascii?Q?7ty4ccuM2z+v5vwBghQxHK8qy8Dp2lUdOGGM0p3r8mNRkYbwZjhDe6WY/fFO?=
 =?us-ascii?Q?6HlEpgfEUfCJOY1bZQ2lSpWQg+FGmbkgGSkqpGf3EFvsgZ804NrJOxQG5ymC?=
 =?us-ascii?Q?+OQrUUkxQqWNoPP0TRreUNjYRJC9fHa1O0t91BjRhYPfUy+eJWWlXqXjlMZD?=
 =?us-ascii?Q?0zhks7/YM1GYwNjAL09sVXtiWhY17rMq4wzvW+rknf+R9Wi6eKVOIXIOgZDu?=
 =?us-ascii?Q?eWUczCZCLITzLmIDat3TBYcDCTtqbburJZMKXPTj8UN0srWAB3ev59fEZ/i9?=
 =?us-ascii?Q?r6YJ4r0bpCiFFjDECqWdA4yYTn979fieblMeNz4vJRPhkEOKPDjvuMUFpe4Z?=
 =?us-ascii?Q?6YarocM7ZdVUXf8vv1d3BdXGcj2nr76OLVcE4et1Q1ksQ/IjkVPYu2YHZLbc?=
 =?us-ascii?Q?4UOmPDh2PVNqsBezb04OahIh80631YSaBHsNf1QYyrMKEJZeIb8d98rNCHfy?=
 =?us-ascii?Q?PKhQNzmwpE5Wj3SiRv5f2tFs6Yy6Ic3MxhWRmmcxOzu82TZ0Kp/FAWavlgBO?=
 =?us-ascii?Q?JaCC6NR8N5rlP/YpY0qfM/FgccdnPaZ5y84lj7ZoSgzI5whYc0ZpCDJsBM3q?=
 =?us-ascii?Q?OSq2MoDUBHMz42e+6C1rx3ROsPUWwfhBIqm8knLEKc8rAvulo7boaz0LHCfX?=
 =?us-ascii?Q?ho7qQxG0QJb4SZXHGy6A2v2nUpdDtXt7kW9PGAHX?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4316.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d394c9d5-d42c-4dff-4b32-08db19560366
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Feb 2023 06:35:41.7662
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2koSEh1mUsjwlB+MvUPv19kVrmW0kjO02cKCuSOgzDZrfM6LVtqE+n2kBC4NgvZq3THlxLAyn2cgomjLuTLQCQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB7300
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Sakari,

few questions as switching to v4l2 sub-dev framework.

> -----Original Message-----
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> Sent: Wednesday, February 15, 2023 12:06 AM
>=20
> Hi Wentong,
>=20
> Thanks for the patchset.
>=20
> On Mon, Feb 13, 2023 at 10:23:47AM +0800, Wentong Wu wrote:
> > IVSC directly connects to camera sensor on source side, and on output
> > side it not only connects ISH via I2C, but also exposes MIPI CSI-2
> > interface to output camera sensor data. IVSC can use the camera sensor
> > data to do AI algorithm, and send the results to ISH. On the other
> > end, IVSC can share camera sensor to host by routing the raw camera
> > sensor data to the exposed MIPI CSI-2 interface. But they can not work
> > at the same time, so software APIs are defined to sync the ownership.
> >
> > This commit defines the interfaces between IVSC and camera sensor
> > driver in include/linux/ivsc.h. The camera driver controls ownership
> > of the CSI-2 link and sensor with the acquire/release APIs. When
> > acquiring camera, lane number and link freq are also required by IVSC
> > frame router.
>=20
> The more I learn about this system, the more I'm inclined to think this
> functionality should be exposed as a V4L2 sub-device. IVSC doesn't really=
 do
> anything to the data (as long as it directs it towards the CSI-2 receiver=
 in the
> SoC), but it is definitely part of the image pipeline.
>=20
> I suppose the intended use cases assume a single instance of IVSC (as wel=
l as
> MEI) but there can, and often are, be multiple camera sensors in the syst=
em. The
> decision whether to request pass-through from IVCS can't be done in the c=
amera
> sensor driver, and should not be visible to the camera sensor driver. Exp=
osing
> IVSC as a V4L2 sub-device makes this trivial to address, as the IVSC driv=
er's V4L2
> sub-device video s_stream() operation gets called before streaming is sta=
rted.
>=20
> The information whether IVSC is found between the camera sensor and the
> host's CSI-2 receiver (IPU in this case) should come from system firmware=
 and
> accessed most probably by what is called cio2-bridge at the moment.
>=20
> The privacy status can be a V4L2 control.

This should be a control or event? If control, how user-space handle privac=
y stuff?

For the required link freq and lane number, is v4l2 control the correct way=
 to configure
them? If yes, seems there is no CID value for them so that we should custom=
 some
CID value(link freqm, lane number, and privacy) for ivsc in linux/v4l2-cont=
rols.h, is this
acceptable?

Thanks,
Wentong
>=20
> Also cc Hans.
>=20
> >