From:   "Wu, Wentong" <wentong.wu@intel.com>
To:     Bingbu Cao <bingbu.cao@linux.intel.com>,
        Sakari Ailus <sakari.ailus@linux.intel.com>
CC:     Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
        "mchehab@kernel.org" <mchehab@kernel.org>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "Pandruvada, Srinivas" <srinivas.pandruvada@intel.com>,
        "pierre-louis.bossart@linux.intel.com" 
        <pierre-louis.bossart@linux.intel.com>,
        "Wang, Zhifeng" <zhifeng.wang@intel.com>,
        "Ye, Xiang" <xiang.ye@intel.com>,
        "Qiu, Tian Shu" <tian.shu.qiu@intel.com>,
        "Cao, Bingbu" <bingbu.cao@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v2 0/3] media: pci: intel: ivsc: Add driver of Intel
 Visual Sensing Controller(IVSC)
Thread-Topic: [PATCH v2 0/3] media: pci: intel: ivsc: Add driver of Intel
 Visual Sensing Controller(IVSC)
Thread-Index: AQHZP1HlDdnw2M/ymEOJ6dDL3JcZGa7PxSqAgAApAACAAaPNgIAA0d4AgABNrbA=
Date:   Fri, 17 Feb 2023 06:28:32 +0000
Message-ID: <DM6PR11MB4316F7C9854B113556466BF58DA19@DM6PR11MB4316.namprd11.prod.outlook.com>
References: <20230213022347.2480307-1-wentong.wu@intel.com>
 <Y+ypKYI4c6fHL4Eu@pendragon.ideasonboard.com>
 <00c5c16e-c94e-a328-06d9-5f09ad35258d@linux.intel.com>
 <Y+4rtSMArg5ow0Sh@kekkonen.localdomain>
 <8022b9da-e033-b77b-914c-d35a31aac2e2@linux.intel.com>
In-Reply-To: <8022b9da-e033-b77b-914c-d35a31aac2e2@linux.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?zuVPp8kk6fcrlLq2Urt7BkLmr3lSVE3WizUy81mvVTVWp+Y0YqrnCpIbY3Pf?=
 =?us-ascii?Q?XvYn8/9Kijibq+7fOSmwYVj7DhTm7g+Mi27dXz+xaqL2psjVUWdGY3XM7Quw?=
 =?us-ascii?Q?A0U5AnCSc39CeDtwNFXLohWlb0rmWcEbo+tqy+C7iSCDgE3G10Q043uv+tnD?=
 =?us-ascii?Q?P9q45jK+p06dJGLPbR+GaVvRkCP+e41nhNoqazhA71lOhVm45iLcSyvtQqid?=
 =?us-ascii?Q?ucrVCYEqAafJe9t+cwSxSQR9q2O6mZ9syJCB+pxRLEBshgS9N9dEGAJZb2YT?=
 =?us-ascii?Q?mtVvG/Wt4Vw125qJQVm7P4QyzlbUJUzerJdK99VPBOQqSwB/R7jqmsSCA/L+?=
 =?us-ascii?Q?ilqF42NpkuP/Oy4plsMRxtBor4wpVI/nJsgOu+p68oJGBWuvKuYjoz0Xjjfn?=
 =?us-ascii?Q?fPm/mqN3TZp2bqnZYR+3ikwOtZewOSqIS0h1d+PYol0P2SFMFl9NPv4B0lH6?=
 =?us-ascii?Q?Mq8GtZdhDY+F1FcfhbaINLJRM2Gu+yoqGEZPDMvj+RGKuVtreMfvMJhT/37c?=
 =?us-ascii?Q?hYK3SBhwUpBJpNTeRz7tVk91zlr5BYpVIG5g/R6/wNAM6X4tdYjNhEb9rgy6?=
 =?us-ascii?Q?Tl0nu5rFmg54mbMwWAb7yY5ockl9mjtJeIpp8Nz/g/0h3mJ+mr7tY8okDiX5?=
 =?us-ascii?Q?eJgsN39rlRhW40rwiWcnzd5WRIubaMr971nj4ZbT7yyw4BtaMa0UDe+td/NN?=
 =?us-ascii?Q?Pi1BJsv6/DHZMgCrHO7dJfvCsCSkKfTKcbeoGAdG+Ks2YKkzy/Xm8oAZFS4e?=
 =?us-ascii?Q?uDTxIpBimhuCiP8Hb5LN/jDVWfnZqOXt4dAMqOQ+RI1T8/vL0/OqYvQpvG3c?=
 =?us-ascii?Q?DbUoGapl+xYHKGiPfgGzfSaqFP58lI5nJsUSGch/NBQYip72CuC2rH5bKWuZ?=
 =?us-ascii?Q?qtKCAMW/SHrVi9R+8ESkEwntQpDgubcgmrklWRJVm+G5eJnK+WyTKWTes8jX?=
 =?us-ascii?Q?jPLSNSg1CC2nv5oWHZeN+BbS3m4Nk5tf362UNkCQXOnJpJSZZx3BCz8dVSs+?=
 =?us-ascii?Q?jldRSzDJkJS1zyCO6vB/7mRpwSY1ywSEblf5yTlGznzxVVHefWZCT3WkO3eU?=
 =?us-ascii?Q?6IXisjvg/PQBO7z36AAu+dWt58A0eIO+4pg5GicbVacrlnFJjx+Bi45eXS8h?=
 =?us-ascii?Q?zzLzLIHUVy6Im+l5EzhBsSNAlcyHV8fBlOB5t+97GvnozNjGY4ErTfX22PLd?=
 =?us-ascii?Q?dIVFDmAKIcXfh/WbqkyVQRiF8Xjr4yHDIUCI1+/YU7LffZqwZ8ABKHT/GIhj?=
 =?us-ascii?Q?Adsw0+3JNwAVa3fVvsMHof/Lxfl2i3NECV9JOj8/Gu9vnbjxWhHH5glGPgTX?=
 =?us-ascii?Q?iMGzxN7TFDOd5gWrZslinYJxKaz9aoh4qJUFw3e9wqCtRjOoDjkK2KVp2z4T?=
 =?us-ascii?Q?eX73gULQ3pq30QOl2ws9nMRvJdkgxi3f3ps84583mRGc2pG+dOoLDN9QQHxD?=
 =?us-ascii?Q?eXWPl0ShNEoKssmSkA/4usybn3OYH2IrZnFD+fKFugY0Ps6OWGj1j8paYoiN?=
 =?us-ascii?Q?HTWWrtv0aO5t8R1SbakFWjifnkdlruQU7kaEHqgzdW2/LoCqqasjrVCU38Ui?=
 =?us-ascii?Q?BV1O59MsTky6J45kedBBitUwB6m+je5CluzySSMR?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4316.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8533c315-0243-4a55-b459-08db10b030fd
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Feb 2023 06:28:32.4653
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ZqAjsqyXFUdNdwPuwtqOCXA61WHdEG3lbPMY/Wo6HQx+wS46cuprxUFmHB7rhG0QWL4PCGart8LGWm0SmbOl3A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR11MB6000
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Bingbu Cao <bingbu.cao@linux.intel.com>
> Sent: Friday, February 17, 2023 9:44 AM
>=20
> Hi, Sakari,
>=20
> On 2/16/23 9:12 PM, Sakari Ailus wrote:
> > Hi Bingbu, Wentong,
> >
> > On Wed, Feb 15, 2023 at 08:09:50PM +0800, Bingbu Cao wrote:
> >>
> >> Hi, Wentong,
> >>
> >> On 2/15/23 5:43 PM, Laurent Pinchart wrote:
> >>> Hello Wentong,
> >>>
> >>> On Mon, Feb 13, 2023 at 10:23:44AM +0800, Wentong Wu wrote:
> >>>> Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls",
> >>>> is a companion chip designed to provide secure and low power vision
> >>>> capability to IA platforms. IVSC is available in existing
> >>>> commercial platforms from multiple OEMs.
> >>>>
> >>>> The primary use case of IVSC is to bring in context awareness. IVSC
> >>>> interfaces directly with the platform main camera sensor via a
> >>>> CSI-2 link and processes the image data with the embedded AI
> >>>> engine. The detected events are sent over I2C to ISH (Intel Sensor
> >>>> Hub) for additional data fusion from multiple sensors. The fusion
> >>>> results are used to implement advanced use cases like:
> >>>>  - Face detection to unlock screen
> >>>>  - Detect user presence to manage backlight setting or waking up
> >>>> system
> >>>
> >>> Do you have plan to support these features in the ivsc driver in the
> >>> future ?
> >>>
> >>>> Since the Image Processing Unit(IPU) used on the host processor
> >>>> needs to configure the CSI-2 link in normal camera usages, the
> >>>> CSI-2 link and camera sensor can only be used in mutually-exclusive
> >>>> ways by host IPU and IVSC. By default the IVSC owns the CSI-2 link
> >>>> and camera sensor. The IPU driver can take ownership of the CSI-2
> >>>> link and camera sensor using interfaces provided by this IVSC driver=
.
> >>>>
> >>>> Switching ownership requires an interface with two different
> >>>> hardware modules inside IVSC. The software interface to these
> >>>> modules is via Intel MEI (The Intel Management Engine) commands.
> >>>> These two hardware modules have two different MEI UUIDs to enumerate=
.
> These hardware modules are:
> >>>>  - ACE (Algorithm Context Engine): This module is for algorithm
> >>>> computing when IVSC owns camera sensor. Also ACE module controls
> >>>> camera sensor's ownership. This hardware module is used to set owner=
ship
> of camera sensor.
> >>>>  - CSI (Camera Serial Interface): This module is used to route
> >>>> camera sensor data either to IVSC or to host for IPU driver and appl=
ication.
> >>>>
> >>>> IVSC also provides a privacy mode. When privacy mode is turned on,
> >>>> camera sensor can't be used. This means that both ACE and host IPU
> >>>> can't get image data. And when this mode is turned on, host IPU
> >>>> driver is informed via a registered callback, so that user can be no=
tified.
> >>>
> >>> How does the privacy mode work, and how can the user trust that the
> >>> closed-source IVSC and IME firmwares will honour the privacy settings=
 ?
>=20
> As I know, without IVSC, once user enable the privacy mode, the Intel
> Converged Security Engine will configure the IPU camera mask (security re=
gister),
> which will mask the specific CSI2 port and produce dummy imaging data. Fo=
r the
> case with IVSC, there is no final solution on Linux so far I think.
>=20
> Wentong, is IVSC trying to cut off the stream and then notify user and IP=
U?

yes

>=20
> >>>
> >>
> >> Continue with question from Laurent,
> >>
> >> How IVSC handle the privacy request from user? Is there some
> >> notification mechanism to user-space?

IVSC has already defined privacy callback for host IPU/camera driver.

> > I'd have concern if IVSC driver
> >> need private callback to request back-end(e.g. ISP driver) to handle s=
tream
> cutting.
> >
> > How does the privacy mode work, does it just pass zeroes (or other
> > dummy
> > data) towards the host or nothing?

No data on CSI transmitter side

> >
> > A V4L2 control can be used for the purpose of passing the information
> > to the user space at least.

I will take some time to review V4L2 sub-device and control mechanism, and =
then update the driver.

BR,
Wentong =20

> >
>=20
> --
> Best regards,
> Bingbu Cao