Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <aed6672e-b60f-3a02-8566-2b2dcdb05d4a@synaptics.com>
Date:   Fri, 19 Aug 2022 07:28:54 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.12.0
Subject: Re: [PATCH 0/2] Add pixel formats used in Synatpics SoC
Content-Language: en-US
To:     Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc:     dri-devel@lists.freedesktop.org, maarten.lankhorst@linux.intel.com,
        mripard@kernel.org, tzimmermann@suse.de, airlied@linux.ie,
        daniel@ffwll.ch, mchehab@kernel.org, hverkuil-cisco@xs4all.nl,
        ezequiel@vanguardiasur.com.ar, sakari.ailus@linux.intel.com,
        ribalda@chromium.org, linux-media@vger.kernel.org,
        linux-kernel@vger.kernel.org, tfiga@chromium.org,
        sebastian.hesselbarth@gmail.com, jszhang@kernel.org,
        linux-arm-kernel@lists.infradead.org
References: <20220808162750.828001-1-randy.li@synaptics.com>
 <Yv7Gc+mSEXBnV0Oc@pendragon.ideasonboard.com>
From:   Hsia-Jun Li <Randy.Li@synaptics.com>
In-Reply-To: <Yv7Gc+mSEXBnV0Oc@pendragon.ideasonboard.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGl3T095eCs0Z1pIemJYM0lkRjFhaUFmZ2ZTcm9DMFVvVzdpR3NaNVBwaEMx?=
 =?utf-8?B?OXh4STRwYmVKc2NOMmhpejJ2SStQeVZVRVpKaVoxYlg1YVFTRCtkQWdGbVQ4?=
 =?utf-8?B?MEJ6a1VUZytXKzB4cTM3SlRINUdmZXM4SW5iRnA4Q1oyUHltSVJlaTdxZ3li?=
 =?utf-8?B?cjdPVENUVkk2RHQ1QWVxNURyaUNIUEFaV1g2M01pdzl3TU1nMExwcmtOVVdM?=
 =?utf-8?B?Uk51VDZzVGdKQUhOUXlRbmkrcHVPK25Id2FGUUpnQzdRQm1lY0U0aXZoUmNB?=
 =?utf-8?B?Q3FjWWVKbFhCQWVRQVJBbkNXZVNsMlNSenQ0dExwRnIzeEdvRVV2REtVallr?=
 =?utf-8?B?Y0J1WWNYNk5ZWmpocGJNNlFSZC9ieUw0VTE3REVZL0RSanZqNUhyU1JOWkwy?=
 =?utf-8?B?bjRtM04rWXA1RExwY0J1KzEyd0VsS3RNdVpySG1ETnNwckFqaHQ0S3NkNHBE?=
 =?utf-8?B?aEZMU0FkSmRqcFRSMTV2b3VuQS85WnEyNC9IL0FxOTRuVEtWU1JMYTZoOVU0?=
 =?utf-8?B?dHFNYzNwd0kyMUFhQk5NbTR6TEFhVmd3Q0JPZjJrL3Z4ZzhpSFA0S3drVTE4?=
 =?utf-8?B?bmM2WUFDMkwzUmIrSmhNRHIrTDJTdDNoUk1FT1hWMTVmZWdjUTBWQXpjMXd0?=
 =?utf-8?B?aXYraC9KQk5MejRzellFQmF4U1NGWUlKc0VNbHJuRWVnbk9oRHZEb21haFV5?=
 =?utf-8?B?SUZkQ3F2VlJKQ2QvRGcyOEtSeXVtdlhTQ2N6WnNudXNaYndFaE5rQWNMQkNr?=
 =?utf-8?B?KzZQQ0Vpd2lHK0ZPR2xZYmJiclZDdHo4MVV1TmMyMzNRWXdzY3RzZlhjRDRi?=
 =?utf-8?B?bzArVVBMWjZCTEFXZ1RFNWk5dzR1eGRBN1B6a2N6eXVxMnl5cGNSRDJJaGpv?=
 =?utf-8?B?Skd4QVR6WW5iVWVEV25XMm9VNGIzdlh6VXNNTHNKTHJYQnlnZ2FnK1I4TG91?=
 =?utf-8?B?TU1CSVVBOENTd05QVlUwdGpMMjJSTjB0aXNYd0ZUZ3owQUVFZUVUeGlWUmw5?=
 =?utf-8?B?Y2VNSFdjZDJUME5nb1J5Qnozc2NUR1J5cGJKVzNxUlVZdFlPZnMvb0V0eHF5?=
 =?utf-8?B?MFZKQ1daRDNiK0Y1dEEvT0RpYUJsR2E0by9VOVp4QUVqTjBPVVRlckhQREdp?=
 =?utf-8?B?RUlaR2g0OE0xOHlWSFJSSHYyanpGaEpGQWFJbGlpcXU1SWpqUVJLdzJCQUhw?=
 =?utf-8?B?N0lQMFNMZURqRGE5aEpURmJUN1YyMTc2NmUraHF3Zkl2SFhyVjVzR0d6Y29z?=
 =?utf-8?B?Q1lseHZIZ3ZjNzZROG5YUmVwb0FUN3JlVmppbHZCUURTTnprVHNDdytYOHZZ?=
 =?utf-8?B?OG80ZkVBV0kyUGtpRGtueGp1THdNV2VNNzBLNEhJQlQyYnhLM25kd1pnN1JG?=
 =?utf-8?B?TnBacDd6cGJCOVVCc0VSZWZwMTlSdXZONktVNTF5SnBYKy8vV3RoQW5kV1J0?=
 =?utf-8?B?VklHV3k0SUozRW12dm5yWElXdmx0MVJjSG9oSDY2OHd6T2FLdS8wWnJWb2RN?=
 =?utf-8?B?QUs3VDJZTXorNnQ4NzVxWXFDNWNyeFBxK0lUVUhhSGhkVmkwYk5Va1k1WUhM?=
 =?utf-8?B?MW1PQjliNXJvNXRZamk3SEZPdjhXWUQ3aDJNQmZ2ZnVPRXFDOUVDVGRFVUpV?=
 =?utf-8?B?VElCRUtHcG5pMTdVTmZpcE5zaTVtbnorYjBkVndTNHdlQUR5NjVYUmVtaVdl?=
 =?utf-8?B?cTFnSnFxUGRGc2pxMUs1YnkvYjFBUExVK1NrZjJOT0UxejZwc0hWSzhQYmJL?=
 =?utf-8?B?eGdVQ08vK1l4MzBvMy85aUk2MkptWWY2VjNJNEdBVFh1VURFaTgzekMrdWtw?=
 =?utf-8?B?akV6SW1qdFg0L0VETFlmRHFiNTJjVzI5dEtHVUFzRlVxZ0ttT2JweUJCZks5?=
 =?utf-8?B?ZStFNVVlTnA4b2tSdnBkWVI2ZU1MQ2pRRFFmL0Y2TlREb2Rkc2FTVTB5R3Ns?=
 =?utf-8?B?UWE1aHh2STdkeDJpaWI0d1IzaEVLOHJybis5alVsMHRCYko5TDB5UXlvb2FL?=
 =?utf-8?B?MTRrVVZZQmgzcjlOQW5oVmlZODlaRnV2L3NocEI4ZU1zV1lUWG1GUnY4RHU4?=
 =?utf-8?B?eWhBQ3RDS3NlRkdlL2hjT1h6Z0xSMkpYMTU0QTBTNUJ2SVdNSmpFZGdHendW?=
 =?utf-8?Q?v/oBpTcRGyXq355YRWC+qs5X5?=
X-OriginatorOrg: synaptics.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ba429377-319a-473e-3cf5-08da8171712d
X-MS-Exchange-CrossTenant-AuthSource: DM6PR03MB5196.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2022 23:29:05.7830
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335d1fbc-2124-4173-9863-17e7051a2a0e
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: m3u3fSGWP5uu7PE0QcaD2kQUuU2tpvlxDx7WstnpZUnq2YBZZs9/0NldeecNDx3KTzHoQ0w9TOVzog0Yjtv/AA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR03MB5059
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,
        SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 8/19/22 07:08, Laurent Pinchart wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> Hi Hsia-Jun,
> 
> On Tue, Aug 09, 2022 at 12:27:48AM +0800, Hsia-Jun Li wrote:
>> From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>
>>
>> Those pixel formats are used in Synaptics's VideoSmart series SoCs,
>> likes VS640, VS680.  I just disclose the pixel formats used in the video
>> codecs and display pipeline this time. Actually any device with a MTR
>> module could support those tiled and compressed pixel formats. The more
>> detail about MTR module could be found in the first patch of this serial
>> of mail.
>>
>> We may not be able to post any drivers here in a short time, the most of
>> work in this platform is done in the Trusted Execution Environment and
>> we didn't use the optee framework.
> 
> Is that so for the display side too, or only for the video decoder ?
These pixel formats are using in both video decoder and display(Not the 
GPU). Besides, ISP and NPU in vs680 support some patterns of them.

Please notice that after I reviewed the compression options of our 
platform, I found using modifies are not enough to store all the 
compression options here. I would post a second version here.

I may use the same way that Intel, I would try to disclose more details 
here, hoping we could find a better way to describe them.
> 
>> Please notice that, the memory planes used for video codecs would be 5
>> when the compression is invoked while it would be 4 for display, the
>> extra planes in the video codecs is for the decoding internally usage,
>> it can't append the luma or chroma buffer as many other drivers do,
>> because this buffer could be only accessed by the video codecs itself,
>> it requests a different memory security attributes. Any other reason is
>> described in the v4l pixel formats's patch. I don't know whether a
>> different numbers of memory planes between drm and v4l2 is acceptable.
> 
> I don't think that's a problem as such, as long as both the V4L2 and DRM
> formats make sense on their own.
> 
>> I only posted the compression fourcc for the v4l2, because it is really
>> hard to put the uncompression version of pixel formats under the fourcc.
>> I would be better that we could have something likes format modifers in
>> drm here.
> 
> Agreed, we need modifiers support in V4L2. This has been discussed
> previously ([1]), and a proposal ([2]) has been submitted two years ago,
> it needs to be revived.
Thank you, I have found those v4l2_ext_pix_format, I would relay my 
comment in the email that posting synaptics v4l2 pixel formats.
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_linux-2Dmedia_20170821155203.GB38943-40e107564-2Dlin.cambridge.arm.com_&d=DwIBaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=Ktu-e-R1Mn89Laxioh6RlL6Y2aycZ9NrJTIyONaDdRQvnlv-Nd570KldQ51vmigK&s=_7eMTIYwWUOWkXijcRfotLJlpR7G5yx-ZXuTwh9uZw4&e=
> [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_linux-2Dmedia_20200804192939.2251988-2D1-2Dhelen.koike-40collabora.com_&d=DwIBaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=Ktu-e-R1Mn89Laxioh6RlL6Y2aycZ9NrJTIyONaDdRQvnlv-Nd570KldQ51vmigK&s=f1dbc5ciUeIkO6VMtlRuEvXqJad2NsoaDBFyNUsSdpg&e=
> 
>> https://synaptics.com/products/multimedia-solutions
>>
>> Hsia-Jun(Randy) Li (2):
>>    drm/fourcc: Add Synaptics VideoSmart tiled modifiers
>>    [WIP]: media: Add Synaptics compressed tiled format
>>
>>   drivers/media/v4l2-core/v4l2-common.c |  1 +
>>   drivers/media/v4l2-core/v4l2-ioctl.c  |  2 ++
>>   include/uapi/drm/drm_fourcc.h         | 49 +++++++++++++++++++++++++++
>>   include/uapi/linux/videodev2.h        |  2 ++
>>   4 files changed, 54 insertions(+)
> 
> --
> Regards,
> 
> Laurent Pinchart

-- 
Hsia-Jun(Randy) Li