Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Hsia-Jun Li <randy.li@synaptics.com>
To:     linux-media@vger.kernel.org
Cc:     ayaka@soulik.info, hans.verkuil@cisco.com, tfiga@chromium.org,
        mchehab@kernel.org, laurent.pinchart@ideasonboard.com,
        hiroh@chromium.org, hverkuil@xs4all.nl,
        linux-kernel@vger.kernel.org, nicolas@ndufresne.ca,
        "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>
Subject: [PATCH 2/2] media: v4l2-mem2mem: add a list for buf used by hw
Date:   Tue,  4 Jul 2023 12:00:38 +0800
Message-ID: <20230704040044.681850-3-randy.li@synaptics.com>
In-Reply-To: <20230704040044.681850-1-randy.li@synaptics.com>
References: <20230704040044.681850-1-randy.li@synaptics.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MmeS6IVp4CBKoqmkE+X3YUhqTmU6sTOiA1AMsyz+QZB5vG9eNu80Klpv9yJ8?=
 =?us-ascii?Q?IqTaKKg7b3e9di6lBHKEj4wijrQ/eSQeBfgzf8C9VMO6UCn8509dKtwXfWDO?=
 =?us-ascii?Q?buf9jCmJCN2hqkc38pSp7jym0Qp8Cz74tiwjK3UFXKQRuwmd6ZxIUiGQ7FyM?=
 =?us-ascii?Q?YKqHBTyGMVgQWHHiRauMm9kk6vj8BXdU3oUM5sSlv7Zg+BQZ2VySxn9UEfvg?=
 =?us-ascii?Q?dAMg2Qm4iNNJ+HrrLMJhg/2EDnXI6MByzCJwM+ezy1P9RfUlIQjzqR2/QtPA?=
 =?us-ascii?Q?bPD4MQC1HwmzQb6wm1/Xokn5c+E9/HpmqBcHsCuUAGMLlyzLoxnmy356SWIi?=
 =?us-ascii?Q?O43PNUe5sOayWEgjhoGJ0A4JBy27P2umcARYX+ivJ2/n9tVdqPBcnGFgzYGa?=
 =?us-ascii?Q?atgBqlabzde5aKGdnwZmQxM9TGHTqJgX+FQlTIB5Hw0iCdIZEhDPWBKn0nQY?=
 =?us-ascii?Q?dhhDR6M48bDK9cDNRuhUgeibK/TPRCNkERRNnayrMwYhtexWvTg7xGh0FHWE?=
 =?us-ascii?Q?kR/raXKRWey7TGtB8kdsCcA2O8n1PMMrjiOqnYHaNtkTKWXVIm/4a95lb6wL?=
 =?us-ascii?Q?HXPZP1/eXWqmN/zHhGyIQ4ZKF3R3E2dAcO+Z6/J1G9hkMq5bk3UV3Lae9jCq?=
 =?us-ascii?Q?9QwMPVam0tukojtelMeqKKKK9nBJuoyUB9LrAgReKAsjuOhJdGkPF/xu0Vzg?=
 =?us-ascii?Q?2Jac69DbbkJpXvgn/iVcHjay3u0udRH+pcgluEnknxXA+vX8DZcqWXxsBV9x?=
 =?us-ascii?Q?VyHB+vTWU53eheXT9iSpf520V4SyFN2/8WFH4S66DOK7ece7MUKXY0sBhdaH?=
 =?us-ascii?Q?OdqsDaHtvrDUMmsvqHFe1KzmhkP7Lsa0DkWcr+UXJT+7HGZ2jAiraagPLFUT?=
 =?us-ascii?Q?/cv+WxbTS7UG845fsDQSmUirbALfGxjPdDkNOxVO08Sd+ORNsC9/0+0gqYya?=
 =?us-ascii?Q?qoR3PpqVn8v1r4RFbcffYZEF48r6ToTHCxZLK5AkmCwfuRhdyxBDouSzUsuE?=
 =?us-ascii?Q?CFwKpLYdMqL3xdK27MLbnANFbcGwPfnMkj3ODpy53lmOHFrcIIKzGcX5BIo5?=
 =?us-ascii?Q?9XJhYY8O67AEZJjkP6KfXTyFwYxizYoVuJFOfC63S5YzcAv8pPJ7smNUMsFH?=
 =?us-ascii?Q?RQgkGno45upFnctTH75CVhnJN8lywmUllpWNTlYLKDuz/ueU8TL3Gj4pwnrG?=
 =?us-ascii?Q?6vsSc9V42VDrf6o21ZggFVGx85eSL4FAFGJ0SbRcOFxP+b2cg99gihDhbWmz?=
 =?us-ascii?Q?hkE8aEMQe9OG2nNRhp9ErT9Ch9MuYNV/C4Hl4LY2K3JuL25K3SCLBzVkPVyZ?=
 =?us-ascii?Q?9TgOhrmCAkGHjWm0dHndsKNjWKQU+mRgFlJpkMr7mPKKyHi6WpbyTMAfMM8N?=
 =?us-ascii?Q?PjppzjMs2mZmCCWPjKcJi5eSpcKKMmCFX7GuNK70GDduYFjhqPf2ivrkf5UZ?=
 =?us-ascii?Q?GGErwHecfV+k08Jtv5vh/zYAgwEfGitWRD0PvfG+60vlo/3C1NDhoiLsmtlq?=
 =?us-ascii?Q?mx1mClmN8o1iDUp9L7StdlzEINvX2vuoh1a/1PvDm1YfHsDoM/4WL5TWbl6h?=
 =?us-ascii?Q?2uvy7Q19QCVpTym0s7coIqaqvyhBlM5bpkCSYypZ?=
X-OriginatorOrg: synaptics.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c0e4f16-8fed-4e12-e637-08db7c435760
X-MS-Exchange-CrossTenant-AuthSource: DM6PR03MB5196.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jul 2023 04:01:27.6491
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335d1fbc-2124-4173-9863-17e7051a2a0e
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Wt0sz6+IVAzTPmXPC/ldagP26DYPTBd+AKO2IIWPMhF/rEphzwGqjxTm2CKGIV7wpoVzQYLEXbEIS4SP+JZuSg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR03MB7385
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>

Many drivers have to create its own buf_struct for a
vb2_queue to track such a state. Also driver has to
iterate over rdy_queue every times to find out a buffer
which is not sent to hardware(or firmware), this new
list just offers the driver a place to store the buffer
that hardware(firmware) has acknowledged.

One important advance about this list, it doesn't like
rdy_queue which both bottom half of the user calling
could operate it, while the v4l2 worker would as well.
The v4l2 core could only operate this queue when its
v4l2_context is not running, the driver would only
access this new hw_queue in its own worker.

Signed-off-by: Hsia-Jun(Randy) Li <randy.li@synaptics.com>
---
 drivers/media/v4l2-core/v4l2-mem2mem.c | 25 +++++++++++++++++--------
 include/media/v4l2-mem2mem.h           | 10 +++++++++-
 2 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index c771aba42015..b4151147d5bd 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -321,15 +321,21 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev,
 		goto job_unlock;
 	}
 
-	src = v4l2_m2m_next_src_buf(m2m_ctx);
-	dst = v4l2_m2m_next_dst_buf(m2m_ctx);
-	if (!src && !m2m_ctx->out_q_ctx.buffered) {
-		dprintk("No input buffers available\n");
-		goto job_unlock;
+	if (list_empty(&m2m_ctx->out_q_ctx.hw_queue)) {
+		src = v4l2_m2m_next_src_buf(m2m_ctx);
+
+		if (!src && !m2m_ctx->out_q_ctx.buffered) {
+			dprintk("No input buffers available\n");
+			goto job_unlock;
+		}
 	}
-	if (!dst && !m2m_ctx->cap_q_ctx.buffered) {
-		dprintk("No output buffers available\n");
-		goto job_unlock;
+
+	if (list_empty(&m2m_ctx->cap_q_ctx.hw_queue)) {
+		dst = v4l2_m2m_next_dst_buf(m2m_ctx);
+		if (!dst && !m2m_ctx->cap_q_ctx.buffered) {
+			dprintk("No output buffers available\n");
+			goto job_unlock;
+		}
 	}
 
 	m2m_ctx->new_frame = true;
@@ -896,6 +902,7 @@ int v4l2_m2m_streamoff(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
 	INIT_LIST_HEAD(&q_ctx->rdy_queue);
 	q_ctx->num_rdy = 0;
 	spin_unlock_irqrestore(&q_ctx->rdy_spinlock, flags);
+	INIT_LIST_HEAD(&q_ctx->hw_queue);
 
 	if (m2m_dev->curr_ctx == m2m_ctx) {
 		m2m_dev->curr_ctx = NULL;
@@ -1234,6 +1241,8 @@ struct v4l2_m2m_ctx *v4l2_m2m_ctx_init(struct v4l2_m2m_dev *m2m_dev,
 
 	INIT_LIST_HEAD(&out_q_ctx->rdy_queue);
 	INIT_LIST_HEAD(&cap_q_ctx->rdy_queue);
+	INIT_LIST_HEAD(&out_q_ctx->hw_queue);
+	INIT_LIST_HEAD(&cap_q_ctx->hw_queue);
 	spin_lock_init(&out_q_ctx->rdy_spinlock);
 	spin_lock_init(&cap_q_ctx->rdy_spinlock);
 
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index d6c8eb2b5201..2342656e582d 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -53,9 +53,16 @@ struct v4l2_m2m_dev;
  *	processed
  *
  * @q:		pointer to struct &vb2_queue
- * @rdy_queue:	List of V4L2 mem-to-mem queues
+ * @rdy_queue:	List of V4L2 mem-to-mem queues. If v4l2_m2m_buf_queue() is
+ *		called in struct vb2_ops->buf_queue(), the buffer enqueued
+ *		by user would be added to this list.
  * @rdy_spinlock: spin lock to protect the struct usage
  * @num_rdy:	number of buffers ready to be processed
+ * @hw_queue:	A list for tracking the buffer is occupied by the hardware
+ * 		(or device's firmware). A buffer could only be in either
+ * 		this list or @rdy_queue.
+ * 		Driver may choose not to use this list while uses its own
+ * 		private data to do this work.
  * @buffered:	is the queue buffered?
  *
  * Queue for buffers ready to be processed as soon as this
@@ -68,6 +75,7 @@ struct v4l2_m2m_queue_ctx {
 	struct list_head	rdy_queue;
 	spinlock_t		rdy_spinlock;
 	u8			num_rdy;
+	struct list_head	hw_queue;
 	bool			buffered;
 };
 
-- 
2.17.1