2020-02-04 02:57:55

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 00/12] Implement V4L2_BUF_FLAG_NO_CACHE_* flags

Hello,

V2 of the patchset, reshuffled, added more documentation,
addressed some of the feedback ;) Still in RFC, tho

v1 link: https://lore.kernel.org/lkml/[email protected]/

---
This is a reworked version of the vb2 cache hints
(V4L2_BUF_FLAG_NO_CACHE_INVALIDATE / V4L2_BUF_FLAG_NO_CACHE_CLEAN)
support patch series which previsouly was developed by Sakari and
Laurent [0].

The patch set attempts to preserve the existing behvaiour - cache
sync is performed in ->prepare() and ->finish() (unless the buffer
is DMA exported). User space can request “default behavior” override
with cache management hints, which are handled on a per-buffer basis
and should be supplied with v4l2_buffer ->flags during buffer
preparation. There are two possible hints:

- V4L2_BUF_FLAG_NO_CACHE_INVALIDATE
No cache sync on ->finish()

- V4L2_BUF_FLAG_NO_CACHE_CLEAN
No cache sync on ->prepare()

In order to keep things on the safe side, we also require driver
to explicitly state which of its queues (if any) support user space
cache management hints (such queues should have ->allow_cache_hints
bit set).

The patch set also (to some extent) simplifies allocators' ->prepare()
and ->finish() callbacks. Namely, we move cache management decision
making to the upper - core - layer. For example, if, previously, we
would have something like this

vb2_buffer_done()
vb2_dc_finish()
if (buf->db_attach)
return;

where each allocators' ->finish() callback would either bail
out (DMA exported buffer, for instance) or sync, now that "bail
out or sync" decision is made before we call into the allocator.

Along with cache management hints, user space is also able to
adjust queue's memory consistency attributes. Memory consistency
attribute (dma_attrs) is per-queue, yet it plays its role on the
allocator level, when we allocate buffers’ private memory (planes).
For the time being, only one consistency attribute is supported:
DMA_ATTR_NON_CONSISTENT.

[0] https://www.mail-archive.com/[email protected]/msg112459.html

Sergey Senozhatsky (12):
videobuf2: add cache management members
videobuf2: handle V4L2 buffer cache flags
videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag
videobuf2: add queue memory consistency parameter
videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag
videobuf2: factor out planes prepare/finish functions
videobuf2: do not sync caches when we are allowed not to
videobuf2: check ->synced flag in prepare() and finish()
videobuf2: let user-space know if driver supports cache hints
videobuf2: add begin/end cpu_access callbacks to dma-contig
videobuf2: add begin/end cpu_access callbacks to dma-sg
videobuf2: don't test db_attach in dma-contig prepare and finish

Documentation/media/uapi/v4l/buffer.rst | 27 +++++
.../media/uapi/v4l/vidioc-create-bufs.rst | 9 +-
.../media/uapi/v4l/vidioc-reqbufs.rst | 22 +++-
.../media/common/videobuf2/videobuf2-core.c | 110 +++++++++++++-----
.../common/videobuf2/videobuf2-dma-contig.c | 39 ++++++-
.../media/common/videobuf2/videobuf2-dma-sg.c | 30 +++--
.../media/common/videobuf2/videobuf2-v4l2.c | 59 +++++++++-
drivers/media/dvb-core/dvb_vb2.c | 2 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 5 +-
include/media/videobuf2-core.h | 17 ++-
include/uapi/linux/videodev2.h | 11 +-
11 files changed, 273 insertions(+), 58 deletions(-)

--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:57:57

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 01/12] videobuf2: add cache management members

Extend vb2_buffer and vb2_queue structs with cache management
members.

V4L2 UAPI already contains two buffer flags which user-space,
supposedly, can use to control buffer cache sync:

- V4L2_BUF_FLAG_NO_CACHE_INVALIDATE
- V4L2_BUF_FLAG_NO_CACHE_CLEAN

None of these, however, do anything at the moment. This patch
set is intended to change it.

Since user-space cache management hints are supposed to be
implemented on a per-buffer basis we need to extend vb2_buffer
struct with two new memebers ->need_cache_sync_on_prepare and
->need_cache_sync_on_finish, which will store corresponding
user-space hints.

In order to preserve the existing behaviour, user-space cache
managements flags will be handled only by those drivers that
permit user-space cache hints. That's the purpose of vb2_queue
->allow_cache_hints member. Driver must set ->allow_cache_hints
during queue initialisation to enable cache management hints
mechanism.

Only drivers that set ->allow_cache_hints during queue initialisation
will handle user-space cache management hints. Otherwise hints
will be ignored.

Change-Id: I52beec2a0d021b7a3715b4f6ae4bfd9dc5e94f0d
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
include/media/videobuf2-core.h | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index a2b2208b02da..026004180440 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -263,6 +263,10 @@ struct vb2_buffer {
* after the 'buf_finish' op is called.
* copied_timestamp: the timestamp of this capture buffer was copied
* from an output buffer.
+ * need_cache_sync_on_prepare: do not sync/invalidate cache from
+ * buffer's ->prepare() callback.
+ * need_cache_sync_on_finish: do not sync/invalidate cache from buffer's
+ * ->finish() callback.
* queued_entry: entry on the queued buffers list, which holds
* all buffers queued from userspace
* done_entry: entry on the list that stores all buffers ready
@@ -273,6 +277,8 @@ struct vb2_buffer {
unsigned int synced:1;
unsigned int prepared:1;
unsigned int copied_timestamp:1;
+ unsigned int need_cache_sync_on_prepare:1;
+ unsigned int need_cache_sync_on_finish:1;

struct vb2_plane planes[VB2_MAX_PLANES];
struct list_head queued_entry;
@@ -491,6 +497,9 @@ struct vb2_buf_ops {
* @uses_requests: requests are used for this queue. Set to 1 the first time
* a request is queued. Set to 0 when the queue is canceled.
* If this is 1, then you cannot queue buffers directly.
+ * @allow_cache_hints: when set user-space can pass cache management hints in
+ * order to skip cache flush/invalidation on ->prepare() or/and
+ * ->finish().
* @lock: pointer to a mutex that protects the &struct vb2_queue. The
* driver can set this to a mutex to let the v4l2 core serialize
* the queuing ioctls. If the driver wants to handle locking
@@ -564,6 +573,7 @@ struct vb2_queue {
unsigned requires_requests:1;
unsigned uses_qbuf:1;
unsigned uses_requests:1;
+ unsigned allow_cache_hints:1;

struct mutex *lock;
void *owner;
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:12

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

Set video buffer cache management flags corresponding to V4L2 cache
flags.

Both ->prepare() and ->finish() cache management hints should be
passed during this stage (buffer preparation), because there is no
other way for user-space to skip ->finish() cache flush.

There are two possible alternative approaches:
- The first one is to move cache sync from ->finish() to dqbuf().
But this breaks some drivers, that need to fix-up buffers before
dequeueing them.

- The second one is to move ->finish() call from ->done() to dqbuf.

Change-Id: I3bef1d1fb93a5fba290ce760eaeecdc8e7d6885a
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/common/videobuf2/videobuf2-v4l2.c | 36 +++++++++++++++++++
1 file changed, 36 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index eb5d5db96552..2da06a2ad6c4 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -337,6 +337,41 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
return 0;
}

+static void set_buffer_cache_hints(struct vb2_queue *q,
+ struct vb2_buffer *vb,
+ struct v4l2_buffer *b)
+{
+ /*
+ * DMA exporter should take care of cache syncs, so we can avoid
+ * explicit ->prepare()/->finish() syncs. For other ->memory types
+ * we always need ->prepare() or/and ->finish() cache sync.
+ */
+ if (q->memory == VB2_MEMORY_DMABUF) {
+ vb->need_cache_sync_on_finish = 0;
+ vb->need_cache_sync_on_prepare = 0;
+ return;
+ }
+
+ if (!q->allow_cache_hints)
+ return;
+
+ vb->need_cache_sync_on_prepare = 1;
+ /*
+ * ->finish() cache sync can be avoided when queue direction is
+ * TO_DEVICE.
+ */
+ if (q->dma_dir != DMA_TO_DEVICE)
+ vb->need_cache_sync_on_finish = 1;
+ else
+ vb->need_cache_sync_on_finish = 0;
+
+ if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
+ vb->need_cache_sync_on_finish = 0;
+
+ if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
+ vb->need_cache_sync_on_prepare = 0;
+}
+
static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b, bool is_prepare,
struct media_request **p_req)
@@ -381,6 +416,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
}

if (!vb->prepared) {
+ set_buffer_cache_hints(q, vb, b);
/* Copy relevant information provided by the userspace */
memset(vbuf->planes, 0,
sizeof(vbuf->planes[0]) * vb->num_planes);
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:21

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 03/12] videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag

By setting or clearing V4L2_FLAG_MEMORY_NON_CONSISTENT flag
user-space should be able to set or clear queue's NON_CONSISTENT
->dma_attrs. Queue's ->dma_attrs are passed to the underlying
allocator in __vb2_buf_mem_alloc(), so thus user-space is able
to request vb2 buffer's memory to be either consistent (coherent)
or non-consistent.

Change-Id: Ib333081c482e23c9a89386078293e19c3fd59076
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
Documentation/media/uapi/v4l/buffer.rst | 27 +++++++++++++++++++++++++
include/uapi/linux/videodev2.h | 2 ++
2 files changed, 29 insertions(+)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 9149b57728e5..af007daf0591 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -705,6 +705,33 @@ Buffer Flags

.. c:type:: v4l2_memory

+Memory Consistency Flags
+========================
+
+.. tabularcolumns:: |p{7.0cm}|p{2.2cm}|p{8.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 3 1 4
+
+ * .. _`V4L2_FLAG_MEMORY_NON_CONSISTENT`:
+
+ - ``V4L2_FLAG_MEMORY_NON_CONSISTENT``
+ - 0x00000001
+ - vb2 buffer is allocated either in consistent (it will be automatically
+ coherent between CPU and bus) or non-consistent memory. The latter
+ can provide performance gains, for instance CPU cache sync/flush
+ operations can be avoided if the buffer is accesed by the corresponding
+ device only and CPU does not read/write to/from that buffer. However,
+ this requires extra care from the driver -- it must guarantee memory
+ consistency by issuing cache flush/sync when consistency is needed.
+ If this flag is set V4L2 will attempt to allocate vb2 buffer in
+ non-consistent memory. This flag is ignored if queue does not report
+ :ret:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
+
enum v4l2_memory
================

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 5f9357dcb060..72efc1c544cd 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -189,6 +189,8 @@ enum v4l2_memory {
V4L2_MEMORY_DMABUF = 4,
};

+#define V4L2_FLAG_MEMORY_NON_CONSISTENT (1 << 0)
+
/* see also http://vektor.theorem.ca/graphics/ycbcr/ */
enum v4l2_colorspace {
/*
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:27

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 04/12] videobuf2: add queue memory consistency parameter

Preparations for future V4L2_FLAG_MEMORY_NON_CONSISTENT support.

Extend vb2_core_reqbufs() with queue memory consistency flag
that is applied to the newly allocated buffers.

An attempt to allocate a buffer with consistency requirements
which don't match queue's consistency model will fail.

Change-Id: Ia40362553b6bd96fea3b652f4a376e7a3467df0a
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/common/videobuf2/videobuf2-core.c | 47 +++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 6 +--
drivers/media/dvb-core/dvb_vb2.c | 2 +-
include/media/videobuf2-core.h | 7 ++-
4 files changed, 47 insertions(+), 15 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 4489744fbbd9..56fd17eafb6c 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -664,8 +664,19 @@ int vb2_verify_memory_type(struct vb2_queue *q,
}
EXPORT_SYMBOL(vb2_verify_memory_type);

+static void set_queue_consistency(struct vb2_queue *q, bool consistent_mem)
+{
+ if (!q->allow_cache_hints)
+ return;
+
+ if (consistent_mem)
+ q->dma_attrs &= ~DMA_ATTR_NON_CONSISTENT;
+ else
+ q->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
+}
+
int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
- unsigned int *count)
+ bool consistent_mem, unsigned int *count)
{
unsigned int num_buffers, allocated_buffers, num_planes = 0;
unsigned plane_sizes[VB2_MAX_PLANES] = { };
@@ -720,6 +731,7 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
num_buffers = min_t(unsigned int, num_buffers, VB2_MAX_FRAME);
memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
q->memory = memory;
+ set_queue_consistency(q, consistent_mem);

/*
* Ask the driver how many buffers and planes per buffer it requires.
@@ -803,9 +815,21 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
}
EXPORT_SYMBOL_GPL(vb2_core_reqbufs);

+static bool verify_consistency_attr(struct vb2_queue *q, bool consistent_mem)
+{
+ bool queue_attr = q->dma_attrs & DMA_ATTR_NON_CONSISTENT;
+
+ if (consistent_mem != queue_attr) {
+ dprintk(1, "memory consistency model mismatch\n");
+ return false;
+ }
+ return true;
+}
+
int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
- unsigned int *count, unsigned requested_planes,
- const unsigned requested_sizes[])
+ bool consistent_mem, unsigned int *count,
+ unsigned requested_planes,
+ const unsigned requested_sizes[])
{
unsigned int num_planes = 0, num_buffers, allocated_buffers;
unsigned plane_sizes[VB2_MAX_PLANES] = { };
@@ -823,10 +847,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
}
memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
q->memory = memory;
+ set_queue_consistency(q, consistent_mem);
q->waiting_for_buffers = !q->is_output;
- } else if (q->memory != memory) {
- dprintk(1, "memory model mismatch\n");
- return -EINVAL;
+ } else {
+ if (q->memory != memory) {
+ dprintk(1, "memory model mismatch\n");
+ return -EINVAL;
+ }
+ if (!verify_consistency_attr(q, consistent_mem))
+ return -EINVAL;
}

num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);
@@ -2498,7 +2527,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
fileio->memory = VB2_MEMORY_MMAP;
fileio->type = q->type;
q->fileio = fileio;
- ret = vb2_core_reqbufs(q, fileio->memory, &fileio->count);
+ ret = vb2_core_reqbufs(q, fileio->memory, true, &fileio->count);
if (ret)
goto err_kfree;

@@ -2555,7 +2584,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)

err_reqbufs:
fileio->count = 0;
- vb2_core_reqbufs(q, fileio->memory, &fileio->count);
+ vb2_core_reqbufs(q, fileio->memory, true, &fileio->count);

err_kfree:
q->fileio = NULL;
@@ -2575,7 +2604,7 @@ static int __vb2_cleanup_fileio(struct vb2_queue *q)
vb2_core_streamoff(q, q->type);
q->fileio = NULL;
fileio->count = 0;
- vb2_core_reqbufs(q, fileio->memory, &fileio->count);
+ vb2_core_reqbufs(q, fileio->memory, true, &fileio->count);
kfree(fileio);
dprintk(3, "file io emulator closed\n");
}
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 2da06a2ad6c4..7cdfcd1baf82 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -709,7 +709,7 @@ int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
int ret = vb2_verify_memory_type(q, req->memory, req->type);

fill_buf_caps(q, &req->capabilities);
- return ret ? ret : vb2_core_reqbufs(q, req->memory, &req->count);
+ return ret ? ret : vb2_core_reqbufs(q, req->memory, true, &req->count);
}
EXPORT_SYMBOL_GPL(vb2_reqbufs);

@@ -783,7 +783,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
for (i = 0; i < requested_planes; i++)
if (requested_sizes[i] == 0)
return -EINVAL;
- return ret ? ret : vb2_core_create_bufs(q, create->memory,
+ return ret ? ret : vb2_core_create_bufs(q, create->memory, true,
&create->count, requested_planes, requested_sizes);
}
EXPORT_SYMBOL_GPL(vb2_create_bufs);
@@ -959,7 +959,7 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv,
return res;
if (vb2_queue_is_busy(vdev, file))
return -EBUSY;
- res = vb2_core_reqbufs(vdev->queue, p->memory, &p->count);
+ res = vb2_core_reqbufs(vdev->queue, p->memory, true, &p->count);
/* If count == 0, then the owner has released all buffers and he
is no longer owner of the queue. Otherwise we have a new owner. */
if (res == 0)
diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
index 6974f1731529..e60063652164 100644
--- a/drivers/media/dvb-core/dvb_vb2.c
+++ b/drivers/media/dvb-core/dvb_vb2.c
@@ -342,7 +342,7 @@ int dvb_vb2_reqbufs(struct dvb_vb2_ctx *ctx, struct dmx_requestbuffers *req)

ctx->buf_siz = req->size;
ctx->buf_cnt = req->count;
- ret = vb2_core_reqbufs(&ctx->vb_q, VB2_MEMORY_MMAP, &req->count);
+ ret = vb2_core_reqbufs(&ctx->vb_q, VB2_MEMORY_MMAP, true, &req->count);
if (ret) {
ctx->state = DVB_VB2_STATE_NONE;
dprintk(1, "[%s] count=%d size=%d errno=%d\n", ctx->name,
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 026004180440..5e5450bdabbd 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -726,6 +726,7 @@ void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb);
* vb2_core_reqbufs() - Initiate streaming.
* @q: pointer to &struct vb2_queue with videobuf2 queue.
* @memory: memory type, as defined by &enum vb2_memory.
+ * @consistent_mem: memory consistency model.
* @count: requested buffer count.
*
* Videobuf2 core helper to implement VIDIOC_REQBUF() operation. It is called
@@ -750,12 +751,13 @@ void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb);
* Return: returns zero on success; an error code otherwise.
*/
int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
- unsigned int *count);
+ bool consistent_mem, unsigned int *count);

/**
* vb2_core_create_bufs() - Allocate buffers and any required auxiliary structs
* @q: pointer to &struct vb2_queue with videobuf2 queue.
* @memory: memory type, as defined by &enum vb2_memory.
+ * @consistent_mem: memory consistency model.
* @count: requested buffer count.
* @requested_planes: number of planes requested.
* @requested_sizes: array with the size of the planes.
@@ -773,7 +775,8 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
* Return: returns zero on success; an error code otherwise.
*/
int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
- unsigned int *count, unsigned int requested_planes,
+ bool consistent_mem, unsigned int *count,
+ unsigned int requested_planes,
const unsigned int requested_sizes[]);

/**
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:33

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

This patch lets user-space to request a non-consistent memory
allocation during CREATE_BUFS and REQBUFS ioctl calls.

= CREATE_BUFS

struct v4l2_create_buffers has seven 4-byte reserved areas,
so reserved[0] is renamed to ->flags. The struct, thus, now
has six reserved 4-byte regions.

= REQBUFS

We use one bit of a ->reserved[1] member of struct v4l2_requestbuffers,
which is now renamed to ->flags. Unlike v4l2_create_buffers, struct
v4l2_requestbuffers does not have enough reserved room. Therefore for
backward compatibility ->reserved and ->flags were put into anonymous
union.

Change-Id: I0eaab3428de499ce1bce6fc6b26c5ca5ff405882
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/uapi/v4l/vidioc-create-bufs.rst | 9 +++++++-
.../media/uapi/v4l/vidioc-reqbufs.rst | 15 ++++++++++---
.../media/common/videobuf2/videobuf2-v4l2.c | 21 ++++++++++++++++---
drivers/media/v4l2-core/v4l2-ioctl.c | 5 +----
include/uapi/linux/videodev2.h | 8 +++++--
5 files changed, 45 insertions(+), 13 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
index bd08e4f77ae4..68185e94b686 100644
--- a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
+++ b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
@@ -121,7 +121,14 @@ than the number requested.
other changes, then set ``count`` to 0, ``memory`` to
``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type.
* - __u32
- - ``reserved``\ [7]
+ - ``flags``
+ - Specifies additional buffer management attributes. Valid only when
+ queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
+ Old drivers and applications must set it to zero.
+
+
+ * - __u32
+ - ``reserved``\ [6]
- A place holder for future extensions. Drivers and applications
must set the array to zero.

diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
index d0c643db477a..9741dac0d5b3 100644
--- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
+++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
@@ -112,10 +112,19 @@ aborting or finishing any DMA in progress, an implicit
``V4L2_MEMORY_MMAP`` and ``type`` set to the buffer type. This will
free any previously allocated buffers, so this is typically something
that will be done at the start of the application.
- * - __u32
+ * - union
+ - (anonymous)
+ * -
+ - __u32
+ - ``flags``\ [1]
+ - Specifies additional buffer management attributes. Valid only when
+ queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
+ Old drivers and applications must set it to zero.
+
+ * -
+ - __u32
- ``reserved``\ [1]
- - A place holder for future extensions. Drivers and applications
- must set the array to zero.
+ - Kept for backwards compatibility. Use ``flags`` instead.

.. tabularcolumns:: |p{6.1cm}|p{2.2cm}|p{8.7cm}|

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 7cdfcd1baf82..eb5d1306cb03 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -707,9 +707,15 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
{
int ret = vb2_verify_memory_type(q, req->memory, req->type);
+ bool consistent = true;
+
+ if (req->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
+ consistent = false;

fill_buf_caps(q, &req->capabilities);
- return ret ? ret : vb2_core_reqbufs(q, req->memory, true, &req->count);
+ if (ret)
+ return ret;
+ return vb2_core_reqbufs(q, req->memory, consistent, &req->count);
}
EXPORT_SYMBOL_GPL(vb2_reqbufs);

@@ -738,6 +744,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
unsigned requested_sizes[VIDEO_MAX_PLANES];
struct v4l2_format *f = &create->format;
int ret = vb2_verify_memory_type(q, create->memory, f->type);
+ bool consistent = true;
unsigned i;

fill_buf_caps(q, &create->capabilities);
@@ -783,7 +790,11 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
for (i = 0; i < requested_planes; i++)
if (requested_sizes[i] == 0)
return -EINVAL;
- return ret ? ret : vb2_core_create_bufs(q, create->memory, true,
+
+ if (create->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
+ consistent = false;
+
+ return ret ? ret : vb2_core_create_bufs(q, create->memory, consistent,
&create->count, requested_planes, requested_sizes);
}
EXPORT_SYMBOL_GPL(vb2_create_bufs);
@@ -953,13 +964,17 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv,
{
struct video_device *vdev = video_devdata(file);
int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type);
+ bool consistent = true;

fill_buf_caps(vdev->queue, &p->capabilities);
if (res)
return res;
if (vb2_queue_is_busy(vdev, file))
return -EBUSY;
- res = vb2_core_reqbufs(vdev->queue, p->memory, true, &p->count);
+ if (p->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
+ consistent = false;
+
+ res = vb2_core_reqbufs(vdev->queue, p->memory, consistent, &p->count);
/* If count == 0, then the owner has released all buffers and he
is no longer owner of the queue. Otherwise we have a new owner. */
if (res == 0)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index aaf83e254272..9791e2882382 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1973,9 +1973,6 @@ static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops,

if (ret)
return ret;
-
- CLEAR_AFTER_FIELD(p, capabilities);
-
return ops->vidioc_reqbufs(file, fh, p);
}

@@ -2015,7 +2012,7 @@ static int v4l_create_bufs(const struct v4l2_ioctl_ops *ops,
if (ret)
return ret;

- CLEAR_AFTER_FIELD(create, capabilities);
+ CLEAR_AFTER_FIELD(create, flags);

v4l_sanitize_format(&create->format);

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 72efc1c544cd..169a8cf345ed 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -938,7 +938,10 @@ struct v4l2_requestbuffers {
__u32 type; /* enum v4l2_buf_type */
__u32 memory; /* enum v4l2_memory */
__u32 capabilities;
- __u32 reserved[1];
+ union {
+ __u32 flags;
+ __u32 reserved[1];
+ };
};

/* capabilities for struct v4l2_requestbuffers and v4l2_create_buffers */
@@ -2445,7 +2448,8 @@ struct v4l2_create_buffers {
__u32 memory;
struct v4l2_format format;
__u32 capabilities;
- __u32 reserved[7];
+ __u32 flags;
+ __u32 reserved[6];
};

/*
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:42

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 07/12] videobuf2: do not sync caches when we are allowed not to

Skip ->prepare() or/and ->finish() cache synchronisation if
user-space requested us to do so (or when queue dma direction
permits us to skip cache syncs).

Change-Id: I37c89d666542ed8d536eac329953d921bb1c94b6
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/media/common/videobuf2/videobuf2-core.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 46be9c790ff6..c3637ca0c65b 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -304,8 +304,11 @@ static void __vb2_buf_mem_prepare(struct vb2_buffer *vb)
{
unsigned int plane;

- for (plane = 0; plane < vb->num_planes; ++plane)
- call_void_memop(vb, prepare, vb->planes[plane].mem_priv);
+ if (vb->need_cache_sync_on_prepare) {
+ for (plane = 0; plane < vb->num_planes; ++plane)
+ call_void_memop(vb, prepare,
+ vb->planes[plane].mem_priv);
+ }
vb->synced = 1;
}

@@ -317,8 +320,11 @@ static void __vb2_buf_mem_finish(struct vb2_buffer *vb)
{
unsigned int plane;

- for (plane = 0; plane < vb->num_planes; ++plane)
- call_void_memop(vb, finish, vb->planes[plane].mem_priv);
+ if (vb->need_cache_sync_on_finish) {
+ for (plane = 0; plane < vb->num_planes; ++plane)
+ call_void_memop(vb, finish,
+ vb->planes[plane].mem_priv);
+ }
vb->synced = 0;
}

--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:48

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 06/12] videobuf2: factor out planes prepare/finish functions

Factor out the code, no functional changes.

Change-Id: I47044c0ba57ccc47a5d23f36976ce9f1e3b0f67f
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/common/videobuf2/videobuf2-core.c | 52 +++++++++++--------
1 file changed, 31 insertions(+), 21 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 56fd17eafb6c..46be9c790ff6 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -296,6 +296,32 @@ static void __vb2_buf_dmabuf_put(struct vb2_buffer *vb)
__vb2_plane_dmabuf_put(vb, &vb->planes[plane]);
}

+/*
+ * __vb2_buf_mem_prepare() - call ->prepare() on buffer's private memory
+ * to sync caches
+ */
+static void __vb2_buf_mem_prepare(struct vb2_buffer *vb)
+{
+ unsigned int plane;
+
+ for (plane = 0; plane < vb->num_planes; ++plane)
+ call_void_memop(vb, prepare, vb->planes[plane].mem_priv);
+ vb->synced = 1;
+}
+
+/*
+ * __vb2_buf_mem_finish() - call ->finish on buffer's private memory
+ * to sync caches
+ */
+static void __vb2_buf_mem_finish(struct vb2_buffer *vb)
+{
+ unsigned int plane;
+
+ for (plane = 0; plane < vb->num_planes; ++plane)
+ call_void_memop(vb, finish, vb->planes[plane].mem_priv);
+ vb->synced = 0;
+}
+
/*
* __setup_offsets() - setup unique offsets ("cookies") for every plane in
* the buffer.
@@ -951,7 +977,6 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
{
struct vb2_queue *q = vb->vb2_queue;
unsigned long flags;
- unsigned int plane;

if (WARN_ON(vb->state != VB2_BUF_STATE_ACTIVE))
return;
@@ -971,12 +996,8 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
dprintk(4, "done processing on buffer %d, state: %d\n",
vb->index, state);

- if (state != VB2_BUF_STATE_QUEUED) {
- /* sync buffers */
- for (plane = 0; plane < vb->num_planes; ++plane)
- call_void_memop(vb, finish, vb->planes[plane].mem_priv);
- vb->synced = 0;
- }
+ if (state != VB2_BUF_STATE_QUEUED)
+ __vb2_buf_mem_finish(vb);

spin_lock_irqsave(&q->done_lock, flags);
if (state == VB2_BUF_STATE_QUEUED) {
@@ -1301,7 +1322,6 @@ static int __buf_prepare(struct vb2_buffer *vb)
{
struct vb2_queue *q = vb->vb2_queue;
enum vb2_buffer_state orig_state = vb->state;
- unsigned int plane;
int ret;

if (q->error) {
@@ -1345,11 +1365,7 @@ static int __buf_prepare(struct vb2_buffer *vb)
return ret;
}

- /* sync buffers */
- for (plane = 0; plane < vb->num_planes; ++plane)
- call_void_memop(vb, prepare, vb->planes[plane].mem_priv);
-
- vb->synced = 1;
+ __vb2_buf_mem_prepare(vb);
vb->prepared = 1;
vb->state = orig_state;

@@ -1969,14 +1985,8 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
call_void_vb_qop(vb, buf_request_complete, vb);
}

- if (vb->synced) {
- unsigned int plane;
-
- for (plane = 0; plane < vb->num_planes; ++plane)
- call_void_memop(vb, finish,
- vb->planes[plane].mem_priv);
- vb->synced = 0;
- }
+ if (vb->synced)
+ __vb2_buf_mem_finish(vb);

if (vb->prepared) {
call_void_vb_qop(vb, buf_finish, vb);
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:58:54

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 09/12] videobuf2: let user-space know if driver supports cache hints

Add V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS to fill_buf_caps(), which
is set when queue supports user-space cache management hints.

Change-Id: Ieac93f3726c61fd3b88e02c36980c1f3c7a82ecc
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
Documentation/media/uapi/v4l/vidioc-reqbufs.rst | 7 +++++++
drivers/media/common/videobuf2/videobuf2-v4l2.c | 2 ++
include/uapi/linux/videodev2.h | 1 +
3 files changed, 10 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
index 9741dac0d5b3..80603f57eb8a 100644
--- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
+++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
@@ -165,6 +165,13 @@ aborting or finishing any DMA in progress, an implicit
- Only valid for stateless decoders. If set, then userspace can set the
``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag to hold off on returning the
capture buffer until the OUTPUT timestamp changes.
+ * - ``V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS``
+ - 0x00000040
+ - Set when the queue/buffer support memory consistency and cache
+ management hints. See :ref:`V4L2_FLAG_MEMORY_NON_CONSISTENT`,
+ :ref:`V4L2_BUF_FLAG_NO_CACHE_INVALIDATE` and
+ :ref:`V4L2_BUF_FLAG_NO_CACHE_CLEAN`.
+

Return Value
============
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index eb5d1306cb03..22ae0ff64684 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -698,6 +698,8 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
*caps |= V4L2_BUF_CAP_SUPPORTS_DMABUF;
if (q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)
*caps |= V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF;
+ if (q->allow_cache_hints)
+ *caps |= V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS;
#ifdef CONFIG_MEDIA_CONTROLLER_REQUEST_API
if (q->supports_requests)
*caps |= V4L2_BUF_CAP_SUPPORTS_REQUESTS;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 169a8cf345ed..12b1bd220347 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -951,6 +951,7 @@ struct v4l2_requestbuffers {
#define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3)
#define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
#define V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF (1 << 5)
+#define V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS (1 << 6)

/**
* struct v4l2_plane - plane info for multi-planar buffers
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:59:00

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 12/12] videobuf2: don't test db_attach in dma-contig prepare and finish

We moved cache management decision making to the upper layer and
rely on buffer's need_cache_sync flags and videobuf2 core. If the
upper layer (core) has decided to invoke ->prepare() or ->finish()
then we must sync.

For DMABUF ->need_cache_sync_on_prepare and ->need_cache_sync_on_flush
are always false so videobuf core does not call ->prepare() and
->finish() on such buffers.

Additionally, scratch the DMABUF comment.

Change-Id: I8f6c0b246ccb63f775dcf7881dd5f848c38e7604
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/media/common/videobuf2/videobuf2-dma-contig.c | 6 ++----
drivers/media/common/videobuf2/videobuf2-dma-sg.c | 8 --------
2 files changed, 2 insertions(+), 12 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index a387260fb321..6ea0961149d7 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -100,8 +100,7 @@ static void vb2_dc_prepare(void *buf_priv)
struct vb2_dc_buf *buf = buf_priv;
struct sg_table *sgt = buf->dma_sgt;

- /* DMABUF exporter will flush the cache for us */
- if (!sgt || buf->db_attach)
+ if (!sgt)
return;

dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->orig_nents,
@@ -113,8 +112,7 @@ static void vb2_dc_finish(void *buf_priv)
struct vb2_dc_buf *buf = buf_priv;
struct sg_table *sgt = buf->dma_sgt;

- /* DMABUF exporter will flush the cache for us */
- if (!sgt || buf->db_attach)
+ if (!sgt)
return;

dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->orig_nents, buf->dma_dir);
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index bfc99a0cb7b9..1fd25eda0bf2 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -198,10 +198,6 @@ static void vb2_dma_sg_prepare(void *buf_priv)
struct vb2_dma_sg_buf *buf = buf_priv;
struct sg_table *sgt = buf->dma_sgt;

- /* DMABUF exporter will flush the cache for us */
- if (buf->db_attach)
- return;
-
dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->orig_nents,
buf->dma_dir);
}
@@ -211,10 +207,6 @@ static void vb2_dma_sg_finish(void *buf_priv)
struct vb2_dma_sg_buf *buf = buf_priv;
struct sg_table *sgt = buf->dma_sgt;

- /* DMABUF exporter will flush the cache for us */
- if (buf->db_attach)
- return;
-
dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->orig_nents, buf->dma_dir);
}

--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:59:00

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 11/12] videobuf2: add begin/end cpu_access callbacks to dma-sg

Provide begin_cpu_access() and end_cpu_access() dma_buf_ops
callbacks for cache synchronisation on exported buffers.

V4L2_FLAG_MEMORY_NON_CONSISTENT has no effect on dma-sg buffers.
dma-sg allocates memory using the page allocator directly, so
there is no memory consistency guarantee.

Change-Id: Ia0d9d72a8c2a9fe3264ac148f59201573289ed2c
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/common/videobuf2/videobuf2-dma-sg.c | 22 +++++++++++++++++++
1 file changed, 22 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index 6db60e9d5183..bfc99a0cb7b9 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -470,6 +470,26 @@ static void vb2_dma_sg_dmabuf_ops_release(struct dma_buf *dbuf)
vb2_dma_sg_put(dbuf->priv);
}

+static int vb2_dma_sg_dmabuf_ops_begin_cpu_access(struct dma_buf *dbuf,
+ enum dma_data_direction direction)
+{
+ struct vb2_dma_sg_buf *buf = dbuf->priv;
+ struct sg_table *sgt = buf->dma_sgt;
+
+ dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
+ return 0;
+}
+
+static int vb2_dma_sg_dmabuf_ops_end_cpu_access(struct dma_buf *dbuf,
+ enum dma_data_direction direction)
+{
+ struct vb2_dma_sg_buf *buf = dbuf->priv;
+ struct sg_table *sgt = buf->dma_sgt;
+
+ dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
+ return 0;
+}
+
static void *vb2_dma_sg_dmabuf_ops_vmap(struct dma_buf *dbuf)
{
struct vb2_dma_sg_buf *buf = dbuf->priv;
@@ -488,6 +508,8 @@ static const struct dma_buf_ops vb2_dma_sg_dmabuf_ops = {
.detach = vb2_dma_sg_dmabuf_ops_detach,
.map_dma_buf = vb2_dma_sg_dmabuf_ops_map,
.unmap_dma_buf = vb2_dma_sg_dmabuf_ops_unmap,
+ .begin_cpu_access = vb2_dma_sg_dmabuf_ops_begin_cpu_access,
+ .end_cpu_access = vb2_dma_sg_dmabuf_ops_end_cpu_access,
.vmap = vb2_dma_sg_dmabuf_ops_vmap,
.mmap = vb2_dma_sg_dmabuf_ops_mmap,
.release = vb2_dma_sg_dmabuf_ops_release,
--
2.25.0.341.g760bfbb309-goog

2020-02-04 02:59:45

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 08/12] videobuf2: check ->synced flag in prepare() and finish()

This simplifies the code a tiny bit and let's us to avoid
unneeded ->prepare()/->finish() calls.

Change-Id: Ia7c8b4d75a72d0fe1bf37382780e173f6dd9b7ff
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/media/common/videobuf2/videobuf2-core.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index c3637ca0c65b..631667db7094 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -304,6 +304,9 @@ static void __vb2_buf_mem_prepare(struct vb2_buffer *vb)
{
unsigned int plane;

+ if (vb->synced)
+ return;
+
if (vb->need_cache_sync_on_prepare) {
for (plane = 0; plane < vb->num_planes; ++plane)
call_void_memop(vb, prepare,
@@ -320,6 +323,9 @@ static void __vb2_buf_mem_finish(struct vb2_buffer *vb)
{
unsigned int plane;

+ if (!vb->synced)
+ return;
+
if (vb->need_cache_sync_on_finish) {
for (plane = 0; plane < vb->num_planes; ++plane)
call_void_memop(vb, finish,
@@ -1991,8 +1997,7 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
call_void_vb_qop(vb, buf_request_complete, vb);
}

- if (vb->synced)
- __vb2_buf_mem_finish(vb);
+ __vb2_buf_mem_finish(vb);

if (vb->prepared) {
call_void_vb_qop(vb, buf_finish, vb);
--
2.25.0.341.g760bfbb309-goog

2020-02-04 03:00:04

by Sergey Senozhatsky

Subject: [RFC][PATCHv2 10/12] videobuf2: add begin/end cpu_access callbacks to dma-contig

Provide begin_cpu_access() and end_cpu_access() callbacks for
cache synchronisation on exported buffers.

The patch also adds a new helper function - vb2_dc_buffer_consistent(),
which returns true is if the buffer is consistent (DMA_ATTR_NON_CONSISTENT
bit cleared), so then we don't need to sync anything.

Change-Id: I653ee20302014920b4705f3eba27c0b1232ab89d
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../common/videobuf2/videobuf2-dma-contig.c | 33 +++++++++++++++++++
1 file changed, 33 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index d0c9dffe49e5..a387260fb321 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -42,6 +42,11 @@ struct vb2_dc_buf {
struct dma_buf_attachment *db_attach;
};

+static inline bool vb2_dc_buffer_consistent(unsigned long attr)
+{
+ return !(attr & DMA_ATTR_NON_CONSISTENT);
+}
+
/*********************************************/
/* scatterlist table functions */
/*********************************************/
@@ -335,6 +340,32 @@ static void vb2_dc_dmabuf_ops_release(struct dma_buf *dbuf)
vb2_dc_put(dbuf->priv);
}

+static int vb2_dc_dmabuf_ops_begin_cpu_access(struct dma_buf *dbuf,
+ enum dma_data_direction direction)
+{
+ struct vb2_dc_buf *buf = dbuf->priv;
+ struct sg_table *sgt = buf->dma_sgt;
+
+ if (vb2_dc_buffer_consistent(buf->attrs))
+ return 0;
+
+ dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
+ return 0;
+}
+
+static int vb2_dc_dmabuf_ops_end_cpu_access(struct dma_buf *dbuf,
+ enum dma_data_direction direction)
+{
+ struct vb2_dc_buf *buf = dbuf->priv;
+ struct sg_table *sgt = buf->dma_sgt;
+
+ if (vb2_dc_buffer_consistent(buf->attrs))
+ return 0;
+
+ dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
+ return 0;
+}
+
static void *vb2_dc_dmabuf_ops_vmap(struct dma_buf *dbuf)
{
struct vb2_dc_buf *buf = dbuf->priv;
@@ -353,6 +384,8 @@ static const struct dma_buf_ops vb2_dc_dmabuf_ops = {
.detach = vb2_dc_dmabuf_ops_detach,
.map_dma_buf = vb2_dc_dmabuf_ops_map,
.unmap_dma_buf = vb2_dc_dmabuf_ops_unmap,
+ .begin_cpu_access = vb2_dc_dmabuf_ops_begin_cpu_access,
+ .end_cpu_access = vb2_dc_dmabuf_ops_end_cpu_access,
.vmap = vb2_dc_dmabuf_ops_vmap,
.mmap = vb2_dc_dmabuf_ops_mmap,
.release = vb2_dc_dmabuf_ops_release,
--
2.25.0.341.g760bfbb309-goog

2020-02-05 08:15:30

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

On (20/02/04 11:56), Sergey Senozhatsky wrote:
> +static void set_buffer_cache_hints(struct vb2_queue *q,
> + struct vb2_buffer *vb,
> + struct v4l2_buffer *b)
> +{
> + /*
> + * DMA exporter should take care of cache syncs, so we can avoid
> + * explicit ->prepare()/->finish() syncs. For other ->memory types
> + * we always need ->prepare() or/and ->finish() cache sync.
> + */
> + if (q->memory == VB2_MEMORY_DMABUF) {
> + vb->need_cache_sync_on_finish = 0;
> + vb->need_cache_sync_on_prepare = 0;
> + return;
> + }
> +
> + if (!q->allow_cache_hints)
> + return;
> +
> + vb->need_cache_sync_on_prepare = 1;
> + /*
> + * ->finish() cache sync can be avoided when queue direction is
> + * TO_DEVICE.
> + */
> + if (q->dma_dir != DMA_TO_DEVICE)
> + vb->need_cache_sync_on_finish = 1;
> + else
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
> + vb->need_cache_sync_on_prepare = 0;
> +}

Last minute changes (tm), sorry. This is not right.


====8<====8<====

From: Sergey Senozhatsky <[email protected]>
Subject: [PATCH] videobuf2: handle V4L2 buffer cache flags

Set video buffer cache management flags corresponding to V4L2 cache
flags.

Both ->prepare() and ->finish() cache management hints should be
passed during this stage (buffer preparation), because there is no
other way for user-space to skip ->finish() cache flush.

There are two possible alternative approaches:
- The first one is to move cache sync from ->finish() to dqbuf().
But this breaks some drivers, that need to fix-up buffers before
dequeueing them.

- The second one is to move ->finish() call from ->done() to dqbuf.

Signed-off-by: Sergey Senozhatsky <[email protected]>
---
.../media/common/videobuf2/videobuf2-v4l2.c | 36 +++++++++++++++++++
1 file changed, 36 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index eb5d5db96552..8ef57496b34a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -337,6 +337,41 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
return 0;
}

+static void set_buffer_cache_hints(struct vb2_queue *q,
+ struct vb2_buffer *vb,
+ struct v4l2_buffer *b)
+{
+ /*
+ * DMA exporter should take care of cache syncs, so we can avoid
+ * explicit ->prepare()/->finish() syncs. For other ->memory types
+ * we always need ->prepare() or/and ->finish() cache sync.
+ */
+ if (q->memory == VB2_MEMORY_DMABUF) {
+ vb->need_cache_sync_on_finish = 0;
+ vb->need_cache_sync_on_prepare = 0;
+ return;
+ }
+
+ vb->need_cache_sync_on_prepare = 1;
+ vb->need_cache_sync_on_finish = 1;
+
+ if (!q->allow_cache_hints)
+ return;
+
+ /*
+ * ->finish() cache sync can be avoided when queue direction is
+ * TO_DEVICE.
+ */
+ if (q->dma_dir == DMA_TO_DEVICE)
+ vb->need_cache_sync_on_finish = 0;
+
+ if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
+ vb->need_cache_sync_on_finish = 0;
+
+ if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
+ vb->need_cache_sync_on_prepare = 0;
+}
+
static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b, bool is_prepare,
struct media_request **p_req)
@@ -381,6 +416,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
}

if (!vb->prepared) {
+ set_buffer_cache_hints(q, vb, b);
/* Copy relevant information provided by the userspace */
memset(vbuf->planes, 0,
sizeof(vbuf->planes[0]) * vb->num_planes);
--
2.25.0.341.g760bfbb309-goog

2020-02-13 07:09:00

by Tomasz Figa

Subject: Re: [RFC][PATCHv2 03/12] videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On Tue, Feb 4, 2020 at 11:57 AM Sergey Senozhatsky
<[email protected]> wrote:
>
> By setting or clearing V4L2_FLAG_MEMORY_NON_CONSISTENT flag
> user-space should be able to set or clear queue's NON_CONSISTENT
> ->dma_attrs. Queue's ->dma_attrs are passed to the underlying
> allocator in __vb2_buf_mem_alloc(), so thus user-space is able
> to request vb2 buffer's memory to be either consistent (coherent)
> or non-consistent.
>
> Change-Id: Ib333081c482e23c9a89386078293e19c3fd59076
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> Documentation/media/uapi/v4l/buffer.rst | 27 +++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 2 ++
> 2 files changed, 29 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..af007daf0591 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -705,6 +705,33 @@ Buffer Flags
>
> .. c:type:: v4l2_memory
>
> +Memory Consistency Flags
> +========================
> +
> +.. tabularcolumns:: |p{7.0cm}|p{2.2cm}|p{8.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 3 1 4
> +
> + * .. _`V4L2_FLAG_MEMORY_NON_CONSISTENT`:
> +
> + - ``V4L2_FLAG_MEMORY_NON_CONSISTENT``
> + - 0x00000001
> + - vb2 buffer is allocated either in consistent (it will be automatically
> + coherent between CPU and bus) or non-consistent memory. The latter
> + can provide performance gains, for instance CPU cache sync/flush
> + operations can be avoided if the buffer is accesed by the corresponding
> + device only and CPU does not read/write to/from that buffer. However,
> + this requires extra care from the driver -- it must guarantee memory
> + consistency by issuing cache flush/sync when consistency is needed.
> + If this flag is set V4L2 will attempt to allocate vb2 buffer in
> + non-consistent memory. This flag is ignored if queue does not report
> + :ret:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.

nit: Should the patch adding the capability flag precede this one?

> +
> enum v4l2_memory
> ================
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 5f9357dcb060..72efc1c544cd 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -189,6 +189,8 @@ enum v4l2_memory {
> V4L2_MEMORY_DMABUF = 4,
> };
>
> +#define V4L2_FLAG_MEMORY_NON_CONSISTENT (1 << 0)
> +
> /* see also http://vektor.theorem.ca/graphics/ycbcr/ */
> enum v4l2_colorspace {
> /*
> --
> 2.25.0.341.g760bfbb309-goog
>

2020-02-19 08:06:25

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 01/12] videobuf2: add cache management members

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Extend vb2_buffer and vb2_queue structs with cache management
> members.
>
> V4L2 UAPI already contains two buffer flags which user-space,
> supposedly, can use to control buffer cache sync:
>
> - V4L2_BUF_FLAG_NO_CACHE_INVALIDATE
> - V4L2_BUF_FLAG_NO_CACHE_CLEAN
>
> None of these, however, do anything at the moment. This patch
> set is intended to change it.
>
> Since user-space cache management hints are supposed to be
> implemented on a per-buffer basis we need to extend vb2_buffer
> struct with two new memebers ->need_cache_sync_on_prepare and
> ->need_cache_sync_on_finish, which will store corresponding
> user-space hints.
>
> In order to preserve the existing behaviour, user-space cache
> managements flags will be handled only by those drivers that
> permit user-space cache hints. That's the purpose of vb2_queue
> ->allow_cache_hints member. Driver must set ->allow_cache_hints
> during queue initialisation to enable cache management hints
> mechanism.
>
> Only drivers that set ->allow_cache_hints during queue initialisation
> will handle user-space cache management hints. Otherwise hints
> will be ignored.
>
> Change-Id: I52beec2a0d021b7a3715b4f6ae4bfd9dc5e94f0d
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> include/media/videobuf2-core.h | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index a2b2208b02da..026004180440 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -263,6 +263,10 @@ struct vb2_buffer {
> * after the 'buf_finish' op is called.
> * copied_timestamp: the timestamp of this capture buffer was copied
> * from an output buffer.
> + * need_cache_sync_on_prepare: do not sync/invalidate cache from
> + * buffer's ->prepare() callback.
> + * need_cache_sync_on_finish: do not sync/invalidate cache from buffer's
> + * ->finish() callback.

Shouldn't 'do not' be deleted from the flag descriptions? If the flag is set,
then you need to sync/validate, right?

Regards,

Hans

> * queued_entry: entry on the queued buffers list, which holds
> * all buffers queued from userspace
> * done_entry: entry on the list that stores all buffers ready
> @@ -273,6 +277,8 @@ struct vb2_buffer {
> unsigned int synced:1;
> unsigned int prepared:1;
> unsigned int copied_timestamp:1;
> + unsigned int need_cache_sync_on_prepare:1;
> + unsigned int need_cache_sync_on_finish:1;
>
> struct vb2_plane planes[VB2_MAX_PLANES];
> struct list_head queued_entry;
> @@ -491,6 +497,9 @@ struct vb2_buf_ops {
> * @uses_requests: requests are used for this queue. Set to 1 the first time
> * a request is queued. Set to 0 when the queue is canceled.
> * If this is 1, then you cannot queue buffers directly.
> + * @allow_cache_hints: when set user-space can pass cache management hints in
> + * order to skip cache flush/invalidation on ->prepare() or/and
> + * ->finish().
> * @lock: pointer to a mutex that protects the &struct vb2_queue. The
> * driver can set this to a mutex to let the v4l2 core serialize
> * the queuing ioctls. If the driver wants to handle locking
> @@ -564,6 +573,7 @@ struct vb2_queue {
> unsigned requires_requests:1;
> unsigned uses_qbuf:1;
> unsigned uses_requests:1;
> + unsigned allow_cache_hints:1;
>
> struct mutex *lock;
> void *owner;
>

2020-02-19 08:08:58

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Set video buffer cache management flags corresponding to V4L2 cache
> flags.
>
> Both ->prepare() and ->finish() cache management hints should be
> passed during this stage (buffer preparation), because there is no
> other way for user-space to skip ->finish() cache flush.
>
> There are two possible alternative approaches:
> - The first one is to move cache sync from ->finish() to dqbuf().
> But this breaks some drivers, that need to fix-up buffers before
> dequeueing them.
>
> - The second one is to move ->finish() call from ->done() to dqbuf.
>
> Change-Id: I3bef1d1fb93a5fba290ce760eaeecdc8e7d6885a
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> .../media/common/videobuf2/videobuf2-v4l2.c | 36 +++++++++++++++++++
> 1 file changed, 36 insertions(+)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index eb5d5db96552..2da06a2ad6c4 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -337,6 +337,41 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
> return 0;
> }
>
> +static void set_buffer_cache_hints(struct vb2_queue *q,
> + struct vb2_buffer *vb,
> + struct v4l2_buffer *b)
> +{
> + /*
> + * DMA exporter should take care of cache syncs, so we can avoid
> + * explicit ->prepare()/->finish() syncs. For other ->memory types
> + * we always need ->prepare() or/and ->finish() cache sync.
> + */
> + if (q->memory == VB2_MEMORY_DMABUF) {
> + vb->need_cache_sync_on_finish = 0;
> + vb->need_cache_sync_on_prepare = 0;
> + return;
> + }
> +
> + if (!q->allow_cache_hints)
> + return;
> +
> + vb->need_cache_sync_on_prepare = 1;

This needs a comment explaining why prepare is set to 1 by default. I remember
we discussed this earlier, and the conclusion of that discussion needs to be
documented here in a comment.

Regards,

Hans

> + /*
> + * ->finish() cache sync can be avoided when queue direction is
> + * TO_DEVICE.
> + */
> + if (q->dma_dir != DMA_TO_DEVICE)
> + vb->need_cache_sync_on_finish = 1;
> + else
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
> + vb->need_cache_sync_on_prepare = 0;
> +}
> +
> static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b, bool is_prepare,
> struct media_request **p_req)
> @@ -381,6 +416,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
> }
>
> if (!vb->prepared) {
> + set_buffer_cache_hints(q, vb, b);
> /* Copy relevant information provided by the userspace */
> memset(vbuf->planes, 0,
> sizeof(vbuf->planes[0]) * vb->num_planes);
>

2020-02-19 08:15:28

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

On (20/02/19 09:07), Hans Verkuil wrote:
[..]
> > +static void set_buffer_cache_hints(struct vb2_queue *q,
> > + struct vb2_buffer *vb,
> > + struct v4l2_buffer *b)
> > +{
> > + /*
> > + * DMA exporter should take care of cache syncs, so we can avoid
> > + * explicit ->prepare()/->finish() syncs. For other ->memory types
> > + * we always need ->prepare() or/and ->finish() cache sync.
> > + */
> > + if (q->memory == VB2_MEMORY_DMABUF) {
> > + vb->need_cache_sync_on_finish = 0;
> > + vb->need_cache_sync_on_prepare = 0;
> > + return;
> > + }
> > +
> > + if (!q->allow_cache_hints)
> > + return;
> > +
> > + vb->need_cache_sync_on_prepare = 1;
>
> This needs a comment explaining why prepare is set to 1 by default. I remember
> we discussed this earlier, and the conclusion of that discussion needs to be
> documented here in a comment.

Please ignore this patch. There is a follow up which sets _both_
flags by default. The purpose is to preserve the existing behaviour,
we can do all sorts of incremental changes (clear flags in more cases,
etc.) later on. Do you want me to document this in the code?

-ss

2020-02-19 08:18:11

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 01/12] videobuf2: add cache management members

On (20/02/19 09:05), Hans Verkuil wrote:
> On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:

[..]

> > diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> > index a2b2208b02da..026004180440 100644
> > --- a/include/media/videobuf2-core.h
> > +++ b/include/media/videobuf2-core.h
> > @@ -263,6 +263,10 @@ struct vb2_buffer {
> > * after the 'buf_finish' op is called.
> > * copied_timestamp: the timestamp of this capture buffer was copied
> > * from an output buffer.
> > + * need_cache_sync_on_prepare: do not sync/invalidate cache from
> > + * buffer's ->prepare() callback.
> > + * need_cache_sync_on_finish: do not sync/invalidate cache from buffer's
> > + * ->finish() callback.
>
> Shouldn't 'do not' be deleted from the flag descriptions? If the flag is set,
> then you need to sync/validate, right?

Hmm, kind of work both ways. Maybe the wording can be more specific,
e.g. "Do/skip cache sync/invalidation" even more detailed "When set
perform cache sync/invalidation from ..."

-ss

2020-02-19 08:20:47

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 03/12] videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> By setting or clearing V4L2_FLAG_MEMORY_NON_CONSISTENT flag
> user-space should be able to set or clear queue's NON_CONSISTENT
> ->dma_attrs. Queue's ->dma_attrs are passed to the underlying
> allocator in __vb2_buf_mem_alloc(), so thus user-space is able
> to request vb2 buffer's memory to be either consistent (coherent)
> or non-consistent.
>
> Change-Id: Ib333081c482e23c9a89386078293e19c3fd59076
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> Documentation/media/uapi/v4l/buffer.rst | 27 +++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 2 ++
> 2 files changed, 29 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..af007daf0591 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -705,6 +705,33 @@ Buffer Flags
>
> .. c:type:: v4l2_memory
>
> +Memory Consistency Flags

This new part should be added *above* the '.. c:type:: v4l2_memory' line.

You also need to add '.. _memory-flags:' just before this section so that
you can link to it from the create_bufs and reqbufs ioctl descriptions.

> +========================
> +
> +.. tabularcolumns:: |p{7.0cm}|p{2.2cm}|p{8.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 3 1 4
> +
> + * .. _`V4L2_FLAG_MEMORY_NON_CONSISTENT`:
> +
> + - ``V4L2_FLAG_MEMORY_NON_CONSISTENT``
> + - 0x00000001
> + - vb2 buffer is allocated either in consistent (it will be automatically
> + coherent between CPU and bus) or non-consistent memory. The latter
> + can provide performance gains, for instance CPU cache sync/flush
> + operations can be avoided if the buffer is accesed by the corresponding

accesed -> accessed

> + device only and CPU does not read/write to/from that buffer. However,
> + this requires extra care from the driver -- it must guarantee memory
> + consistency by issuing cache flush/sync when consistency is needed.
> + If this flag is set V4L2 will attempt to allocate vb2 buffer in
> + non-consistent memory. This flag is ignored if queue does not report
> + :ret:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.

V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS is not defined. Add that first in a separate
patch.

Also update the current description of the V4L2_BUF_FLAG_NO_CACHE_INVALIDATE/CLEAN
flags to indicate that they are only valid if V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS
is set (that should be done in the patch adding the V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS
capability).

Regards,

Hans

> +
> enum v4l2_memory
> ================
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 5f9357dcb060..72efc1c544cd 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -189,6 +189,8 @@ enum v4l2_memory {
> V4L2_MEMORY_DMABUF = 4,
> };
>
> +#define V4L2_FLAG_MEMORY_NON_CONSISTENT (1 << 0)
> +
> /* see also http://vektor.theorem.ca/graphics/ycbcr/ */
> enum v4l2_colorspace {
> /*
>

2020-02-19 08:26:00

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

On 2/19/20 9:13 AM, Sergey Senozhatsky wrote:
> On (20/02/19 09:07), Hans Verkuil wrote:
> [..]
>>> +static void set_buffer_cache_hints(struct vb2_queue *q,
>>> + struct vb2_buffer *vb,
>>> + struct v4l2_buffer *b)
>>> +{
>>> + /*
>>> + * DMA exporter should take care of cache syncs, so we can avoid
>>> + * explicit ->prepare()/->finish() syncs. For other ->memory types
>>> + * we always need ->prepare() or/and ->finish() cache sync.
>>> + */
>>> + if (q->memory == VB2_MEMORY_DMABUF) {
>>> + vb->need_cache_sync_on_finish = 0;
>>> + vb->need_cache_sync_on_prepare = 0;
>>> + return;
>>> + }
>>> +
>>> + if (!q->allow_cache_hints)
>>> + return;
>>> +
>>> + vb->need_cache_sync_on_prepare = 1;
>>
>> This needs a comment explaining why prepare is set to 1 by default. I remember
>> we discussed this earlier, and the conclusion of that discussion needs to be
>> documented here in a comment.
>
> Please ignore this patch. There is a follow up which sets _both_
> flags by default. The purpose is to preserve the existing behaviour,
> we can do all sorts of incremental changes (clear flags in more cases,
> etc.) later on. Do you want me to document this in the code?

Yes please!

Regards,

Hans

>
> -ss
>

2020-02-19 08:27:19

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> This patch lets user-space to request a non-consistent memory
> allocation during CREATE_BUFS and REQBUFS ioctl calls.
>
> = CREATE_BUFS
>
> struct v4l2_create_buffers has seven 4-byte reserved areas,
> so reserved[0] is renamed to ->flags. The struct, thus, now
> has six reserved 4-byte regions.
>
> = REQBUFS
>
> We use one bit of a ->reserved[1] member of struct v4l2_requestbuffers,
> which is now renamed to ->flags. Unlike v4l2_create_buffers, struct
> v4l2_requestbuffers does not have enough reserved room. Therefore for
> backward compatibility ->reserved and ->flags were put into anonymous
> union.
>
> Change-Id: I0eaab3428de499ce1bce6fc6b26c5ca5ff405882
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> .../media/uapi/v4l/vidioc-create-bufs.rst | 9 +++++++-
> .../media/uapi/v4l/vidioc-reqbufs.rst | 15 ++++++++++---
> .../media/common/videobuf2/videobuf2-v4l2.c | 21 ++++++++++++++++---
> drivers/media/v4l2-core/v4l2-ioctl.c | 5 +----
> include/uapi/linux/videodev2.h | 8 +++++--
> 5 files changed, 45 insertions(+), 13 deletions(-)
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> index bd08e4f77ae4..68185e94b686 100644
> --- a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> @@ -121,7 +121,14 @@ than the number requested.
> other changes, then set ``count`` to 0, ``memory`` to
> ``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type.
> * - __u32
> - - ``reserved``\ [7]
> + - ``flags``
> + - Specifies additional buffer management attributes. Valid only when
> + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
> + Old drivers and applications must set it to zero.

Drop the 'Valid only' sentence. The V4L2_FLAG_MEMORY_NON_CONSISTENT depends
on that capability, but other flags added in the future may not.

Inside add a reference to the memory flags section created in patch 3.

> +
> +
> + * - __u32
> + - ``reserved``\ [6]
> - A place holder for future extensions. Drivers and applications
> must set the array to zero.
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> index d0c643db477a..9741dac0d5b3 100644
> --- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> @@ -112,10 +112,19 @@ aborting or finishing any DMA in progress, an implicit
> ``V4L2_MEMORY_MMAP`` and ``type`` set to the buffer type. This will
> free any previously allocated buffers, so this is typically something
> that will be done at the start of the application.
> - * - __u32
> + * - union
> + - (anonymous)
> + * -
> + - __u32
> + - ``flags``\ [1]
> + - Specifies additional buffer management attributes. Valid only when
> + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.

Same comment as above.

> + Old drivers and applications must set it to zero.
> +
> + * -
> + - __u32
> - ``reserved``\ [1]
> - - A place holder for future extensions. Drivers and applications
> - must set the array to zero.
> + - Kept for backwards compatibility. Use ``flags`` instead.
>
> .. tabularcolumns:: |p{6.1cm}|p{2.2cm}|p{8.7cm}|
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 7cdfcd1baf82..eb5d1306cb03 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -707,9 +707,15 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
> int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
> {
> int ret = vb2_verify_memory_type(q, req->memory, req->type);
> + bool consistent = true;
> +
> + if (req->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;
>
> fill_buf_caps(q, &req->capabilities);
> - return ret ? ret : vb2_core_reqbufs(q, req->memory, true, &req->count);
> + if (ret)
> + return ret;
> + return vb2_core_reqbufs(q, req->memory, consistent, &req->count);
> }
> EXPORT_SYMBOL_GPL(vb2_reqbufs);
>
> @@ -738,6 +744,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> unsigned requested_sizes[VIDEO_MAX_PLANES];
> struct v4l2_format *f = &create->format;
> int ret = vb2_verify_memory_type(q, create->memory, f->type);
> + bool consistent = true;
> unsigned i;
>
> fill_buf_caps(q, &create->capabilities);
> @@ -783,7 +790,11 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> for (i = 0; i < requested_planes; i++)
> if (requested_sizes[i] == 0)
> return -EINVAL;
> - return ret ? ret : vb2_core_create_bufs(q, create->memory, true,
> +
> + if (create->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;
> +
> + return ret ? ret : vb2_core_create_bufs(q, create->memory, consistent,
> &create->count, requested_planes, requested_sizes);
> }
> EXPORT_SYMBOL_GPL(vb2_create_bufs);
> @@ -953,13 +964,17 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv,
> {
> struct video_device *vdev = video_devdata(file);
> int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type);
> + bool consistent = true;
>
> fill_buf_caps(vdev->queue, &p->capabilities);
> if (res)
> return res;
> if (vb2_queue_is_busy(vdev, file))
> return -EBUSY;
> - res = vb2_core_reqbufs(vdev->queue, p->memory, true, &p->count);
> + if (p->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;
> +
> + res = vb2_core_reqbufs(vdev->queue, p->memory, consistent, &p->count);
> /* If count == 0, then the owner has released all buffers and he
> is no longer owner of the queue. Otherwise we have a new owner. */
> if (res == 0)
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index aaf83e254272..9791e2882382 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1973,9 +1973,6 @@ static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops,
>
> if (ret)
> return ret;
> -
> - CLEAR_AFTER_FIELD(p, capabilities);
> -
> return ops->vidioc_reqbufs(file, fh, p);
> }
>
> @@ -2015,7 +2012,7 @@ static int v4l_create_bufs(const struct v4l2_ioctl_ops *ops,
> if (ret)
> return ret;
>
> - CLEAR_AFTER_FIELD(create, capabilities);
> + CLEAR_AFTER_FIELD(create, flags);
>
> v4l_sanitize_format(&create->format);
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 72efc1c544cd..169a8cf345ed 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -938,7 +938,10 @@ struct v4l2_requestbuffers {
> __u32 type; /* enum v4l2_buf_type */
> __u32 memory; /* enum v4l2_memory */
> __u32 capabilities;
> - __u32 reserved[1];
> + union {
> + __u32 flags;
> + __u32 reserved[1];
> + };

How about this:

__u8 flags;
__u8 reserved[3];

That avoids the anonymous union and allows some space for future additions.

Regards,

Hans

> };
>
> /* capabilities for struct v4l2_requestbuffers and v4l2_create_buffers */
> @@ -2445,7 +2448,8 @@ struct v4l2_create_buffers {
> __u32 memory;
> struct v4l2_format format;
> __u32 capabilities;
> - __u32 reserved[7];
> + __u32 flags;
> + __u32 reserved[6];
> };
>
> /*
>

2020-02-19 08:27:43

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 01/12] videobuf2: add cache management members

On 2/19/20 9:16 AM, Sergey Senozhatsky wrote:
> On (20/02/19 09:05), Hans Verkuil wrote:
>> On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
>
> [..]
>
>>> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index a2b2208b02da..026004180440 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -263,6 +263,10 @@ struct vb2_buffer {
>>> * after the 'buf_finish' op is called.
>>> * copied_timestamp: the timestamp of this capture buffer was copied
>>> * from an output buffer.
>>> + * need_cache_sync_on_prepare: do not sync/invalidate cache from
>>> + * buffer's ->prepare() callback.
>>> + * need_cache_sync_on_finish: do not sync/invalidate cache from buffer's
>>> + * ->finish() callback.
>>
>> Shouldn't 'do not' be deleted from the flag descriptions? If the flag is set,
>> then you need to sync/validate, right?
>
> Hmm, kind of work both ways. Maybe the wording can be more specific,
> e.g. "Do/skip cache sync/invalidation" even more detailed "When set
> perform cache sync/invalidation from ..."

"When set..." works well. It's explicit.

Regards,

Hans

>
> -ss
>

2020-02-19 08:34:03

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 09/12] videobuf2: let user-space know if driver supports cache hints

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Add V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS to fill_buf_caps(), which
> is set when queue supports user-space cache management hints.

Ah, you add the capability here :-)

This should be moved forward in the series. Actually, I think this should
be merged with the first patch of the series.

Regards,

Hans

>
> Change-Id: Ieac93f3726c61fd3b88e02c36980c1f3c7a82ecc
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> Documentation/media/uapi/v4l/vidioc-reqbufs.rst | 7 +++++++
> drivers/media/common/videobuf2/videobuf2-v4l2.c | 2 ++
> include/uapi/linux/videodev2.h | 1 +
> 3 files changed, 10 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> index 9741dac0d5b3..80603f57eb8a 100644
> --- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> @@ -165,6 +165,13 @@ aborting or finishing any DMA in progress, an implicit
> - Only valid for stateless decoders. If set, then userspace can set the
> ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag to hold off on returning the
> capture buffer until the OUTPUT timestamp changes.
> + * - ``V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS``
> + - 0x00000040
> + - Set when the queue/buffer support memory consistency and cache
> + management hints. See :ref:`V4L2_FLAG_MEMORY_NON_CONSISTENT`,
> + :ref:`V4L2_BUF_FLAG_NO_CACHE_INVALIDATE` and
> + :ref:`V4L2_BUF_FLAG_NO_CACHE_CLEAN`.
> +
>
> Return Value
> ============
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index eb5d1306cb03..22ae0ff64684 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -698,6 +698,8 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
> *caps |= V4L2_BUF_CAP_SUPPORTS_DMABUF;
> if (q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)
> *caps |= V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF;
> + if (q->allow_cache_hints)
> + *caps |= V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS;
> #ifdef CONFIG_MEDIA_CONTROLLER_REQUEST_API
> if (q->supports_requests)
> *caps |= V4L2_BUF_CAP_SUPPORTS_REQUESTS;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 169a8cf345ed..12b1bd220347 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -951,6 +951,7 @@ struct v4l2_requestbuffers {
> #define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3)
> #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> #define V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF (1 << 5)
> +#define V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS (1 << 6)
>
> /**
> * struct v4l2_plane - plane info for multi-planar buffers
>

2020-02-19 08:35:41

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 11/12] videobuf2: add begin/end cpu_access callbacks to dma-sg

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Provide begin_cpu_access() and end_cpu_access() dma_buf_ops
> callbacks for cache synchronisation on exported buffers.
>
> V4L2_FLAG_MEMORY_NON_CONSISTENT has no effect on dma-sg buffers.
> dma-sg allocates memory using the page allocator directly, so
> there is no memory consistency guarantee.

This should also be a comment in the code.

Regards,

Hans

>
> Change-Id: Ia0d9d72a8c2a9fe3264ac148f59201573289ed2c
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> .../media/common/videobuf2/videobuf2-dma-sg.c | 22 +++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index 6db60e9d5183..bfc99a0cb7b9 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -470,6 +470,26 @@ static void vb2_dma_sg_dmabuf_ops_release(struct dma_buf *dbuf)
> vb2_dma_sg_put(dbuf->priv);
> }
>
> +static int vb2_dma_sg_dmabuf_ops_begin_cpu_access(struct dma_buf *dbuf,
> + enum dma_data_direction direction)
> +{
> + struct vb2_dma_sg_buf *buf = dbuf->priv;
> + struct sg_table *sgt = buf->dma_sgt;
> +
> + dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
> + return 0;
> +}
> +
> +static int vb2_dma_sg_dmabuf_ops_end_cpu_access(struct dma_buf *dbuf,
> + enum dma_data_direction direction)
> +{
> + struct vb2_dma_sg_buf *buf = dbuf->priv;
> + struct sg_table *sgt = buf->dma_sgt;
> +
> + dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);
> + return 0;
> +}
> +
> static void *vb2_dma_sg_dmabuf_ops_vmap(struct dma_buf *dbuf)
> {
> struct vb2_dma_sg_buf *buf = dbuf->priv;
> @@ -488,6 +508,8 @@ static const struct dma_buf_ops vb2_dma_sg_dmabuf_ops = {
> .detach = vb2_dma_sg_dmabuf_ops_detach,
> .map_dma_buf = vb2_dma_sg_dmabuf_ops_map,
> .unmap_dma_buf = vb2_dma_sg_dmabuf_ops_unmap,
> + .begin_cpu_access = vb2_dma_sg_dmabuf_ops_begin_cpu_access,
> + .end_cpu_access = vb2_dma_sg_dmabuf_ops_end_cpu_access,
> .vmap = vb2_dma_sg_dmabuf_ops_vmap,
> .mmap = vb2_dma_sg_dmabuf_ops_mmap,
> .release = vb2_dma_sg_dmabuf_ops_release,
>

2020-02-19 08:43:12

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 11/12] videobuf2: add begin/end cpu_access callbacks to dma-sg

On (20/02/19 09:35), Hans Verkuil wrote:
> On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> > Provide begin_cpu_access() and end_cpu_access() dma_buf_ops
> > callbacks for cache synchronisation on exported buffers.
> >
> > V4L2_FLAG_MEMORY_NON_CONSISTENT has no effect on dma-sg buffers.
> > dma-sg allocates memory using the page allocator directly, so
> > there is no memory consistency guarantee.
>
> This should also be a comment in the code.

OK.

-ss

2020-02-19 08:45:44

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 09/12] videobuf2: let user-space know if driver supports cache hints

On (20/02/19 09:33), Hans Verkuil wrote:
> On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> > Add V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS to fill_buf_caps(), which
> > is set when queue supports user-space cache management hints.
>
> Ah, you add the capability here :-)
>
> This should be moved forward in the series. Actually, I think this should
> be merged with the first patch of the series.

OK, can squash. This way I don't have to split 03/12.

I can also update V4L2_BUF_FLAG_NO_CACHE_INVALIDATE/CLEAN in 01/12 then.
Would that work?

-ss

2020-02-19 08:48:02

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 02/12] videobuf2: handle V4L2 buffer cache flags

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Set video buffer cache management flags corresponding to V4L2 cache
> flags.
>
> Both ->prepare() and ->finish() cache management hints should be
> passed during this stage (buffer preparation), because there is no
> other way for user-space to skip ->finish() cache flush.
>
> There are two possible alternative approaches:
> - The first one is to move cache sync from ->finish() to dqbuf().
> But this breaks some drivers, that need to fix-up buffers before
> dequeueing them.
>
> - The second one is to move ->finish() call from ->done() to dqbuf.
>
> Change-Id: I3bef1d1fb93a5fba290ce760eaeecdc8e7d6885a
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> .../media/common/videobuf2/videobuf2-v4l2.c | 36 +++++++++++++++++++
> 1 file changed, 36 insertions(+)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index eb5d5db96552..2da06a2ad6c4 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -337,6 +337,41 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
> return 0;
> }
>
> +static void set_buffer_cache_hints(struct vb2_queue *q,
> + struct vb2_buffer *vb,
> + struct v4l2_buffer *b)
> +{
> + /*
> + * DMA exporter should take care of cache syncs, so we can avoid
> + * explicit ->prepare()/->finish() syncs. For other ->memory types
> + * we always need ->prepare() or/and ->finish() cache sync.
> + */
> + if (q->memory == VB2_MEMORY_DMABUF) {
> + vb->need_cache_sync_on_finish = 0;
> + vb->need_cache_sync_on_prepare = 0;
> + return;
> + }
> +
> + if (!q->allow_cache_hints)

If q->allow_cache_hints is 0, then if userspace attempts to set these
flags in v4l2_buffer, they should be cleared. That's to indicate to
userspace that these flags won't work.

That should be done in vb2_fill_vb2_v4l2_buffer().

Regards,

Hans

> + return;
> +
> + vb->need_cache_sync_on_prepare = 1;
> + /*
> + * ->finish() cache sync can be avoided when queue direction is
> + * TO_DEVICE.
> + */
> + if (q->dma_dir != DMA_TO_DEVICE)
> + vb->need_cache_sync_on_finish = 1;
> + else
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
> + vb->need_cache_sync_on_finish = 0;
> +
> + if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
> + vb->need_cache_sync_on_prepare = 0;
> +}
> +
> static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b, bool is_prepare,
> struct media_request **p_req)
> @@ -381,6 +416,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
> }
>
> if (!vb->prepared) {
> + set_buffer_cache_hints(q, vb, b);
> /* Copy relevant information provided by the userspace */
> memset(vbuf->planes, 0,
> sizeof(vbuf->planes[0]) * vb->num_planes);
>

2020-02-19 08:49:30

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> This patch lets user-space to request a non-consistent memory
> allocation during CREATE_BUFS and REQBUFS ioctl calls.
>
> = CREATE_BUFS
>
> struct v4l2_create_buffers has seven 4-byte reserved areas,
> so reserved[0] is renamed to ->flags. The struct, thus, now
> has six reserved 4-byte regions.
>
> = REQBUFS
>
> We use one bit of a ->reserved[1] member of struct v4l2_requestbuffers,
> which is now renamed to ->flags. Unlike v4l2_create_buffers, struct
> v4l2_requestbuffers does not have enough reserved room. Therefore for
> backward compatibility ->reserved and ->flags were put into anonymous
> union.
>
> Change-Id: I0eaab3428de499ce1bce6fc6b26c5ca5ff405882
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> .../media/uapi/v4l/vidioc-create-bufs.rst | 9 +++++++-
> .../media/uapi/v4l/vidioc-reqbufs.rst | 15 ++++++++++---
> .../media/common/videobuf2/videobuf2-v4l2.c | 21 ++++++++++++++++---
> drivers/media/v4l2-core/v4l2-ioctl.c | 5 +----
> include/uapi/linux/videodev2.h | 8 +++++--
> 5 files changed, 45 insertions(+), 13 deletions(-)
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> index bd08e4f77ae4..68185e94b686 100644
> --- a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> @@ -121,7 +121,14 @@ than the number requested.
> other changes, then set ``count`` to 0, ``memory`` to
> ``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type.
> * - __u32
> - - ``reserved``\ [7]
> + - ``flags``
> + - Specifies additional buffer management attributes. Valid only when
> + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
> + Old drivers and applications must set it to zero.
> +
> +
> + * - __u32
> + - ``reserved``\ [6]
> - A place holder for future extensions. Drivers and applications
> must set the array to zero.
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> index d0c643db477a..9741dac0d5b3 100644
> --- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
> @@ -112,10 +112,19 @@ aborting or finishing any DMA in progress, an implicit
> ``V4L2_MEMORY_MMAP`` and ``type`` set to the buffer type. This will
> free any previously allocated buffers, so this is typically something
> that will be done at the start of the application.
> - * - __u32
> + * - union
> + - (anonymous)
> + * -
> + - __u32
> + - ``flags``\ [1]
> + - Specifies additional buffer management attributes. Valid only when
> + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
> + Old drivers and applications must set it to zero.
> +
> + * -
> + - __u32
> - ``reserved``\ [1]
> - - A place holder for future extensions. Drivers and applications
> - must set the array to zero.
> + - Kept for backwards compatibility. Use ``flags`` instead.
>
> .. tabularcolumns:: |p{6.1cm}|p{2.2cm}|p{8.7cm}|
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 7cdfcd1baf82..eb5d1306cb03 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -707,9 +707,15 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
> int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
> {
> int ret = vb2_verify_memory_type(q, req->memory, req->type);
> + bool consistent = true;
> +
> + if (req->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;

There is no check against allow_cache_hints: if that's 0, then
the V4L2_FLAG_MEMORY_NON_CONSISTENT flag should be cleared since it is
not supported.

>
> fill_buf_caps(q, &req->capabilities);
> - return ret ? ret : vb2_core_reqbufs(q, req->memory, true, &req->count);
> + if (ret)
> + return ret;
> + return vb2_core_reqbufs(q, req->memory, consistent, &req->count);
> }
> EXPORT_SYMBOL_GPL(vb2_reqbufs);
>
> @@ -738,6 +744,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> unsigned requested_sizes[VIDEO_MAX_PLANES];
> struct v4l2_format *f = &create->format;
> int ret = vb2_verify_memory_type(q, create->memory, f->type);
> + bool consistent = true;
> unsigned i;
>
> fill_buf_caps(q, &create->capabilities);
> @@ -783,7 +790,11 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> for (i = 0; i < requested_planes; i++)
> if (requested_sizes[i] == 0)
> return -EINVAL;
> - return ret ? ret : vb2_core_create_bufs(q, create->memory, true,
> +
> + if (create->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;

Ditto.

Regards,

Hans

> +
> + return ret ? ret : vb2_core_create_bufs(q, create->memory, consistent,
> &create->count, requested_planes, requested_sizes);
> }
> EXPORT_SYMBOL_GPL(vb2_create_bufs);
> @@ -953,13 +964,17 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv,
> {
> struct video_device *vdev = video_devdata(file);
> int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type);
> + bool consistent = true;
>
> fill_buf_caps(vdev->queue, &p->capabilities);
> if (res)
> return res;
> if (vb2_queue_is_busy(vdev, file))
> return -EBUSY;
> - res = vb2_core_reqbufs(vdev->queue, p->memory, true, &p->count);
> + if (p->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> + consistent = false;
> +
> + res = vb2_core_reqbufs(vdev->queue, p->memory, consistent, &p->count);
> /* If count == 0, then the owner has released all buffers and he
> is no longer owner of the queue. Otherwise we have a new owner. */
> if (res == 0)
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index aaf83e254272..9791e2882382 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1973,9 +1973,6 @@ static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops,
>
> if (ret)
> return ret;
> -
> - CLEAR_AFTER_FIELD(p, capabilities);
> -
> return ops->vidioc_reqbufs(file, fh, p);
> }
>
> @@ -2015,7 +2012,7 @@ static int v4l_create_bufs(const struct v4l2_ioctl_ops *ops,
> if (ret)
> return ret;
>
> - CLEAR_AFTER_FIELD(create, capabilities);
> + CLEAR_AFTER_FIELD(create, flags);
>
> v4l_sanitize_format(&create->format);
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 72efc1c544cd..169a8cf345ed 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -938,7 +938,10 @@ struct v4l2_requestbuffers {
> __u32 type; /* enum v4l2_buf_type */
> __u32 memory; /* enum v4l2_memory */
> __u32 capabilities;
> - __u32 reserved[1];
> + union {
> + __u32 flags;
> + __u32 reserved[1];
> + };
> };
>
> /* capabilities for struct v4l2_requestbuffers and v4l2_create_buffers */
> @@ -2445,7 +2448,8 @@ struct v4l2_create_buffers {
> __u32 memory;
> struct v4l2_format format;
> __u32 capabilities;
> - __u32 reserved[7];
> + __u32 flags;
> + __u32 reserved[6];
> };
>
> /*
>

2020-02-19 08:55:29

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 00/12] Implement V4L2_BUF_FLAG_NO_CACHE_* flags

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> Hello,
>
> V2 of the patchset, reshuffled, added more documentation,
> addressed some of the feedback ;) Still in RFC, tho
>
> v1 link: https://lore.kernel.org/lkml/[email protected]/
>
> ---
> This is a reworked version of the vb2 cache hints
> (V4L2_BUF_FLAG_NO_CACHE_INVALIDATE / V4L2_BUF_FLAG_NO_CACHE_CLEAN)
> support patch series which previsouly was developed by Sakari and
> Laurent [0].
>
> The patch set attempts to preserve the existing behvaiour - cache
> sync is performed in ->prepare() and ->finish() (unless the buffer
> is DMA exported). User space can request “default behavior” override
> with cache management hints, which are handled on a per-buffer basis
> and should be supplied with v4l2_buffer ->flags during buffer
> preparation. There are two possible hints:
>
> - V4L2_BUF_FLAG_NO_CACHE_INVALIDATE
> No cache sync on ->finish()
>
> - V4L2_BUF_FLAG_NO_CACHE_CLEAN
> No cache sync on ->prepare()
>
> In order to keep things on the safe side, we also require driver
> to explicitly state which of its queues (if any) support user space
> cache management hints (such queues should have ->allow_cache_hints
> bit set).
>
> The patch set also (to some extent) simplifies allocators' ->prepare()
> and ->finish() callbacks. Namely, we move cache management decision
> making to the upper - core - layer. For example, if, previously, we
> would have something like this
>
> vb2_buffer_done()
> vb2_dc_finish()
> if (buf->db_attach)
> return;
>
> where each allocators' ->finish() callback would either bail
> out (DMA exported buffer, for instance) or sync, now that "bail
> out or sync" decision is made before we call into the allocator.
>
> Along with cache management hints, user space is also able to
> adjust queue's memory consistency attributes. Memory consistency
> attribute (dma_attrs) is per-queue, yet it plays its role on the
> allocator level, when we allocate buffers’ private memory (planes).
> For the time being, only one consistency attribute is supported:
> DMA_ATTR_NON_CONSISTENT.

This is starting to look pretty good. I think you can drop the RFC
for the next time you post this.

One thing I would like to see as well is test code in v4l2-compliance,
specifically for testing the various flags and capabilities. I.e.,
if V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS, then it should test that it can
set the cache flags and the NON_CONSISTENT flag. If it is not set,
then those flags should be cleared when attempting to set them.

Also code in v4l2-ctl and common/v4l2-info.cpp to support the new
flags, both reporting them, but also setting them.

Regards,

Hans

>
> [0] https://www.mail-archive.com/[email protected]/msg112459.html
>
> Sergey Senozhatsky (12):
> videobuf2: add cache management members
> videobuf2: handle V4L2 buffer cache flags
> videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag
> videobuf2: add queue memory consistency parameter
> videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag
> videobuf2: factor out planes prepare/finish functions
> videobuf2: do not sync caches when we are allowed not to
> videobuf2: check ->synced flag in prepare() and finish()
> videobuf2: let user-space know if driver supports cache hints
> videobuf2: add begin/end cpu_access callbacks to dma-contig
> videobuf2: add begin/end cpu_access callbacks to dma-sg
> videobuf2: don't test db_attach in dma-contig prepare and finish
>
> Documentation/media/uapi/v4l/buffer.rst | 27 +++++
> .../media/uapi/v4l/vidioc-create-bufs.rst | 9 +-
> .../media/uapi/v4l/vidioc-reqbufs.rst | 22 +++-
> .../media/common/videobuf2/videobuf2-core.c | 110 +++++++++++++-----
> .../common/videobuf2/videobuf2-dma-contig.c | 39 ++++++-
> .../media/common/videobuf2/videobuf2-dma-sg.c | 30 +++--
> .../media/common/videobuf2/videobuf2-v4l2.c | 59 +++++++++-
> drivers/media/dvb-core/dvb_vb2.c | 2 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 5 +-
> include/media/videobuf2-core.h | 17 ++-
> include/uapi/linux/videodev2.h | 11 +-
> 11 files changed, 273 insertions(+), 58 deletions(-)
>

2020-02-19 08:56:36

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 03/12] videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
> By setting or clearing V4L2_FLAG_MEMORY_NON_CONSISTENT flag
> user-space should be able to set or clear queue's NON_CONSISTENT
> ->dma_attrs. Queue's ->dma_attrs are passed to the underlying
> allocator in __vb2_buf_mem_alloc(), so thus user-space is able
> to request vb2 buffer's memory to be either consistent (coherent)
> or non-consistent.
>
> Change-Id: Ib333081c482e23c9a89386078293e19c3fd59076
> Signed-off-by: Sergey Senozhatsky <[email protected]>
> ---
> Documentation/media/uapi/v4l/buffer.rst | 27 +++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 2 ++
> 2 files changed, 29 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index 9149b57728e5..af007daf0591 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -705,6 +705,33 @@ Buffer Flags
>
> .. c:type:: v4l2_memory
>
> +Memory Consistency Flags
> +========================
> +
> +.. tabularcolumns:: |p{7.0cm}|p{2.2cm}|p{8.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table::
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 3 1 4
> +
> + * .. _`V4L2_FLAG_MEMORY_NON_CONSISTENT`:
> +
> + - ``V4L2_FLAG_MEMORY_NON_CONSISTENT``
> + - 0x00000001
> + - vb2 buffer is allocated either in consistent (it will be automatically
> + coherent between CPU and bus) or non-consistent memory. The latter
> + can provide performance gains, for instance CPU cache sync/flush
> + operations can be avoided if the buffer is accesed by the corresponding
> + device only and CPU does not read/write to/from that buffer. However,
> + this requires extra care from the driver -- it must guarantee memory
> + consistency by issuing cache flush/sync when consistency is needed.
> + If this flag is set V4L2 will attempt to allocate vb2 buffer in
> + non-consistent memory. This flag is ignored if queue does not report
> + :ret:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.

This flag only makes sense for the MMAP memory model, right? That should be
documented and checked in the code.

An attempt to use this flag with the wrong memory model should just clear it,
I think (something to test as well in v4l2-compliance).

Regards,

Hans

> +
> enum v4l2_memory
> ================
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 5f9357dcb060..72efc1c544cd 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -189,6 +189,8 @@ enum v4l2_memory {
> V4L2_MEMORY_DMABUF = 4,
> };
>
> +#define V4L2_FLAG_MEMORY_NON_CONSISTENT (1 << 0)
> +
> /* see also http://vektor.theorem.ca/graphics/ycbcr/ */
> enum v4l2_colorspace {
> /*
>

2020-02-19 08:57:08

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 09/12] videobuf2: let user-space know if driver supports cache hints

On 2/19/20 9:45 AM, Sergey Senozhatsky wrote:
> On (20/02/19 09:33), Hans Verkuil wrote:
>> On 2/4/20 3:56 AM, Sergey Senozhatsky wrote:
>>> Add V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS to fill_buf_caps(), which
>>> is set when queue supports user-space cache management hints.
>>
>> Ah, you add the capability here :-)
>>
>> This should be moved forward in the series. Actually, I think this should
>> be merged with the first patch of the series.
>
> OK, can squash. This way I don't have to split 03/12.
>
> I can also update V4L2_BUF_FLAG_NO_CACHE_INVALIDATE/CLEAN in 01/12 then.
> Would that work?

Yes, that makes sense.

Hans

>
> -ss
>

2020-02-19 09:00:10

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On (20/02/19 09:25), Hans Verkuil wrote:
[..]
> > diff --git a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> > index bd08e4f77ae4..68185e94b686 100644
> > --- a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> > +++ b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
> > @@ -121,7 +121,14 @@ than the number requested.
> > other changes, then set ``count`` to 0, ``memory`` to
> > ``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type.
> > * - __u32
> > - - ``reserved``\ [7]
> > + - ``flags``
> > + - Specifies additional buffer management attributes. Valid only when
> > + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
> > + Old drivers and applications must set it to zero.
>
> Drop the 'Valid only' sentence. The V4L2_FLAG_MEMORY_NON_CONSISTENT depends
> on that capability, but other flags added in the future may not.

The whole sentence, right?

> Inside add a reference to the memory flags section created in patch 3.

Sorry. Inside?

[..]
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 72efc1c544cd..169a8cf345ed 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -938,7 +938,10 @@ struct v4l2_requestbuffers {
> > __u32 type; /* enum v4l2_buf_type */
> > __u32 memory; /* enum v4l2_memory */
> > __u32 capabilities;
> > - __u32 reserved[1];
> > + union {
> > + __u32 flags;
> > + __u32 reserved[1];
> > + };
>
> How about this:
>
> __u8 flags;
> __u8 reserved[3];
>
> That avoids the anonymous union and allows some space for future additions.

Hmm. This way old apps, which clear out ->reserved, e.g.
memset(&x.reserved, 0x00, sizeof(x.reserved)), won't clear
out x.flags and can accidentally submit some unintended
garbage. It's not the case with anon union.

-ss

2020-02-19 09:04:50

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/19/20 9:59 AM, Sergey Senozhatsky wrote:
> On (20/02/19 09:25), Hans Verkuil wrote:
> [..]
>>> diff --git a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
>>> index bd08e4f77ae4..68185e94b686 100644
>>> --- a/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
>>> +++ b/Documentation/media/uapi/v4l/vidioc-create-bufs.rst
>>> @@ -121,7 +121,14 @@ than the number requested.
>>> other changes, then set ``count`` to 0, ``memory`` to
>>> ``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type.
>>> * - __u32
>>> - - ``reserved``\ [7]
>>> + - ``flags``
>>> + - Specifies additional buffer management attributes. Valid only when
>>> + queue reports :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
>>> + Old drivers and applications must set it to zero.
>>
>> Drop the 'Valid only' sentence. The V4L2_FLAG_MEMORY_NON_CONSISTENT depends
>> on that capability, but other flags added in the future may not.
>
> The whole sentence, right?

Yes, "Valid only ... capability."

>
>> Inside add a reference to the memory flags section created in patch 3.
>
> Sorry. Inside?

Oops: Inside -> Instead

>
> [..]
>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index 72efc1c544cd..169a8cf345ed 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -938,7 +938,10 @@ struct v4l2_requestbuffers {
>>> __u32 type; /* enum v4l2_buf_type */
>>> __u32 memory; /* enum v4l2_memory */
>>> __u32 capabilities;
>>> - __u32 reserved[1];
>>> + union {
>>> + __u32 flags;
>>> + __u32 reserved[1];
>>> + };
>>
>> How about this:
>>
>> __u8 flags;
>> __u8 reserved[3];
>>
>> That avoids the anonymous union and allows some space for future additions.
>
> Hmm. This way old apps, which clear out ->reserved, e.g.
> memset(&x.reserved, 0x00, sizeof(x.reserved)), won't clear
> out x.flags and can accidentally submit some unintended
> garbage. It's not the case with anon union.

Hmm. I need to think about this some more, so leave in the anon union
for now.

Regards,

Hans

>
> -ss
>

2020-02-19 09:07:54

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On (20/02/19 09:48), Hans Verkuil wrote:
[..]
> > int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
> > {
> > int ret = vb2_verify_memory_type(q, req->memory, req->type);
> > + bool consistent = true;
> > +
> > + if (req->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
> > + consistent = false;
>
> There is no check against allow_cache_hints: if that's 0, then
> the V4L2_FLAG_MEMORY_NON_CONSISTENT flag should be cleared since it is
> not supported.

The check is in set_queue_consistency()

static void set_queue_consistency(struct vb2_queue *q, bool consistent_mem)
{
if (!q->allow_cache_hints)
return;

if (consistent_mem)
q->dma_attrs &= ~DMA_ATTR_NON_CONSISTENT;
else
q->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
}

I don't explicitly clear DMA_ATTR_NON_CONSISTENT attr for
!->allow_cache_hints queues just in case if the driver for
some reason sets that flag. ->allow_cache_hints is, thus,
only for cases when user-space asks us to set or clear it.

-ss

2020-02-19 09:12:05

by Hans Verkuil

Subject: Re: [RFC][PATCHv2 05/12] videobuf2: handle V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On 2/19/20 10:05 AM, Sergey Senozhatsky wrote:
> On (20/02/19 09:48), Hans Verkuil wrote:
> [..]
>>> int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
>>> {
>>> int ret = vb2_verify_memory_type(q, req->memory, req->type);
>>> + bool consistent = true;
>>> +
>>> + if (req->flags & V4L2_FLAG_MEMORY_NON_CONSISTENT)
>>> + consistent = false;
>>
>> There is no check against allow_cache_hints: if that's 0, then
>> the V4L2_FLAG_MEMORY_NON_CONSISTENT flag should be cleared since it is
>> not supported.
>
> The check is in set_queue_consistency()

That's the check against the functionality. I'm talking about the API level:
if !q->allow_cache_hints, then clear V4L2_FLAG_MEMORY_NON_CONSISTENT from
req->flags so that, when the ioctl returns to userspace, the application can
tell that that flag was rejected.

Regards,

Hans

>
> static void set_queue_consistency(struct vb2_queue *q, bool consistent_mem)
> {
> if (!q->allow_cache_hints)
> return;
>
> if (consistent_mem)
> q->dma_attrs &= ~DMA_ATTR_NON_CONSISTENT;
> else
> q->dma_attrs |= DMA_ATTR_NON_CONSISTENT;
> }
>
> I don't explicitly clear DMA_ATTR_NON_CONSISTENT attr for
> !->allow_cache_hints queues just in case if the driver for
> some reason sets that flag. ->allow_cache_hints is, thus,
> only for cases when user-space asks us to set or clear it.
>
> -ss
>

2020-02-25 07:46:50

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 03/12] videobuf2: add V4L2_FLAG_MEMORY_NON_CONSISTENT flag

On (20/02/19 09:56), Hans Verkuil wrote:
> > +Memory Consistency Flags
> > +========================
> > +
> > +.. tabularcolumns:: |p{7.0cm}|p{2.2cm}|p{8.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table::
> > + :header-rows: 0
> > + :stub-columns: 0
> > + :widths: 3 1 4
> > +
> > + * .. _`V4L2_FLAG_MEMORY_NON_CONSISTENT`:
> > +
> > + - ``V4L2_FLAG_MEMORY_NON_CONSISTENT``
> > + - 0x00000001
> > + - vb2 buffer is allocated either in consistent (it will be automatically
> > + coherent between CPU and bus) or non-consistent memory. The latter
> > + can provide performance gains, for instance CPU cache sync/flush
> > + operations can be avoided if the buffer is accesed by the corresponding
> > + device only and CPU does not read/write to/from that buffer. However,
> > + this requires extra care from the driver -- it must guarantee memory
> > + consistency by issuing cache flush/sync when consistency is needed.
> > + If this flag is set V4L2 will attempt to allocate vb2 buffer in
> > + non-consistent memory. This flag is ignored if queue does not report
> > + :ret:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.
>
> This flag only makes sense for the MMAP memory model, right? That should be
> documented and checked in the code.

Not all buffer allocators respect DMA attrs (V4L2_FLAG_MEMORY_NON_CONSISTENT
is a DMA attribute) even if we use MMAP memory model. Right? E.g. dma-cont
does, dma-sg - does not.

So the list is:

a) buffer allocated for MMAP I/O
b) buffer allocator which does not allocate pages from kernel page
allocator
c) queue that supports user space cache hints

If the driver does not set vb2_dma_contig_memops as q->mem_ops then
that queue should not have ->allow_cache_hints set. But even if it does,
the flag is ignored by the allocator.

So maybe the text can be:

+ ................... The flag takes effect only if the buffer is
+ used for :ref:`memory mapping <mmap>` I/O and the queue reports
+ :ref:`V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS` capability.

-ss

2020-02-26 11:21:32

by Sergey Senozhatsky

Subject: Re: [RFC][PATCHv2 00/12] Implement V4L2_BUF_FLAG_NO_CACHE_* flags

On (20/02/19 09:53), Hans Verkuil wrote:
[..]
> This is starting to look pretty good. I think you can drop the RFC
> for the next time you post this.

OK!

> One thing I would like to see as well is test code in v4l2-compliance,
> specifically for testing the various flags and capabilities. I.e.,
> if V4L2_BUF_CAP_SUPPORTS_CACHE_HINTS, then it should test that it can
> set the cache flags and the NON_CONSISTENT flag. If it is not set,
> then those flags should be cleared when attempting to set them.
>
> Also code in v4l2-ctl and common/v4l2-info.cpp to support the new
> flags, both reporting them, but also setting them.

I do have two trivial patches - "sync videobuf kernel header"
and "add new capability to common/v4l2-info.cpp".

But v4l2-ctl and v4l2-compliance will require much more time,
I think. So I, hopefully, will do it sometime later.

-ss