On Tue, 12 Sep 2023 09:19:59 +1000 "NeilBrown" <[email protected]> wrote:
> Plain old list_heads (which the code currently uses) require a spinlock
> to be taken to insert something into the queue. As this is usually in
> bh context, it needs to be a spin_lock_bh(). My understanding is that
> the real-time developers don't much like us disabling bh. It isn't an
> enormous win switching from a list_head list to a llist_node list, but
> there are small gains such as object size reduction and less locking. I
> particularly wanted an easy-to-use library facility that could be
> plugged in to two different uses cases in the sunrpc code and there
> didn't seem to be one. I could have written one using list_head, but
> llist seemed a better fix. I think the code in sunrpc that uses this
> lwq looks a lot neater after the conversion.
Thanks. Could we please get words such as these into the changelog,
describing why it was felt necessary to add more library code?
And also into the .c file, to help people who are looking at it and
wondering "can I use this". And to help reviewers who are wondering
"could they have used Neil's thing".
lwq is a FIFO single-linked queue that only requires a spinlock
for dequeueing, which happens in process context. Enqueueing is atomic
with no spinlock and can happen in any context.
This is particularly useful when work items are queued from BH or IRQ
context, and when they are handled one at a time by dedicated threads.
Avoiding any locking when enqueueing means there is no need to disable
BH or interrupts, which is generally best avoided (particularly when
there are any RT tasks on the machine).
This solution is superior to using "list_head" links because we need
half as many pointers in the data structures, and because list_head
lists would need locking to add items to the queue.
This solution is superior to a bespoke solution as all locking and
container_of casting is integrated, so the interface is simple.
Despite the similar name, this solution meets a distinctly different
need to kfifo. kfifo provides a fixed sized circular buffer to which
data can be added at one end and removed at the other, and does not
provide any locking. lwq does not have any size limit and works with
data structures (objects?) rather than data (bytes).
A unit test for basic functionality, which runs at boot time, is included.
Signed-off-by: NeilBrown <[email protected]>
---
include/linux/lwq.h | 4 ++++
lib/lwq.c | 4 ++++
2 files changed, 8 insertions(+)
diff --git a/include/linux/lwq.h b/include/linux/lwq.h
index 52b9c81b493a..c4148fe1cf72 100644
--- a/include/linux/lwq.h
+++ b/include/linux/lwq.h
@@ -7,6 +7,10 @@
*
* Entries can be enqueued from any context with no locking.
* Entries can be dequeued from process context with integrated locking.
+ *
+ * This is particularly suitable when work items are queued in
+ * BH or IRQ context, and where work items are handled one at a time
+ * by dedicated threads.
*/
#include <linux/container_of.h>
#include <linux/spinlock.h>
diff --git a/lib/lwq.c b/lib/lwq.c
index 7fe6c7125357..eb8324225309 100644
--- a/lib/lwq.c
+++ b/lib/lwq.c
@@ -8,6 +8,10 @@
* Entries are dequeued using a spinlock to protect against
* multiple access. The llist is staged in reverse order, and refreshed
* from the llist when it exhausts.
+ *
+ * This is particularly suitable when work items are queued in
+ * BH or IRQ context, and where work items are handled one at a time
+ * by dedicated threads.
*/
#include <linux/rcupdate.h>
#include <linux/lwq.h>
--
2.42.0