The following changes since commit aac8a59537dfc704ff344f1aacfd143c089ee20f:
Revert "workqueue: Override implicit ordered attribute in workqueue_apply_unbound_cpumask()" (2024-02-05 15:49:06 -1000)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git/ tags/wq-for-6.9
for you to fetch changes up to 1acd92d95fa24edca8f0292b21870025da93e24f:
workqueue: Drain BH work items on hot-unplugged CPUs (2024-02-29 11:51:24 -1000)
----------------------------------------------------------------
workqueue: Changes for v6.9
This cycle, a lot of workqueue changes including some that are significant
and invasive.
- During v6.6 cycle, unbound workqueues were updated so that they are more
topology aware and flexible, which among other things improved workqueue
behavior on modern multi-L3 CPUs. In the process, 636b927eba5b
("workqueue: Make unbound workqueues to use per-cpu pool_workqueues")
switched unbound workqueues to use per-CPU frontend pool_workqueues as a
part of increasing front-back mapping flexibility.
An unwelcome side effect of this change was that this made max concurrency
enforcement per-CPU blowing up the maximum number of allowed concurrent
executions. I incorrectly assumed that this wouldn't cause practical
problems as most unbound workqueue users are self-regulate max
concurrency; however, there definitely are which don't (e.g. on IO paths)
and the drastic increase in the allowed max concurrency led to noticeable
perf regressions in some use cases.
This is now addressed by separating out max concurrency enforcement to a
separate struct - wq_node_nr_active - which makes @max_active consistently
mean system-wide max concurrency regardless of the number of CPUs or
(finally) NUMA nodes. This is a rather invasive and, in places, a bit
clunky; however, the clunkiness rises from the the inherent requirement to
handle the disagreement between the execution locality domain and max
concurrency enforcement domain on some modern machines. See 5797b1c18919
("workqueue: Implement system-wide nr_active enforcement for unbound
workqueues") for more details.
- BH workqueue support is added. They are similar to per-CPU workqueues but
execute work items in the softirq context. This is expected to replace
tasklet. However, currently, it's missing the ability to disable and
enable work items which is needed to convert many tasklet users. To avoid
crowding this merge window too much, this will be included in the next
merge window. A separate pull request will be sent for the couple
conversion patches that are currently pending.
- Waiman plugged a long-standing hole in workqueue CPU isolation where
ordered workqueues didn't follow wq_unbound_cpumask updates. Ordered
workqueues now follow the same rules as other unbound workqueues.
- More CPU isolation improvements: Juri fixed another deficit in workqueue
isolation where unbound rescuers don't respect wq_unbound_cpumask.
Leonardo fixed delayed_work timers firing on isolated CPUs.
- Other misc changes.
----------------------------------------------------------------
Allen Pais (1):
workqueue: Introduce from_work() helper for cleaner callback declarations
Audra Mitchell (2):
workqueue.c: Increase workqueue name length
workqueue: Shorten events_freezable_power_efficient name
Juri Lelli (3):
tools/workqueue: Add rescuers printing to wq_dump.py
kernel/workqueue: Bind rescuer to unbound cpumask for WQ_UNBOUND
kernel/workqueue: Let rescuers follow unbound wq cpumask changes
Leonardo Bras (1):
workqueue: Avoid using isolated cpus' timers on queue_delayed_work
Marcelo Tosatti (1):
workqueue: mark power efficient workqueue as unbounded if nohz_full enabled
Miguel Ojeda (1):
workqueue: rust: sync with `WORK_CPU_UNBOUND` change
Ricardo B. Marliere (1):
workqueue: make wq_subsys const
Tejun Heo (37):
tools/workqueue/wq_dump.py: Clean up code and drop duplicate information
workqueue: Drop unnecessary kick_pool() in create_worker()
workqueue: Break up enum definitions and give names to the types
workqueue: Move pwq->max_active to wq->max_active
workqueue: Factor out pwq_is_empty()
workqueue: Replace pwq_activate_inactive_work() with [__]pwq_activate_work()
workqueue: Move nr_active handling into helpers
workqueue: Make wq_adjust_max_active() round-robin pwqs while activating
workqueue: RCU protect wq->dfl_pwq and implement accessors for it
workqueue: Move pwq_dec_nr_in_flight() to the end of work item handling
workqueue: Introduce struct wq_node_nr_active
workqueue: Implement system-wide nr_active enforcement for unbound workqueues
tools/workqueue/wq_dump.py: Add node_nr/max_active dump
workqueue: Don't call cpumask_test_cpu() with -1 CPU in wq_update_node_max_active()
workqueue: Avoid premature init of wq->node_nr_active[].max
workqueue: Fix pwq->nr_in_flight corruption in try_to_grab_pending()
workqueue: Update lock debugging code
workqueue: Factor out init_cpu_worker_pool()
workqueue: Implement BH workqueues to eventually replace tasklets
Revert "workqueue: make wq_subsys const"
workqueue: Don't implicitly make UNBOUND workqueues w/ @max_active==1 ordered
Merge branch 'for-6.8-fixes' into for-6.9
workqueue: Implement workqueue_set_min_active()
async: Use a dedicated unbound workqueue with raised min_active
workqueue: Fix queue_work_on() with BH workqueues
workqueue, irq_work: Build fix for !CONFIG_IRQ_WORK
workqueue: Cosmetic changes
workqueue: Use rcu_read_lock_any_held() instead of rcu_read_lock_held()
workqueue: Rename __cancel_work_timer() to __cancel_timer_sync()
workqueue: Reorganize flush and cancel[_sync] functions
workqueue: Use variable name irq_flags for saving local irq flags
workqueue: Introduce work_cancel_flags
workqueue: Clean up enum work_bits and related constants
workqueue: Factor out work_grab_pending() from __cancel_work_sync()
workqueue: Remove clear_work_data()
workqueue: Make @flags handling consistent across set_work_data() and friends
workqueue: Drain BH work items on hot-unplugged CPUs
Waiman Long (5):
workqueue: Skip __WQ_DESTROYING workqueues when updating global unbound cpumask
workqueue: Link pwq's into wq->pwqs from oldest to newest
workqueue: Enable unbound cpumask update on ordered workqueues
workqueue: Bind unbound workqueue rescuer to wq_unbound_cpumask
workqueue: Fix kernel-doc comment of unplug_oldest_pwq()
Wang Jinchao (1):
workqueue: fix a typo in comment
Xuewen Yan (2):
workqueue: Add rcu lock check at the end of work item execution
workqueue: Control intensive warning threshold through cmdline
Documentation/admin-guide/kernel-parameters.txt | 9 +
Documentation/core-api/workqueue.rst | 43 +-
include/linux/async.h | 1 +
include/linux/workqueue.h | 141 +-
init/Kconfig | 2 +-
init/main.c | 1 +
kernel/async.c | 17 +-
kernel/softirq.c | 5 +
kernel/workqueue.c | 1891 +++++++++++++++++------
rust/kernel/workqueue.rs | 6 +-
tools/workqueue/wq_dump.py | 104 +-
11 files changed, 1690 insertions(+), 530 deletions(-)
--
tejun
The pull request you sent on Sun, 10 Mar 2024 23:07:27 -1000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git/ tags/wq-for-6.9
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/ff887eb07cf69a5c0a507a1311fb34bcd38450aa
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html