2023-04-14 17:25:28

by Luis Chamberlain

[permalink] [raw]
Subject: [PATCH v4 0/2] modules/kmod: replace implementation with a sempahore

Changes on this v4:

o Really add Matthew Wilcox' preferred tribal knowledge docs
o Add all the pending tags

Changes on v3:

o Tons of spell checks thanks to Miroslav Benes
o Fixed a stupid bug where I was using the timeout without HZ as
reported by Miroslav Benes
o Enanced the tribal knowledge docs for the semaphore Vs mutex
considerations folks might make as suggested by Matthew Wilcox
o Added tags for patches

Changes on v2:
o split the series up into its own
o adopt Peter's patch and extend it with some documentation as to why
some folks stick to binary semaphores over mutexes
o modify kmod.c to use the preferred declaration

Luis Chamberlain (1):
modules/kmod: replace implementation with a semaphore

Peter Zijlstra (1):
Change DEFINE_SEMAPHORE() to take a number argument

arch/mips/cavium-octeon/setup.c | 2 +-
arch/x86/kernel/cpu/intel.c | 2 +-
drivers/firmware/efi/runtime-wrappers.c | 2 +-
drivers/firmware/efi/vars.c | 2 +-
drivers/macintosh/adb.c | 2 +-
.../net/ethernet/broadcom/bnx2x/bnx2x_main.c | 2 +-
drivers/platform/x86/intel/ifs/sysfs.c | 2 +-
drivers/scsi/esas2r/esas2r_ioctl.c | 2 +-
.../interface/vchiq_arm/vchiq_arm.c | 2 +-
include/linux/semaphore.h | 10 +++++--
kernel/module/kmod.c | 26 +++++--------------
kernel/printk/printk.c | 2 +-
net/rxrpc/call_object.c | 6 ++---
13 files changed, 27 insertions(+), 35 deletions(-)

--
2.39.2


2023-04-14 17:25:29

by Luis Chamberlain

[permalink] [raw]
Subject: [PATCH v4 2/2] modules/kmod: replace implementation with a semaphore

Simplify the concurrency delimiter we use for kmod with the semaphore.
I had used the kmod strategy to try to implement a similar concurrency
delimiter for the kernel_read*() calls from the finit_module() path
so to reduce vmalloc() memory pressure. That effort didn't provide yet
conclusive results, but one thing that became clear is we can use
the suggested alternative solution with semaphores which Linus hinted
at instead of using the atomic / wait strategy.

I've stress tested this with kmod test 0008:

time /data/linux-next/tools/testing/selftests/kmod/kmod.sh -t 0008

And I get only a *slight* delay. That delay however is small, a few
seconds for a full test loop run that runs 150 times, for about ~30-40
seconds. The small delay is worth the simplfication IMHO.

Reviewed-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Miroslav Benes <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>
---
kernel/module/kmod.c | 26 +++++++-------------------
1 file changed, 7 insertions(+), 19 deletions(-)

diff --git a/kernel/module/kmod.c b/kernel/module/kmod.c
index b717134ebe17..5899083436a3 100644
--- a/kernel/module/kmod.c
+++ b/kernel/module/kmod.c
@@ -40,8 +40,7 @@
* effect. Systems like these are very unlikely if modules are enabled.
*/
#define MAX_KMOD_CONCURRENT 50
-static atomic_t kmod_concurrent_max = ATOMIC_INIT(MAX_KMOD_CONCURRENT);
-static DECLARE_WAIT_QUEUE_HEAD(kmod_wq);
+static DEFINE_SEMAPHORE(kmod_concurrent_max, MAX_KMOD_CONCURRENT);

/*
* This is a restriction on having *all* MAX_KMOD_CONCURRENT threads
@@ -148,29 +147,18 @@ int __request_module(bool wait, const char *fmt, ...)
if (ret)
return ret;

- if (atomic_dec_if_positive(&kmod_concurrent_max) < 0) {
- pr_warn_ratelimited("request_module: kmod_concurrent_max (%u) close to 0 (max_modprobes: %u), for module %s, throttling...",
- atomic_read(&kmod_concurrent_max),
- MAX_KMOD_CONCURRENT, module_name);
- ret = wait_event_killable_timeout(kmod_wq,
- atomic_dec_if_positive(&kmod_concurrent_max) >= 0,
- MAX_KMOD_ALL_BUSY_TIMEOUT * HZ);
- if (!ret) {
- pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now",
- module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT);
- return -ETIME;
- } else if (ret == -ERESTARTSYS) {
- pr_warn_ratelimited("request_module: sigkill sent for modprobe %s, giving up", module_name);
- return ret;
- }
+ ret = down_timeout(&kmod_concurrent_max, MAX_KMOD_ALL_BUSY_TIMEOUT * HZ);
+ if (ret) {
+ pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now",
+ module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT);
+ return ret;
}

trace_module_request(module_name, wait, _RET_IP_);

ret = call_modprobe(module_name, wait ? UMH_WAIT_PROC : UMH_WAIT_EXEC);

- atomic_inc(&kmod_concurrent_max);
- wake_up(&kmod_wq);
+ up(&kmod_concurrent_max);

return ret;
}
--
2.39.2