Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932392AbcLHSsK (ORCPT ); Thu, 8 Dec 2016 13:48:10 -0500 Received: from mail.kernel.org ([198.145.29.136]:38210 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752727AbcLHSsI (ORCPT ); Thu, 8 Dec 2016 13:48:08 -0500 From: "Luis R. Rodriguez" To: shuah@kernel.org, jeyu@redhat.com, rusty@rustcorp.com.au, ebiederm@xmission.com, dmitry.torokhov@gmail.com, acme@redhat.com, corbet@lwn.net Cc: martin.wilck@suse.com, mmarek@suse.com, pmladek@suse.com, hare@suse.com, rwright@hpe.com, jeffm@suse.com, DSterba@suse.com, fdmanana@suse.com, neilb@suse.com, rgoldwyn@suse.com, subashab@codeaurora.org, xypron.glpk@gmx.de, keescook@chromium.org, atomlin@redhat.com, mbenes@suse.cz, paulmck@linux.vnet.ibm.com, dan.j.williams@intel.com, jpoimboe@redhat.com, davem@davemloft.net, mingo@redhat.com, akpm@linux-foundation.org, torvalds@linux-foundation.org, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, "Luis R. Rodriguez" Subject: [RFC 00/10] kmod: stress test driver, few fixes and enhancements Date: Thu, 8 Dec 2016 10:47:51 -0800 Message-Id: <20161208184801.1689-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.10.2 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3902 Lines: 71 Upon running into an old kmod v19 issue with mount (get_fs_type()) a few of us hunted for the cause of the issue. Although the issue ended up being a userspace issue, a stress test driver was written to help reproduce the issue, and along the way a few other fixes and sanity checks were implemented. I've taken the time to generalize the stress test driver as a kselftest driver with a 9 test cases. The last two test cases reveal an existing issue which is not yet addressed upstream, even if you have kmod v19 present. A fix is proposed in the last patch. Orignally we had discarded this patch as too complex due to the alias handling, but upon further analysis of test cases and memory pressure issues, it seems worth considering. Other than the last patch I don't think much of the other patches are controversial, but sending as RFC first just in case. If its not clear, an end goal here is to make module loading a bit more deterministic with stronger sanity checks and stress tests. Please note, the stress test diver requires 4 GiB of RAM to run all tests without running out of memory. A lot of this has to do with the memory requirements needed for a dynamic test for multiple threads, but note that the final memory pressure and OOMs actually don't come from this allocation, but instead from many finit_module() calls, this consumes quite a bit of memory, specially if you have a lot of dependencies which also need to be loaded prior to your needed module -- as is the case for filesystem drivers. These patches are available on my linux-next git-tree on my branch 20161208-kmod-test-driver-try2 [0], which is based on linux-next tag next-20161208. Patches are also available based on v4.9-rc8 [1] for those looking for a bit more stable tree given x86_64 on linux-next is hosed at the moment. Since kmod.c doesn't seem to get much love, and since I've been digging quite a bit into it for other users (firmware) I suppose I could volunteer myself to maintain this code as well, unless there are oppositions to this. [0] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20161208-kmod-test-driver-try2 [1] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20161208-kmod-test-driver Luis R. Rodriguez (10): kmod: add test driver to stress test the module loader module: fix memory leak on early load_module() failures kmod: add dynamic max concurrent thread count kmod: provide wrappers for kmod_concurrent inc/dec kmod: return -EBUSY if modprobe limit is reached kmod: provide sanity check on kmod_concurrent access kmod: use simplified rate limit printk sysctl: add support for unsigned int properly kmod: add helpers for getting kmod count and limit kmod: add a sanity check on module loading Documentation/admin-guide/kernel-parameters.txt | 7 + include/linux/kmod.h | 9 + include/linux/sysctl.h | 3 + init/Kconfig | 23 + init/main.c | 1 + kernel/kmod.c | 244 ++++- kernel/module.c | 12 +- kernel/sysctl.c | 198 +++- lib/Kconfig.debug | 25 + lib/Makefile | 1 + lib/test_kmod.c | 1248 +++++++++++++++++++++++ tools/testing/selftests/kmod/Makefile | 11 + tools/testing/selftests/kmod/config | 7 + tools/testing/selftests/kmod/kmod.sh | 448 ++++++++ 14 files changed, 2199 insertions(+), 38 deletions(-) create mode 100644 lib/test_kmod.c create mode 100644 tools/testing/selftests/kmod/Makefile create mode 100644 tools/testing/selftests/kmod/config create mode 100755 tools/testing/selftests/kmod/kmod.sh -- 2.10.1