Date: Wed, 14 Dec 2016 16:38:27 +0100
From: Petr Mladek <pmladek@suse.com>
To: "Luis R. Rodriguez" <mcgrof@kernel.org>
Cc: shuah@kernel.org, jeyu@redhat.com, rusty@rustcorp.com.au,
        ebiederm@xmission.com, dmitry.torokhov@gmail.com, acme@redhat.com,
        corbet@lwn.net, martin.wilck@suse.com, mmarek@suse.com, hare@suse.com,
        rwright@hpe.com, jeffm@suse.com, DSterba@suse.com, fdmanana@suse.com,
        neilb@suse.com, linux@roeck-us.net, rgoldwyn@suse.com,
        subashab@codeaurora.org, xypron.glpk@gmx.de, keescook@chromium.org,
        atomlin@redhat.com, mbenes@suse.cz, paulmck@linux.vnet.ibm.com,
        dan.j.williams@intel.com, jpoimboe@redhat.com, davem@davemloft.net,
        mingo@redhat.com, akpm@linux-foundation.org,
        torvalds@linux-foundation.org, linux-kselftest@vger.kernel.org,
        linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC 03/10] kmod: add dynamic max concurrent thread count
Message-ID: <20161214153827.GH16064@pathway.suse.cz>
References: <20161208184801.1689-1-mcgrof@kernel.org>
 <20161208194814.2485-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161208194814.2485-1-mcgrof@kernel.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3886
Lines: 105

On Thu 2016-12-08 11:48:14, Luis R. Rodriguez wrote:
> We currently statically limit the number of modprobe threads which
> we allow to run concurrently to 50. As per Keith Owens, this was a
> completely arbitrary value, and it was set in the 2.3.38 days [0]
> over 16 years ago in year 2000.
> 
> Although we haven't yet hit our lower limits, experimentation [1]
> shows that when and if we hit this limit in the worst case, will be
> fatal -- consider get_fs_type() failures upon mount on a system which
> has many partitions, some of which might even be with the same
> filesystem. Its best to be prudent and increase and set this
> value to something more sensible which ensures we're far from hitting
> the limit and also allows default build/user run time override.
> 
> The worst case is fatal given that once a module fails to load there
> is a period of time during which subsequent request for the same module
> will fail, so in the case of partitions its not just one request that
> could fail, but whole series of partitions. This later issue of a
> module request failure domino effect can be addressed later, but
> increasing the limit to something more meaninful should at least give us
> enough cushion to avoid this for a while.
> 
> Set this value up with a bit more meaninful modern limits:
> 
> Bump this up to 64  max for small systems (CONFIG_BASE_SMALL)
> Bump this up to 128 max for larger systems (!CONFIG_BASE_SMALL)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index 271692a352f1..da2c25746937 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -2111,6 +2111,29 @@ config TRIM_UNUSED_KSYMS
>  
>  	  If unsure, or if you need to build out-of-tree modules, say N.
>  
> +config MAX_KMOD_CONCURRENT
> +	int "Max allowed concurrent request_module() calls (6=>64, 10=>1024)"
> +	range 0 14

Would not too small range break loading module dependencies?
I am not sure how it is implemented but it might require having
some more module loads in progress.

I would give 6 as minimum. Nobody has troubles with the current limit.

> +	default 6 if !BASE_SMALL
> +	default 7 if BASE_SMALL

Aren't the conditions inversed?

> diff --git a/init/main.c b/init/main.c
> index 8161208d4ece..1fa441aa32c6 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -638,6 +638,7 @@ asmlinkage __visible void __init start_kernel(void)
>  	thread_stack_cache_init();
>  	cred_init();
>  	fork_init();
> +	init_kmod_umh();
>  	proc_caches_init();
>  	buffer_init();
>  	key_init();
> diff --git a/kernel/kmod.c b/kernel/kmod.c
> index 0277d1216f80..cb6f7ca7b8a5 100644
> --- a/kernel/kmod.c
> +++ b/kernel/kmod.c
> @@ -186,6 +174,31 @@ int __request_module(bool wait, const char *fmt, ...)
>  	return ret;
>  }
>  EXPORT_SYMBOL(__request_module);
> +
> +/*
> + * If modprobe needs a service that is in a module, we get a recursive
> + * loop.  Limit the number of running kmod threads to max_threads/2 or
> + * CONFIG_MAX_KMOD_CONCURRENT, whichever is the smaller.  A cleaner method
> + * would be to run the parents of this process, counting how many times
> + * kmod was invoked.  That would mean accessing the internals of the
> + * process tables to get the command line, proc_pid_cmdline is static
> + * and it is not worth changing the proc code just to handle this case.
> + *
> + * "trace the ppid" is simple, but will fail if someone's
> + * parent exits.  I think this is as good as it gets.
> + *
> + * You can override with with a kernel parameter, for instance to allow
> + * 4096 concurrent modprobe instances:
> + *
> + *	kmod.max_modprobes=4096
> + */
> +void __init init_kmod_umh(void)
> +{
> +	if (!max_modprobes)
> +		max_modprobes = min(max_threads/2,
> +				    2 << CONFIG_MAX_KMOD_CONCURRENT);

This should be

	1 << CONFIG_MAX_KMOD_CONCURRENT);

1 << 1 = 2;

Note that this calculation is mentioned also some comments and
documentation.

Best Regards,
Petr