Date:   Fri, 3 Feb 2023 15:10:28 +0100
From:   Petr Mladek <pmladek@suse.com>
To:     Tejun Heo <tj@kernel.org>
Cc:     Lai Jiangshan <jiangshanlai@gmail.com>,
        Michal Koutny <mkoutny@suse.com>, linux-kernel@vger.kernel.org
Subject: Re: [RFC 2/5] workqueue: Warn when a new worker could not be created
Message-ID: <Y90V1E9KP785ALMs@alley>
References: <20230201134543.13687-1-pmladek@suse.com>
 <20230201134543.13687-3-pmladek@suse.com>
 <Y9xHnwG39IHN/BBu@slm.duckdns.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y9xHnwG39IHN/BBu@slm.duckdns.org>
Precedence: bulk

On Thu 2023-02-02 13:30:39, Tejun Heo wrote:
> Hello,
> 
> On Wed, Feb 01, 2023 at 02:45:40PM +0100, Petr Mladek wrote:
> > +static __printf(2, 3) __cold
> > +void __print_create_worker_failure(long err, const char *fmt, ...)
> > +{
> > +	spin_lock_irq(&create_worker_failed_lock);
> > +
> > +	/*
> > +	 * Report potentially repeated failures only once during a stall.
> > +	 * Otherwise, it might be noisy. Also slow serial console drivers
> > +	 * touch watchdogs so that more frequent messages would prevent
> > +	 * reaching the watchdog thresh.
> > +	 */
> > +	if (!create_worker_failed) {
> > +		va_list args;
> > +
> > +		va_start(args, fmt);
> > +		vprintk(fmt, args);
> > +		va_end(args);
> > +	}
> > +
> > +	create_worker_failed++;
> > +
> > +	spin_unlock_irq(&create_worker_failed_lock);
> > +}
> 
> That's pretty elaborate.

Yeah, I am not super happy with it either.

> Why not just use printk_ratelimited()?

The default printk_ratelimited() is not usable because it uses:

     #define DEFAULT_RATELIMIT_INTERVAL	(5 * HZ)
     #define DEFAULT_RATELIMIT_BURST		10

It allows 10 messages per 5 seconds. It would be still too noisy.
maybe_create_worker() tries to create a new worker every second.

And more importantly, it would break both softlockup and workqueue
watchdogs. See touch_nmi_watchdog() in serial8250_console_write().
By other words, it would break both softlockup and workqueue watchdogs.


A solution would be to use a custom printk_ratelimited_wq() that would
allow printing one message per 2 * wq_watchdog_thresh.
Something like:

#define printk_ratelimited_wq(fmt, ...)					\
({									\
	static DEFINE_RATELIMIT_STATE(_rs, 60 * HZ, 1);			\
									\
	if (__ratelimit(&_rs))						\
		printk(fmt, ##__VA_ARGS__);				\
})

I admit that it is much easier than __print_create_worker_failure().

The only problem might be that wq_watchdog_thresh might be modified
at runtime. But it can be solved by sharing the same
struct ratelimit_state rs in all printk_ratelimited_wq() calls
and updating it together with wq_watchdog_thresh.

Would you prefer the custom printk_ratelimited_wq(), please?

Best Regards,
Petr