Received-SPF: pass (google.com: domain of linux-kernel+bounces-36222-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Sender: Tejun Heo <htejun@gmail.com>
Date: Tue, 23 Jan 2024 14:30:23 -1000
From: Tejun Heo <tj@kernel.org>
To: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: linux-kernel@vger.kernel.org, Naohiro.Aota@wdc.com,
	kernel-team@meta.com
Subject: Re: [PATCH 9/9] workqueue: Implement system-wide nr_active
 enforcement for unbound workqueues
Message-ID: <ZbBaH2uSqLmJmSsV@slm.duckdns.org>
References: <20240113002911.406791-1-tj@kernel.org>
 <20240113002911.406791-10-tj@kernel.org>
 <CAJhGHyDaNCoXaK4g4fKg3vfBYuQ7r+e8TT8ObrtT3nBcTeeuMg@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJhGHyDaNCoXaK4g4fKg3vfBYuQ7r+e8TT8ObrtT3nBcTeeuMg@mail.gmail.com>

Hello,

On Fri, Jan 19, 2024 at 03:54:24PM +0800, Lai Jiangshan wrote:
> node_nr_cpus[node] and num_online_cpus() are global values, they might
> not suitable
> for this particular workqueue and might cause skewed proportions.

Good point.

> the cache values:
> 
> pwq->pool->attrs->pool_nr_online_cpus =
> cpumask_weight_and(pwq->pool->attrs->__pod_cpumask, cpu_online_mask);
> 
> wq->wq_nr_online_cpus =
> cpumask_weight_and(wq->unbound_attrs->cpumask,  cpu_online_mask);
> 
> can be used instead. They can be calculated at creation and cpuhotplug.
> pool_nr_online_cpus doesn't contribute to the pool's hash value.
> 
> Or the result of wq_node_max_active() can be cached in struct wq_node_nr_active,
> see the comment next.
..
> > +       node_max_active = wq_node_max_active(wq, pool->node);
> 
> It is a hot path for unbound workqueues, I think the result of
> wq_node_max_active()
> should be cached in struct wq_node_nr_active.

I'm skeptical this would make a meaningful difference but if we're
calculating the numbers per-workqueue anyways, maybe this won't add too much
complexity.

> >  /**
> >   * pwq_activate_first_inactive - Activate the first inactive work item on a pwq
> >   * @pwq: pool_workqueue of interest
> > + * @fill: max_active may have increased, try to increase concurrency level
> 
> I think it is also legitimate to increase the concurrency level ASAP
> when called from try_to_grab_pending() path.

Can you elaborate why that'd be useful?

Thanks.

-- 
tejun