Received-SPF: pass (google.com: domain of linux-kernel+bounces-56988-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Sender: Tejun Heo <htejun@gmail.com>
Date: Wed, 7 Feb 2024 09:48:37 -1000
From: Tejun Heo <tj@kernel.org>
To: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Linux PM <linux-pm@vger.kernel.org>,
	Ulf Hansson <ulf.hansson@linaro.org>,
	Nathan Chancellor <nathan@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>,
	Lai Jiangshan <jiangshanlai@gmail.com>, Naohiro.Aota@wdc.com,
	kernel-team@meta.com
Subject: Re: [PATCH v1] PM: sleep: Restore asynchronous device resume
 optimization
Message-ID: <ZcPelerpp3Rr5YFW@slm.duckdns.org>
References: <CGME20240207103144eucas1p16b601a73ff347d2542f8380b25921491@eucas1p1.samsung.com>
 <10423008.nUPlyArG6x@kreacher>
 <708a65cc-79ec-44a6-8454-a93d0f3114c3@samsung.com>
 <CAJZ5v0hn=KgaWn9pwtLsH2a8n61BNxzb1xrNoxUfEi3o9OAZGw@mail.gmail.com>
 <4a043533-009f-4db9-b107-c8374be28d2b@samsung.com>
 <CAJZ5v0hDmwaFEtLc8yDc4cXn2wODXAqATe0+_Hpm9QPODUPMQw@mail.gmail.com>
 <ZcOyW_Q1FC35oxob@slm.duckdns.org>
 <2f125955-8c7c-465c-938c-8768f7ca360b@samsung.com>
 <ZcPSuUBoL_EDvcTF@slm.duckdns.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZcPSuUBoL_EDvcTF@slm.duckdns.org>

Hello,

I couldn't reproduce effective max_active being pushed down to min_active
across suspend/resume cycles on x86. There gotta be something different.

- Can you please apply the following patch along with the WQ_DFL_MIN_ACTIVE
  bump, go through suspend/resume once and report the dmesg?

- Regardless of the root cause, I think async should switch to a dedicated
  workqueue with explicitly raised min_active (will add an interface for
  it).

Thanks.

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index cf514ba0dfc3..93e5b837f2b4 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -368,6 +368,8 @@ struct workqueue_struct {
 	 */
 	struct rcu_head		rcu;
 
+	bool			dbg;
+
 	/* hot fields used during command issue, aligned to cacheline */
 	unsigned int		flags ____cacheline_aligned; /* WQ: WQ_* flags */
 	struct pool_workqueue __percpu __rcu **cpu_pwq; /* I: per-cpu pwqs */
@@ -1557,6 +1559,10 @@ static void wq_update_node_max_active(struct workqueue_struct *wq, int off_cpu)
 	if (off_cpu >= 0)
 		total_cpus--;
 
+	if (wq->dbg)
+		printk("XXX wq_update_node_max_active: wq=%s off_cpu=%d total=%d range=[%d, %d]",
+		       wq->name, off_cpu, total_cpus, min_active, max_active);
+
 	for_each_node(node) {
 		int node_cpus;
 
@@ -1567,7 +1573,12 @@ static void wq_update_node_max_active(struct workqueue_struct *wq, int off_cpu)
 		wq_node_nr_active(wq, node)->max =
 			clamp(DIV_ROUND_UP(max_active * node_cpus, total_cpus),
 			      min_active, max_active);
+		if (wq->dbg)
+			pr_cont(" node[%d] node_cpus=%d max=%d",
+				node, node_cpus, wq_node_nr_active(wq, node)->max);
 	}
+	if (wq->dbg)
+		pr_cont("\n");
 
 	wq_node_nr_active(wq, NUMA_NO_NODE)->max = min_active;
 }
@@ -4886,6 +4897,9 @@ static struct pool_workqueue *install_unbound_pwq(struct workqueue_struct *wq,
 
 	old_pwq = rcu_access_pointer(*slot);
 	rcu_assign_pointer(*slot, pwq);
+	if (wq->dbg)
+		printk("XXX install_unbound_pwq: wq=%s cpu=%d pool=%d\n",
+		       wq->name, cpu, pwq->pool->id);
 	return old_pwq;
 }
 
@@ -7418,6 +7432,8 @@ void __init workqueue_init_early(void)
 	       !system_power_efficient_wq ||
 	       !system_freezable_power_efficient_wq ||
 	       !system_bh_wq || !system_bh_highpri_wq);
+
+	system_unbound_wq->dbg = true;
 }
 
 static void __init wq_cpu_intensive_thresh_init(void)