Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Sender: Tejun Heo <htejun@gmail.com>
Date:   Thu, 11 May 2023 13:03:00 -1000
From:   Tejun Heo <tj@kernel.org>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     jiangshanlai@gmail.com, torvalds@linux-foundation.org,
        linux-kernel@vger.kernel.org, kernel-team@meta.com
Subject: Re: [PATCH 7/7] workqueue: Track and monitor per-workqueue CPU time
 usage
Message-ID: <ZF10JNbflTfojZtA@slm.duckdns.org>
References: <20230511181931.869812-1-tj@kernel.org>
 <20230511181931.869812-8-tj@kernel.org>
 <20230511211134.GG2296992@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230511211134.GG2296992@hirez.programming.kicks-ass.net>
Precedence: bulk

Hello,

On Thu, May 11, 2023 at 11:11:34PM +0200, Peter Zijlstra wrote:
> On Thu, May 11, 2023 at 08:19:31AM -1000, Tejun Heo wrote:
> > Now that wq_worker_tick() is there, we can easily track the rough CPU time
> > consumption of each workqueue by charging the whole tick whenever a tick
> > hits an active workqueue. While not super accurate, it provides reasonable
> > visibility into the workqueues that consume a lot of CPU cycles.
> > wq_monitor.py is updated to report the per-workqueue CPU times.
> 
> I'm utterly failing to read that dragon thing (or possibly snake, but I
> can typically sorta make out what it intends to do).

Yeah, it's basically gdb in python. It's surprisingly widely useful.

> However, while you don't have preempt-out, you still have sched-out
> through wq_worker_sleeping(), so you should be able to compute the time
> spend on the workqueue by past worker runs -- fully accurate.
> 
> Then you only need to add the time since sched-in of any current worker
> and you have a complete picture of time spend on the workqueue, no
> approximation needed.
> 
> Or am I completely missing the point?

One issue is that sum_exec_runtime isn't necessarily updated when a work
item completes and then the work can go execute other work items without any
scheduling event inbetween. So, no matter what I do, if I wanna account CPU
time accurately per work item, I need to read the current time once more
between work items, which is okay but not really necessary, not preemptively
anyway. If there are use cases that need accurate accounting, we can always
revisit the issue.

Thanks.

-- 
tejun