Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230519001709.2563-1-tj@kernel.org> <20230523111818.GH4253@hirez.programming.kicks-ass.net>
 <CAKfTPtB11Zwt9bgOJrNVDfs5yY7Dws_vnQBuXZHErSsbn9Edhw@mail.gmail.com> <20230524073456.GO83892@hirez.programming.kicks-ass.net>
In-Reply-To: <20230524073456.GO83892@hirez.programming.kicks-ass.net>
From:   Vincent Guittot <vincent.guittot@linaro.org>
Date:   Wed, 24 May 2023 15:15:21 +0200
Message-ID: <CAKfTPtBdfyLudf2JpADqEbpzV8nL8UdKcVg2Q7x9zK+=hHSG+A@mail.gmail.com>
Subject: Re: [PATCHSET v1 wq/for-6.5] workqueue: Improve unbound workqueue
 execution locality
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Tejun Heo <tj@kernel.org>, jiangshanlai@gmail.com,
        torvalds@linux-foundation.org, linux-kernel@vger.kernel.org,
        kernel-team@meta.com, joshdon@google.com, brho@google.com,
        briannorris@chromium.org, nhuck@google.com, agk@redhat.com,
        snitzer@kernel.org, void@manifault.com, gautham.shenoy@amd.com
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, 24 May 2023 at 09:35, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Tue, May 23, 2023 at 06:12:45PM +0200, Vincent Guittot wrote:
>
> > Another possibility to investigate would be that each wakeup of a
> > worker is mostly unrelated to the previous one and it cares only
> > waker. so we should use -1 for the prev_cpu
>
> Tejun is actually overriding p->wake_cpu in this series to target a
> specific LLC -- with the explicit purpose to keep the workers near
> enough.

yes, so -1 for prev_cpu was a good way to forgot the irrelevant prev
cpu  without trying to abuse in order to wake it up close to the waker

>
> But the problem is that with lots of short tasks we then overload the
> LLC and are not running long enough for the idle load-balancer to spread
> things, leading to idle time.

I expect to not pile up workers in the same LLC if we keep the
workqueue cpu affinity to the die. The worker will wake up in the LLC
of the callers and callers should be spread on the die

>
> And that is specific to this lots of little LLC topologies.