Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230519001709.2563-1-tj@kernel.org> <ZIewlkGJJJUXPFL0@google.com>
 <ZIfY5zhhHU9IgOqx@slm.duckdns.org>
In-Reply-To: <ZIfY5zhhHU9IgOqx@slm.duckdns.org>
From:   Pin-yen Lin <treapking@chromium.org>
Date:   Tue, 13 Jun 2023 17:26:48 +0800
Message-ID: <CAEXTbpconFReyAKxerNCQW5e51CMG=_RU9JHAquyEfSH1aqajA@mail.gmail.com>
Subject: Re: [PATCHSET v1 wq/for-6.5] workqueue: Improve unbound workqueue
 execution locality
To:     Tejun Heo <tj@kernel.org>
Cc:     Brian Norris <briannorris@chromium.org>, jiangshanlai@gmail.com,
        torvalds@linux-foundation.org, peterz@infradead.org,
        linux-kernel@vger.kernel.org, kernel-team@meta.com,
        joshdon@google.com, brho@google.com, nhuck@google.com,
        agk@redhat.com, snitzer@kernel.org, void@manifault.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Hi Tejun,

On Tue, Jun 13, 2023 at 10:48=E2=80=AFAM Tejun Heo <tj@kernel.org> wrote:
>
> Hello,
>
> On Mon, Jun 12, 2023 at 04:56:06PM -0700, Brian Norris wrote:
> > Thanks for the CC; my colleague tried out your patches (ported to 5.15
> > with some minor difficulty), and aside from some crashes (already noted
> > by others, although we didn't pull the proposed v2 fixes), he didn't
>
> Yeah, there were a few subtle bugs that v2 fixes.
>
> > notice a significant change in performance on our particular test syste=
m
> > and WiFi-throughput workload. I don't think we expected a lot though,
> > per the discussion at:
> >
> > https://lore.kernel.org/all/ZFvpJb9Dh0FCkLQA@google.com/
>
> That's disappointing. I was actually expecting that the default behavior
> would restrain migrations across L3 boundaries strong enough to make a
> meaningful difference. Can you enable WQ_SYSFS and test the following
> configs?
>
>  1. affinity_scope =3D cache, affinity_strict =3D 1
>
>  2. affinity_scope =3D cpu, affinity_strict =3D 0
>
>  3. affinity_scope =3D cpu, affinity_strict =3D 1

I pulled down v2 series and tried these settings on our 5.15 kernel.
Unfortunately none of them showed significant improvement on the
throughput. It's hard to tell which one is the best because of the
noise, but the throughput is still all far from our 4.19 kernel or
simply pinning everything to a single core.

All the 4 settings (3 settings listed above plus the default) yields
results between 90 to 120 Mbps, while pinning tasks to a single core
consistently reaches >250 Mbps.
>
> #3 basically turns it into a percpu workqueue, so it should perform more =
or
> less the same as a percpu workqueue without affecting everyone else.
>
> Any chance you can post the toplogy details on the affected setup? How ar=
e
> the caches and cores laid out?

The core layout is listed at [1], and I'm not familiar with its cache
configuration either.

[1]: https://lore.kernel.org/all/ZFvpJb9Dh0FCkLQA@google.com/

Best regards,
Pin-yen
>
> Thanks.
>
> --
> tejun