Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Sender: Tejun Heo <htejun@gmail.com>
Date:   Mon, 3 Jul 2023 11:47:31 -1000
From:   Tejun Heo <tj@kernel.org>
To:     Pin-yen Lin <treapking@chromium.org>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Brian Norris <briannorris@chromium.org>,
        jiangshanlai@gmail.com, peterz@infradead.org,
        linux-kernel@vger.kernel.org, kernel-team@meta.com,
        joshdon@google.com, brho@google.com, nhuck@google.com,
        agk@redhat.com, snitzer@kernel.org, void@manifault.com
Subject: Re: [PATCHSET v1 wq/for-6.5] workqueue: Improve unbound workqueue
 execution locality
Message-ID: <ZKNB88I41b9DLw1t@slm.duckdns.org>
References: <20230519001709.2563-1-tj@kernel.org>
 <ZIewlkGJJJUXPFL0@google.com>
 <ZIfY5zhhHU9IgOqx@slm.duckdns.org>
 <CAEXTbpconFReyAKxerNCQW5e51CMG=_RU9JHAquyEfSH1aqajA@mail.gmail.com>
 <ZJNMk9oSp1_IYXLU@slm.duckdns.org>
 <CAHk-=wgXoyxy99HnEFcvf+eUZAS5=ekWt_y84LC3P+0osxh6Jw@mail.gmail.com>
 <CAEXTbpcDS4SV=h41QKE_HkBhsmSa6Pe=63JtQZW8h2Fpc8Vo0w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAEXTbpcDS4SV=h41QKE_HkBhsmSa6Pe=63JtQZW8h2Fpc8Vo0w@mail.gmail.com>
Precedence: bulk

Hello, Pin-yen.

On Thu, Jun 29, 2023 at 05:49:27PM +0800, Pin-yen Lin wrote:
> > > I find that perplexing given that switching to a per-cpu workqueue remedies
> > > the situation quite a bit, which is how this patchset came to be. #3 is the
> > > same as per-cpu workqueue, so if you're seeing noticeably different
> > > performance numbers between #3 and per-cpu workqueue, there's something
> > > wrong with either the code or test setup.
> >
> In our case, per-cpu workqueue (removing WQ_UNBOUND) doesn't bring us
> better results. But given that pinning tasks to a single CPU core
> helps, we thought that the regression is related to the behavior of
> WQ_UNBOUND. Our findings are listed in [1].

I see.

> We already use WQ_SYSFS and the sysfs interface to pin the tasks, but
> thanks for the suggestion.

Yeah, I have no idea why there's such a drastic regression and the only way
to alleviate that is pinning the execution to a single CPU, which is also
different from other reports. It seems plausible that there are some
scheduling behavior changes and that's interacting negatively with power
saving but I'm sure that's the first thing you guys looked into.

From workqueue's POV, I'm afraid using WQ_SYSFS and pinning it as needed
seems like about what can be done for your specific case.

Thanks.

-- 
tejun