Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230606093135.GA9077@didi-ThinkCentre-M930t-N000>
 <ZIJbvCcaqjzNteWs@slm.duckdns.org> <CAEQmJ=gYe=d53HHC1xW_epmPmmddA4J28SHybwGmQzUZgxZovg@mail.gmail.com>
 <ZIdbyOBdU6zAGxVw@slm.duckdns.org>
In-Reply-To: <ZIdbyOBdU6zAGxVw@slm.duckdns.org>
From:   Yuanhan Zhang <zyhtheonly@gmail.com>
Date:   Tue, 13 Jun 2023 18:25:44 +0800
Message-ID: <CAEQmJ=geNmoOk37w=owwkpvL6-FgDfzaPhCTPNcKiFtL0pv4hg@mail.gmail.com>
Subject: Re: [PATCH] workqueue: introduce queue_work_cpumask to queue work
 onto a given cpumask
To:     Tejun Heo <tj@kernel.org>
Cc:     jiangshanlai@gmail.com, linux-kernel@vger.kernel.org,
        pmladek@suse.com, zyhtheonly@yeah.net, zwp10758@gmail.com,
        tiozhang@didiglobal.com, fuyuanli@didiglobal.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Hi Tejun,

Tejun Heo <tj@kernel.org> =E4=BA=8E2023=E5=B9=B46=E6=9C=8813=E6=97=A5=E5=91=
=A8=E4=BA=8C 01:54=E5=86=99=E9=81=93=EF=BC=9A
>
> Hello,
>
> On Fri, Jun 09, 2023 at 02:28:19PM +0800, Yuanhan Zhang wrote:
> > // I resend this to put it into the same thread, sorry for the confusio=
n.
>
> This got resent quite a few times and I don't know which one to reply to.
> Just picking the one which seems like the latest.

Thanks for your patience.

>
> > > Can you elaborate the intended use cases?
> >
> > Thanks for your reply! Please let me use myself as an example to explai=
n this.
> >
> > In my scenario, I have 7 cpus on my machine (actually it is uma, so
> > queue_work_node
> > or using UNBOUND do not works for me), and for some unlucky reasons
> > there are always some irqs running on cpu 0 and cpu 6, since I'm using =
arm64
> > with irqs tuning into FIFO threads, those threaded irqs are always runn=
ing on
> > cpu 0 and 6 too (for affinity). And this would not be fixed easily in
> > short terms :(
> >
> > So in order to help async init for better boot times for my devices,
> > I'd like to prevent
> > works from running on cpu 0 and 6. With queue_work_cpumask(), it would =
be simply
> > done by:
> >
> > ...
> > cpumask_clear_cpu(0, cpumask);  // actually I use sysfs to parse my cpu=
mask
> > cpumask_clear_cpu(6, cpumask);
> > queue_work_cpumask(cpumask, my_wq, &my_work->work);
> > ...
>
> But this would require explicit code customization on every call site whi=
ch
> doesn't seem ideal given that this is to work around something which is t=
ied
> to the specific hardware.

Yes, I agree that using wq_unbound_cpumask would be a great idea and a
substitute
solution for devices booting. But wq_unbound_cpumask could only constrain o=
n
WQ_UNBOUND while I'm trying to make each of my work configurable.
Please let me try to explain this by another example:

If I have several kinds of works, and I'd like to make them run on
different cpusets
(so it is not ideal to put them on WQ_UNBOUND).

This would be done like this:
queue_work_cpumask(cpumask_A  /*B or C or D, just maintain different cpumas=
ks*/,
system_wq, work);

And after that, I don't have to customize my codes anymore since I could
control those cpumasks by procfs or sysfs or ioctl or whatever I like.

So I believe this feature to let per work to choose its cpumask would be qu=
ite
convenient sometimes :)

>
> Wouldn't it be better to add a kernel parameter to further constrain
> wq_unbound_cpumask? Right now, on boot, it's only determined by isolcpus =
but
> it shouldn't be difficult to add a workqueue parameter to further constra=
in
> it.

Yes, thanks again, this would be a great solution for device boot. And
I have followed
your suggestion to submit another patch  '[PATCH] sched/isolation: add
a workqueue
parameter to constrain unbound CPUs'.  This patch simply let
"isolcpus=3D" to have a
"workqueue" option, which makes `housekeeping_cpumask(HK_TYPE_WQ)` to copy
a constrained workqueue cpumask to wq_unbound_cpumask.

Please help review, and thank you again for your time.

Thanks,
Tio Zhang

>
> Thanks.

>
> --
> tejun