Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e3010471ee43e7e134f882f320fc4643fe4e4810.camel@kernel.org>
Subject: Re: [RFC PATCH 4/4] cpuset: Support RCU-NOCB toggle on v2 root
 partitions
From:   nicolas saenz julienne <nsaenz@kernel.org>
To:     Frederic Weisbecker <frederic@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>
Cc:     Juri Lelli <juri.lelli@redhat.com>, Tejun Heo <tj@kernel.org>,
        Waiman Long <longman@redhat.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "Paul E . McKenney" <paulmck@kernel.org>,
        Paul Gortmaker <paul.gortmaker@windriver.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Marcelo Tosatti <mtosatti@redhat.com>,
        Phil Auld <pauld@redhat.com>,
        Zefan Li <lizefan.x@bytedance.com>,
        Daniel Bristot de Oliveira <bristot@kernel.org>,
        rcu@vger.kernel.org
Date:   Mon, 30 May 2022 16:29:56 +0200
In-Reply-To: <20220530004049.GA1251147@lothringen>
References: <20220525221055.1152307-1-frederic@kernel.org>
         <20220525221055.1152307-5-frederic@kernel.org>
         <Yo/FGcG+uiBh88sT@slm.duckdns.org> <20220526225141.GA1214445@lothringen>
         <YpAHEt0j30vBw9au@slm.duckdns.org>
         <9e44bb00-955a-dbc6-a863-be649e0c701f@redhat.com>
         <YpAdSW8JXVPOoNJl@slm.duckdns.org>
         <20220527083018.n43nc73vuuzm5ixo@localhost.localdomain>
         <YpIwsiaY2IPK96WO@hirez.programming.kicks-ass.net>
         <20220530004049.GA1251147@lothringen>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.1 (3.44.1-1.fc36) 
MIME-Version: 1.0
Precedence: bulk

On Mon, 2022-05-30 at 02:40 +0200, Frederic Weisbecker wrote:
> On Sat, May 28, 2022 at 04:24:50PM +0200, Peter Zijlstra wrote:
> > On Fri, May 27, 2022 at 10:30:18AM +0200, Juri Lelli wrote:
> > > Hi,
> > >=20
> > > On 26/05/22 14:37, Tejun Heo wrote:
> > > > On Thu, May 26, 2022 at 08:28:43PM -0400, Waiman Long wrote:
> > > > > I am thinking along the line that it will not be hierarchical. Ho=
wever,
> > > > > cpuset can be useful if we want to have multiple isolated partiti=
ons
> > > > > underneath the top cpuset with different isolation attributes, bu=
t no more
> > > > > sub-isolated partition with sub-attributes underneath them. IOW, =
we can only
> > > > > set them at the first level under top_cpuset. Will that be useful=
?
> > > >=20
> > > > At that point, I'd just prefer to have it under /proc or /sys.
> > >=20
> > > FWIW, I was under the impression that this would nicely fit along the
> > > side of other feaures towards implenting dynamic isolation of CPUs (s=
ay
> > > https://lore.kernel.org/lkml/20220510153413.400020-1-longman@redhat.c=
om/
> > > for example). Wouldn't be awkward to have to poke different places to
> > > achieve isolation at runtime?
> >=20
> > This, that's what I was thinking.
> >=20
> > My main objection to the whole thing is that it's an RCU_NOCB specific
> > interface. *That* I think is daft.
> >=20
> > I was thinking a partition would be able to designate a house-keeping
> > sub-partition/mask, but who cares about all the various different
> > housekeeping parties.
>=20
> It's time for the isolation users to step up here! I very rarely hear fro=
m them
> and I just can't figure out by myself all the variants of uses for each o=
f the
> isolation features. May be some people are only interested in nocb for so=
me
> specific uses, or may be it never makes sense without nohz full and all t=
he rest
> of the isolation features. So for now I take the very cautious path to sp=
lit the
> interface.

OK, my 2 cents. I personally deal with virtualisation setups that involve R=
T
and CPU isolation on both host and guests.

The main use-case ATM is running DPDK-like workloads. We want to achieve
latencies in the order of tens of microseconds, so it's essential to avoid
entering the kernel at all cost. So, no HW interrupts, sched tick, RCU
callbacks, clocksource watchdogs, softlockup, intel_pstate, timers, etc...
Everything is deferred onto housekeeping CPUs or disabled.

Then we have setups that need to deal with HW on the host, exposed to the g=
uest
through emulation or VirtIO. The same rules apply really, except for some I=
RQ
affinity tweaks and sched priority magic.

I find it hard to see how running RCU callback locally could be useful to a=
ny
latency sensitive workload.

Frederic, out of curiosity, do you have a use-case in mind that might benef=
it
from nohz_full but not rcu_nocb? Maybe HPC?

Regards,
Nicolas