Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Uladzislau Rezki <urezki@gmail.com>
Date:   Wed, 30 Mar 2022 21:35:26 +0200
To:     "Paul E. McKenney" <paulmck@kernel.org>
Cc:     "Zhang, Qiang1" <qiang1.zhang@intel.com>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        "frederic@kernel.org" <frederic@kernel.org>,
        "urezki@gmail.com" <urezki@gmail.com>,
        "quic_neeraju@quicinc.com" <quic_neeraju@quicinc.com>,
        "josh@joshtriplett.org" <josh@joshtriplett.org>,
        "juri.lelli@redhat.com" <juri.lelli@redhat.com>,
        "rcu@vger.kernel.org" <rcu@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3] rcu: Only boost rcu reader tasks with lower priority
 than boost kthreads
Message-ID: <YkSw/oc496GftWld@pc638.lan>
References: <20220311022226.595905-1-qiang1.zhang@intel.com>
 <Yist4IWWR/6BZzeK@linutronix.de>
 <PH0PR11MB58802B8804EDFFC73BA676F2DA0D9@PH0PR11MB5880.namprd11.prod.outlook.com>
 <PH0PR11MB58807AD9A1BAA122218B92DBDA0D9@PH0PR11MB5880.namprd11.prod.outlook.com>
 <20220316165931.GI4285@paulmck-ThinkPad-P17-Gen-1>
 <PH0PR11MB58806B59FC8148C9C5160A5FDA139@PH0PR11MB5880.namprd11.prod.outlook.com>
 <20220318145738.GY4285@paulmck-ThinkPad-P17-Gen-1>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220318145738.GY4285@paulmck-ThinkPad-P17-Gen-1>
Precedence: bulk

> On Fri, Mar 18, 2022 at 05:50:35AM +0000, Zhang, Qiang1 wrote:
> > On Sat, Mar 12, 2022 at 03:11:04AM +0000, Zhang, Qiang1 wrote:
> > > On 2022-03-11 10:22:26 [+0800], Zqiang wrote:
> > > > When RCU_BOOST is enabled, the boost kthreads will boosting readers
> > > > who are blocking a given grace period, if the current reader tasks
> > >                                        ^ Period.
> > > 
> > > > have a higher priority than boost kthreads(the boost kthreads priority
> > > > not always 1, if the kthread_prio is set), 
> > > 
> > > >>This confuses me:
> > > >>- Why does this matter
> > > 
> > > In preempt-rt system, if the kthread_prio is not set, it prio is 1.
> > > the boost kthreads can preempt almost  rt task, It will affect
> > > the real-time performance of some user rt  tasks.  In preempt-rt systems,
> > > in most scenarios, this kthread_prio will be configured.
> > >
> > >Just following up...  These questions might have been answered, but
> > >I am not seeing those answers right off-hand.
> > >
> > >Is the grace-period latency effect of choosing not to boost high-priority
> > >tasks visible at the system level in any actual workload?
> > >
> > >Suppose that a SCHED_DEADLINE task has exhausted its time quantum,
> > >and has thus been preempted within an RCU read-side critical section.
> > >Can priority boosting from a SCHED_FIFO prio-1 task cause it to start
> > >running?
> > >
> > >Do delays in RCU priority boosting cause excessive grace-period
> > >latencies on real workloads, even when all the to-be-boosted
> > >tasks are SCHED_OTHER?
> > >
> > >Thoughts?
> > 
> > I have tested this modification these days,  I originally planned to generate a Kconfig option to control
> > whether to skip tasks with higher priority than boost kthreads. but it doesn't seem necessary
> >  because I find it's optimization is not particularly
> > obvious in the actual scene, I find that tasks with higher priority than boost kthreads 
> > will quickly exit the rcu critical area , even if be preempted in the rcu critical area.
> > sorry for the noise.
> 
> Thank you for getting back with this information, and no need to
> apologize.  We all get excited about a potential change from time to time.
> Part of us maintainers' jobs is to ask hard questions when that appears
> to be happening.  ;-)
> 
> If you have continued interest in this area, it would be good to keep
> looking.  After all, neither RCU expedited grace periods nor RCU priority
> boosting were designed with these new use cases in mind, so it is quite
> likely that there is a useful change to be made in there somewhere.
> 
> You see, RCU expedited grace periods were designed for throughput rather
> than latency.  The original use case was an old networking API that
> needed to wait for a grace period on each and every one of a series of
> some tens of thousands of system calls.  If one or two of those system
> calls took a few hundred milliseconds, but the rest completed in less than
> a millisecond, no harm done.  (Yes, there are now newer APIs that allow
> many changes to be made with only the one grace-period wait.  But the
> kernel must continue to support the old API: Never Break Userspace.)
> 
> For its part, RCU priority boosting was originally designed for
> debuggging.  The point was to avoid OOMing the system when someone
> misconfigured their application's real-time priorities.  As you know,
> such misconfiguration can easily prevent low-priority RCU readers from
> ever completing.
> 
> So it is reasonably likely that some change or another is needed.  After
> all, new use cases require new functionality and new fixes.  The trick
> is figuring out which change makes sense amongst the huge group of other
> possible changes that each add much more complexity than improvement.
> But part of the process of finding that change that makes sense is trying
> out quite a few changes that don't help all that much.  ;-)
> 
Sorry for the late response, but i think i should comment on it since i
have tried to simulate and test this patch on Android device. Basically
we do have RT tasks in Android and i do not see that the patch that is
in question makes any difference. Actually i was not able to trigger its
functionality at all.

From the other hand, i have tried to simulate it making an RT environment
with SCHED_FIFO tasks and some synchronize_rcu_expedited() users. Indeed
i can trigger it but it is very specific env. and number of triggering or
tasks bypassing(high prio) is almost zero.

--
Uladzislau Rezki