Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <edf7f0c9-6e17-be83-69bb-0ec421a3a352@joelfernandes.org>
Date:   Thu, 1 Sep 2022 12:07:06 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.0
Subject: Re: [PATCH v4 00/14] Implement call_rcu_lazy() and miscellaneous
 fixes
Content-Language: en-US
To:     Frederic Weisbecker <frederic@kernel.org>,
        Uladzislau Rezki <urezki@gmail.com>
Cc:     "Paul E. McKenney" <paulmck@kernel.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Rushikesh S Kadam <rushikesh.s.kadam@intel.com>,
        Neeraj upadhyay <neeraj.iitr10@gmail.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        rcu <rcu@vger.kernel.org>,
        Vineeth Pillai <vineeth@bitbyteword.org>
References: <20220829194622.GA58291@lothringen>
 <CAEXW_YS593n8Gget+REaD-c8vT8Ht_AzOY0kXA_uc674LOyvVw@mail.gmail.com>
 <20220829204202.GQ6159@paulmck-ThinkPad-P17-Gen-1>
 <20220830105324.GA71266@lothringen>
 <20220830114343.GS6159@paulmck-ThinkPad-P17-Gen-1>
 <20220830160316.GC71266@lothringen> <20220830162244.GA73392@lothringen>
 <Yw4+g/0yEf7fpHrh@pc636> <20220901112947.GA105556@lothringen>
 <YxCejoKH8dGIeW22@pc636> <20220901151350.GC105556@lothringen>
From:   Joel Fernandes <joel@joelfernandes.org>
In-Reply-To: <20220901151350.GC105556@lothringen>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 9/1/2022 11:13 AM, Frederic Weisbecker wrote:
> On Thu, Sep 01, 2022 at 01:59:10PM +0200, Uladzislau Rezki wrote:
>> On Thu, Sep 01, 2022 at 01:29:47PM +0200, Frederic Weisbecker wrote:
>>> On Tue, Aug 30, 2022 at 06:44:51PM +0200, Uladzislau Rezki wrote:
>>>> Hello, Frederic.
>>>>
>>>>>
>>>>> Although who knows, may be some periodic file operation while idle are specific
>>>>> to Android. I'll try to trace lazy callbacks while idle and the number of grace
>>>>> periods associated.
>>>>>
>>>>>
>>>> Everything related to lazy call-backs is about not waking "nocb"
>>>> kthreads in order to offload one or i should say few callbacks
>>>> because it is more or less useless. Currently if incoming callback
>>>> is the only one, it will kick a GP whereas a GP will kick nocb_kthread
>>>> to offload.
>>>
>>> Not sure this is only about not waking "nocb" kthreads. The grace period
>>> kthread is also awaken in !NOCB and has quite some work to do. And there,
>>> having a server expands the issue because you may have a lot of CPUs's extended
>>> quiescent states to check.
>>>
>> I mean here the following combination: NOCB + call_rcu_lazy() tandem.
>> The !NOCB is not about power save, IMHO. Because it implies callbacks
>> to be processed on CPUs they are landed.
> 
> I'm sorry but I still feel confused reading that !NOCB is not about power
> save. To me everything is about power save. NOCB just appears to help optimizing
> it without significant tradeoff on some given workloads.

Right, I think Fred is coming from the reasonable perspective of "Save power by
making !NOCB RCU do less" (i.e. delay softirq, main GP thread, QS reports etc)
which I think is also valid (not sure how much power that saves on servers but
sounds reasonable to measure). When you hack your implementation to test that,
it would be reasonable to also apply the old FAST NOHZ patch and measure
with/without that.

> 
>> In this scenario you can not let the EAS scheduler to find a more
>> efficient CPU for further handling.
> 
> Sure but that doesn't mean there wouldn't be a power saving gain anyway.

Yes there might be.

Thanks,

 - Joel