Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 6 Oct 2022 11:55:43 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     "Joel Fernandes (Google)" <joel@joelfernandes.org>
Cc:     rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
        rushikesh.s.kadam@intel.com, urezki@gmail.com,
        neeraj.iitr10@gmail.com, frederic@kernel.org, rostedt@goodmis.org,
        youssefesmat@google.com, surenb@google.com
Subject: Re: [PATCH v7 00/11] rcu: call_rcu() power improvements
Message-ID: <20221006185543.GB4196@paulmck-ThinkPad-P17-Gen-1>
Reply-To: paulmck@kernel.org
References: <20221004024157.2470238-1-joel@joelfernandes.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20221004024157.2470238-1-joel@joelfernandes.org>
Precedence: bulk

On Tue, Oct 04, 2022 at 02:41:46AM +0000, Joel Fernandes (Google) wrote:
> v7 version of RCU lazy patches based on rcu/next branch.
> 
> To facilitate easier merge, I dropped tracing and other patches and just
> implemented the new changes. I will post the tracing patches later along with
> rcutop as I need to add new tracepoints that Frederic suggested.
> 
> Main recent changes:
> 1. rcu_barrier() wake up only for lazy bypass list.
> 2. Make all call_rcu() default-lazy and add call_rcu_flush() API.
> 3. Take care of some callers using call_rcu_flush() API.
> 4. Several refactorings suggested by Paul/Frederic.
> 5. New call_rcu() to call_rcu_flush() conversions by Joel/Vlad/Paul.
> 
> I am seeing good performance and power with these patches on real ChromeOS x86
> asymmetric hardware.
> 
> Earlier cover letter with lots of details is here:
> https://lore.kernel.org/all/20220901221720.1105021-1-joel@joelfernandes.org/
> 
> List of recent changes:
>     
>     [ Frederic Weisbec: Program the lazy timer only if WAKE_NOT, since other
>       deferral levels wake much earlier so for those it is not needed. ]
>     
>     [ Frederic Weisbec: Use flush flags to keep bypass API code clean. ]
>     
>     [ Frederic Weisbec: Make rcu_barrier() wake up only if main list empty. ]
>     
>     [ Frederic Weisbec: Remove extra 'else if' branch in rcu_nocb_try_bypass(). ]
>     
>     [ Joel: Fix issue where I was not resetting lazy_len after moving it to rdp ]
>     
>     [ Paul/Thomas/Joel: Make call_rcu() default lazy so users don't mess up. ]
>     
>     [ Paul/Frederic : Cosmetic changes, split out wakeup of nocb thread. ]
>     
>     [ Vlad/Joel : More call_rcu -> flush conversions ]

Thank you for your continued work on this!

I pulled these into an experimental branch, applied Uladzislau's
Tested-by and ran a quick round of rcutorture.

From TREE02, TREE03, and TREE09 I got this:

In file included from kernel/rcu/tree.c:68:
kernel/rcu/tree.h:449:13: error: ‘wake_nocb_gp’ used but never defined [-Werror]
  449 | static bool wake_nocb_gp(struct rcu_data *rdp, bool force);
      |             ^~~~~~~~~~~~

One could argue that this is not a big deal, except that Linus gets a
bit tetchy when this sort of thing shows up in mainline.

Nevetheless, it looks like we might be getting quite close!

							Thanx, Paul

> Frederic Weisbecker (1):
> rcu: Wake up nocb gp thread on rcu_barrier_entrain()
> 
> Joel Fernandes (Google) (7):
> rcu: Make call_rcu() lazy to save power
> rcu: Refactor code a bit in rcu_nocb_do_flush_bypass()
> rcuscale: Add laziness and kfree tests
> percpu-refcount: Use call_rcu_flush() for atomic switch
> rcu/sync: Use call_rcu_flush() instead of call_rcu
> rcu/rcuscale: Use call_rcu_flush() for async reader test
> rcu/rcutorture: Use call_rcu_flush() where needed
> 
> Uladzislau Rezki (2):
> scsi/scsi_error: Use call_rcu_flush() instead of call_rcu()
> workqueue: Make queue_rcu_work() use call_rcu_flush()
> 
> Vineeth Pillai (1):
> rcu: shrinker for lazy rcu
> 
> drivers/scsi/scsi_error.c |   2 +-
> include/linux/rcupdate.h  |   7 ++
> kernel/rcu/Kconfig        |   8 ++
> kernel/rcu/rcu.h          |   8 ++
> kernel/rcu/rcuscale.c     |  67 +++++++++++-
> kernel/rcu/rcutorture.c   |  16 +--
> kernel/rcu/sync.c         |   2 +-
> kernel/rcu/tiny.c         |   2 +-
> kernel/rcu/tree.c         | 140 +++++++++++++++++--------
> kernel/rcu/tree.h         |  12 ++-
> kernel/rcu/tree_exp.h     |   2 +-
> kernel/rcu/tree_nocb.h    | 213 ++++++++++++++++++++++++++++++++------
> kernel/workqueue.c        |   2 +-
> lib/percpu-refcount.c     |   3 +-
> 14 files changed, 388 insertions(+), 96 deletions(-)
> 
> --
> 2.38.0.rc1.362.ged0d419d3c-goog
>