Date: Thu, 2 Jul 2015 12:22:00 -0700
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>, josh@joshtriplett.org,
        linux-kernel@vger.kernel.org, laijs@cn.fujitsu.com,
        dipankar@in.ibm.com, akpm@linux-foundation.org,
        mathieu.desnoyers@efficios.com, tglx@linutronix.de,
        rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
        dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com,
        bobby.prani@gmail.com
Subject: Re: [PATCH RFC tip/core/rcu 0/5] Expedited grace periods encouraging
 normal ones
Message-ID: <20150702192200.GO3717@linux.vnet.ibm.com>
Reply-To: paulmck@linux.vnet.ibm.com
References: <20150701100939.GR19282@twins.programming.kicks-ass.net>
 <20150701105511.GN18673@twins.programming.kicks-ass.net>
 <20150701140031.GB3717@linux.vnet.ibm.com>
 <20150701141710.GG25159@twins.programming.kicks-ass.net>
 <20150701161705.GK3717@linux.vnet.ibm.com>
 <20150701170242.GL3644@twins.programming.kicks-ass.net>
 <20150701200936.GP3717@linux.vnet.ibm.com>
 <20150702074719.GA27230@gmail.com>
 <20150702135834.GF3717@linux.vnet.ibm.com>
 <20150702183557.GA15331@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150702183557.GA15331@gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4103
Lines: 102

On Thu, Jul 02, 2015 at 08:35:57PM +0200, Ingo Molnar wrote:
> 
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> 
> > > And it's not like it's that hard to stem the flow of algorithmic sloppiness at 
> > > the source, right?
> > 
> > OK, first let me make sure that I understand what you are asking for:
> > 
> > 1.	Completely eliminate synchronize_rcu_expedited() and
> > 	synchronize_sched_expedited(), replacing all uses with their
> > 	unexpedited counterparts.  (Note that synchronize_srcu_expedited()
> > 	does not wake up CPUs, courtesy of its read-side memory barriers.)
> > 	The fast-boot guys are probably going to complain, along with
> > 	the networking guys.
> > 
> > 2.	Keep synchronize_rcu_expedited() and synchronize_sched_expedited(),
> > 	but push back hard on any new uses and question any existing uses.
> > 
> > 3.	Revert 74b51ee152b6 ("ACPI / osl: speedup grace period in
> > 	acpi_os_map_cleanup").
> > 
> > 4.	Something else?
> 
> I'd love to have 1) but 2) would be a realistic second best option? ;-)

OK, how about the following checkpatch.pl patch?

And here are some other actions I have taken and will take to improve
the situation, both for OS jitter and for scalability:

o	Reduce OS jitter by switching from try_stop_cpus() to
	stop_one_cpu_nowait(), courtesy of Peter Zijlstra.

	I expect to send this in v4.3 or v4.4, depending on how
	testing and review goes.

o	Eliminate expedited-grace-period-induced OS jitter on idle CPUs.
	This went into v3.19.  Note that this also avoids IPIing
	nohz_full CPUs.

o	I believe that I can reduce OS jitter by a further factor of two
	(worst case) or factor of five (average case), but I am still
	thinking about exactly how to do this.	(This also has the
	benefit of shutting up a lockdep false positive.)

o	There is a global counter that synchronize_sched_expedited()
	uses to determine when all the CPUs have passed through a
	quiescent state.  This is a scalability bottleneck on modest
	systems under heavy load, so I will be switching this to
	instead use the combining tree.

o	Because both synchronize_sched_expedited() and
	synchronize_rcu_expedited() can potentially wake up each and
	every CPU, on sufficiently large systems, they are quite slow.
	If this scalability problem ever becomes real, I intend to use
	multiple kthreads to do the wakeups on large systems.

Seem reasonable?

							Thanx, Paul

------------------------------------------------------------------------

scripts: Make checkpatch.pl warn on expedited RCU grace periods

The synchronize_rcu_expedited() and synchronize_sched_expedited()
expedited-grace-period primitives induce OS jitter, which can degrade
real-time response.  This commit therefore adds a checkpatch.pl warning
on any patch adding them.

Note that this patch does not warn on synchronize_srcu_expedited()
because it does not induce OS jitter, courtesy of its otherwise
much-maligned read-side memory barriers.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Andy Whitcroft <apw@canonical.com>
Cc: Joe Perches <joe@perches.com>

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 89b1df4e72ab..ddd82d743bba 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -4898,6 +4898,12 @@ sub process {
 				     "memory barrier without comment\n" . $herecurr);
 			}
 		}
+# check for expedited grace periods that interrupt CPUs.
+# note that synchronize_srcu_expedited() does -not- do this, so no complaints.
+		if ($line =~ /\b(synchronize_rcu_expedited|synchronize_sched_expedited)\(/) {
+			WARN("EXPEDITED_RCU_GRACE_PERIOD",
+			     "expedited RCU grace periods should be avoided\n" . $herecurr);
+		}
 # check of hardware specific defines
 		if ($line =~ m@^.\s*\#\s*if.*\b(__i386__|__powerpc64__|__sun__|__s390x__)\b@ && $realfile !~ m@include/asm-@) {
 			CHK("ARCH_DEFINES",

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/