Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 13 Aug 2019 14:30:19 +0200
From:   Frederic Weisbecker <frederic@kernel.org>
To:     "Paul E. McKenney" <paulmck@linux.ibm.com>
Cc:     rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
        mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
        akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
        josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org,
        rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
        fweisbec@gmail.com, oleg@redhat.com, joel@joelfernandes.org
Subject: Re: [PATCH RFC tip/core/rcu 14/14] rcu/nohz: Make multi_cpu_stop()
 enable tick on all online CPUs
Message-ID: <20190813123016.GA11455@lenoir>
References: <20190802151435.GA1081@linux.ibm.com>
 <20190802151501.13069-14-paulmck@linux.ibm.com>
 <20190812210232.GA3648@lenoir>
 <20190812232316.GT28441@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190812232316.GT28441@linux.ibm.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Aug 12, 2019 at 04:23:16PM -0700, Paul E. McKenney wrote:
> On Mon, Aug 12, 2019 at 11:02:33PM +0200, Frederic Weisbecker wrote:
> > On Fri, Aug 02, 2019 at 08:15:01AM -0700, Paul E. McKenney wrote:
> > Looks like it's not the right fix but, should you ever need to set an
> > all-CPUs (system wide) tick dependency in the future, you can use tick_set_dep().
> 
> Indeed, I have dropped this patch, but I now do something similar in
> RCU's CPU-hotplug notifiers.  Which does have an effect, especially on
> the system that isn't subject to the insane-latency cpu_relax().
> 
> Plus I am having to put a similar workaround into RCU's quiescent-state
> forcing logic.
> 
> But how should this really be done?
> 
> Isn't there some sort of monitoring of nohz_full CPUs for accounting
> purposes?  If so, would it make sense for this monitoring to check for
> long-duration kernel execution and enable the tick in this case?  The
> RCU dyntick machinery can be used to remotely detect the long-duration
> kernel execution using something like the following:
> 
> 	int nohz_in_kernel_snap = rcu_dynticks_snap_cpu(cpu);
> 
> 	...
> 
> 	if (rcu_dynticks_in_eqs_cpu(cpu, nohz_in_kernel_snap)
> 		nohz_in_kernel_snap = rcu_dynticks_snap_cpu(cpu);
> 	else
> 		/* Turn on the tick! */
> 
> I would supply rcu_dynticks_snap_cpu() and rcu_dynticks_in_eqs_cpu(),
> which would be simple wrappers around RCU's private rcu_dynticks_snap()
> and rcu_dynticks_in_eqs() functions.
> 
> Would this make sense as a general solution, or am I missing a corner
> case or three?

Oh I see. Until now we considered than running into the kernel (between user/guest/idle)
is supposed to be short but there can be specific places where it doesn't apply.

I'm wondering if, more than just providing wrappers, this shouldn't be entirely
driven by RCU using the tick_set_dep_cpu()/tick_clear_dep_cpu() at appropriate timings.

I don't want to sound like I'm trying to put all the work on you :p  It's just that
the tick shouldn't know much about RCU, it's rather RCU that is a client for the tick and
is probably better suited to determine when a CPU becomes annoying with its extended grace
period.

Arming a CPU timer could also be an alternative to tick_set_dep_cpu() for that.

What do you think?