Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 19 Mar 2020 07:31:38 -0400 (EDT)
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To:     paulmck <paulmck@kernel.org>
Cc:     rcu <rcu@vger.kernel.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        kernel-team <kernel-team@fb.com>, Ingo Molnar <mingo@kernel.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        dipankar <dipankar@in.ibm.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        rostedt <rostedt@goodmis.org>,
        David Howells <dhowells@redhat.com>,
        Eric Dumazet <edumazet@google.com>,
        fweisbec <fweisbec@gmail.com>, Oleg Nesterov <oleg@redhat.com>,
        "Joel Fernandes, Google" <joel@joelfernandes.org>
Message-ID: <1560487611.2836.1584617498827.JavaMail.zimbra@efficios.com>
In-Reply-To: <20200319001024.GA28798@paulmck-ThinkPad-P72>
References: <20200312181618.GA21271@paulmck-ThinkPad-P72> <20200319001024.GA28798@paulmck-ThinkPad-P72>
Subject: Re: [PATCH RFC v2 tip/core/rcu 0/22] Prototype RCU usable from
 idle, exception, offline
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Thread-Topic: Prototype RCU usable from idle, exception, offline
Thread-Index: /+aQCcRF3e2v6wZPv8fU+JWQy6IuTg==
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

----- On Mar 18, 2020, at 8:10 PM, paulmck paulmck@kernel.org wrote:

> Hello!

Hi Paul,

Thanks for pulling this together! Some comments below (based only on the
cover message),

[...]

> There are of course downsides.  The grace-period code can send IPIs to
> CPUs, even when those CPUs are in the idle loop or in nohz_full userspace.
> However, this version enlists the aid of the context-switch hooks,
> which eliminates the need for IPIs in context-switch-heavy workloads.
> It also prohibits sending of IPIs early in the grace period, which
> provides additional opportunity for the hooks to do their job.  Additional
> IPI-reduction mechanisms are under development.

I suspect that on nohz_full cpus, at least some use-cases which really care
about not receiving IPIs will not be doing that many context switches.

What are the possible approaches to have IPI-*elimination* for nohz cpus ?

> 
> The RCU tasks trace mechanism is based off of RCU tasks rather than
> SRCU because the latter is more complex and also because the latter
> uses a CPU-by-CPU approach to tracking quiescent states instead of the
> task-by-task approach that is needed.  It is in theory possible to
> mash RCU tasks trace into the Tree SRCU implementation, but there
> will need to be extremely good reasons for doing so.

I have a hard time buying the "less complexity" argument to justify the
introduction of yet another flavor of RCU when a close match already
exists (SRCU).

The other argument for this task-based RCU (rather than CPU-by-CPU as
done by SRCU) is that "a task-by-task approach is needed". What I
do not get from this explanation is why is such an approach needed ?

Also, another aspect worth discussing here is the use-cases which
need to be covered by tracing-rcu. Is this specific flavor targeting
specifically preempt-off use-cases, or is the goal here to target
use-cases which may trigger major page faults within the read-side
critical section as well ?

Note that doing task-by-task tracking of tracing-rcu rather than
cpu-by-cpu is not free: AFAIU it bloats the task struct (always)
for a use-case which is not always active. My experience with
tracepoints and asm gotos is that we need to be careful not to
slow down the common case (kernel running without any tracing
active, but tracing configured in) if we want to keep distributions
and end users building kernels with introspection facilities in
place.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com