Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 10 Feb 2020 12:33:04 -0500 (EST)
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To:     rostedt <rostedt@goodmis.org>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        "Joel Fernandes, Google" <joel@joelfernandes.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Gustavo A. R. Silva" <gustavo@embeddedor.com>,
        Ingo Molnar <mingo@redhat.com>,
        Richard Fontana <rfontana@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        paulmck <paulmck@kernel.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Message-ID: <1966694237.616758.1581355984287.JavaMail.zimbra@efficios.com>
In-Reply-To: <20200210120552.1a06a7aa@gandalf.local.home>
References: <20200207205656.61938-1-joel@joelfernandes.org> <1997032737.615438.1581179485507.JavaMail.zimbra@efficios.com> <20200210094616.GC14879@hirez.programming.kicks-ass.net> <20200210120552.1a06a7aa@gandalf.local.home>
Subject: Re: [RFC 0/3] Revert SRCU from tracepoint infrastructure
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Thread-Topic: Revert SRCU from tracepoint infrastructure
Thread-Index: cU4abBwtS1928irOMYESy3r+/umCMg==
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

----- On Feb 10, 2020, at 12:05 PM, rostedt rostedt@goodmis.org wrote:

> On Mon, 10 Feb 2020 10:46:16 +0100
> Peter Zijlstra <peterz@infradead.org> wrote:
> 
>> Furthermore, using srcu would be detrimental, because of how it has
>> smp_mb() in the read side primitives.
> 
> I didn't realize that there was a full memory barrier in the srcu read
> side. Seems to me that itself is rational for reverting it. And also a
> big NAK for any suggestion to have any of the function tracing to use
> it as well (which comes up here and there).

The rcu_irq_enter/exit_irqson() does atomic_add_return(), which is even worse
than a memory barrier.

Let me summarize my understanding of a few use-cases we have with tracepoints
and other instrumentation mechanisms and the guarantees they provide (or not):

* Tracepoints
  - Uses sched-rcu (typically)
  - Uses SRCU for _cpuidle callsites
  - Planned use of SRCU to allow syscall entry/exit instrumentation to
    take page faults. (currently all tracers paper over that issue by filling
    with zeroes rather than handle the fault)
  - Grace period waits for both sched-rcu and SRCU.

* kprobes/kretprobes
  - interrupts off around probe invocation

* Hardware performance counters
  - Probe invoked from NMI context 

- Software performance counters
  - preempt off around probe invocation

Moving _rcuidle instrumentation to SRCU aimed at removing a significant
overhead incurred by having all _rcuidle tracepoints perform the atomic_add_return
on the shared variable (which is frequent enough to impact performance).

There are a couple of approaches that perf could take in order to tackle this
without hurting performance for all other tracers:

- If perf wishes to keep using explicit rcu_read_lock/unlock in its probes:

Use is_rcu_watching() within the perf probe, and only invoke rcu_irq_enter/exit_irqson()
when needed.

As an alternative, perf could implement a "trampoline" which would only be used
when registering a perf probe to a _rcuidle tracepoint. That trampoline would
perform rcu_irq_entrer/exit_irqson() around the call to the real perf probe.

- If perf can remove the redundant RCU read-side lock/unlock and replace this
  by waiting for the relevant RCU/SRCU grace periods instead:

Basically, looking at all the instrumentation sources perf uses, all of them
already provide some kind of RCU guarantee, which makes the explicit rcu read-side
locks within the perf probes redundant. Removing the redundant rcu read-side lock/unlock
from the perf probes should bring a slight performance improvement as well.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com