Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 5 May 2020 15:02:24 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     LKML <linux-kernel@vger.kernel.org>, x86@kernel.org,
        Andy Lutomirski <luto@kernel.org>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Brian Gerst <brgerst@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Will Deacon <will@kernel.org>
Subject: Re: [patch V4 part 3 11/29] rcu: Provide rcu_irq_exit_preempt()
Message-ID: <20200505220224.GT2869@paulmck-ThinkPad-P72>
Reply-To: paulmck@kernel.org
References: <20200505134354.774943181@linutronix.de>
 <20200505134904.364456424@linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200505134904.364456424@linutronix.de>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, May 05, 2020 at 03:44:05PM +0200, Thomas Gleixner wrote:
> Interrupts and exceptions invoke rcu_irq_enter() on entry and need to
> invoke rcu_irq_exit() before they either return to the interrupted code or
> invoke the scheduler due to preemption.
> 
> The general assumption is that RCU idle code has to have preemption
> disabled so that a return from interrupt cannot schedule. So the return
> from interrupt code invokes rcu_irq_exit() and preempt_schedule_irq().
> 
> If there is any imbalance in the rcu_irq/nmi* invocations or RCU idle code
> had preemption enabled then this goes unnoticed until the CPU goes idle or
> some other RCU check is executed.
> 
> Provide rcu_irq_exit_preempt() which can be invoked from the
> interrupt/exception return code in case that preemption is enabled. It
> invokes rcu_irq_exit() and contains a few sanity checks in case that
> CONFIG_PROVE_RCU is enabled to catch such issues directly.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: "Paul E. McKenney" <paulmck@kernel.org>
> Cc: Joel Fernandes <joel@joelfernandes.org>

The ->dynticks_nmi_nesting field is going away at some point, but
there is always "git merge".  ;-)

Reviewed-by: Paul E. McKenney <paulmck@kernel.org>

> ---
>  include/linux/rcutiny.h |    1 +
>  include/linux/rcutree.h |    1 +
>  kernel/rcu/tree.c       |   21 +++++++++++++++++++++
>  3 files changed, 23 insertions(+)
> 
> --- a/include/linux/rcutiny.h
> +++ b/include/linux/rcutiny.h
> @@ -71,6 +71,7 @@ static inline void rcu_irq_enter(void) {
>  static inline void rcu_irq_exit_irqson(void) { }
>  static inline void rcu_irq_enter_irqson(void) { }
>  static inline void rcu_irq_exit(void) { }
> +static inline void rcu_irq_exit_preempt(void) { }
>  static inline void exit_rcu(void) { }
>  static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)
>  {
> --- a/include/linux/rcutree.h
> +++ b/include/linux/rcutree.h
> @@ -46,6 +46,7 @@ void rcu_idle_enter(void);
>  void rcu_idle_exit(void);
>  void rcu_irq_enter(void);
>  void rcu_irq_exit(void);
> +void rcu_irq_exit_preempt(void);
>  void rcu_irq_enter_irqson(void);
>  void rcu_irq_exit_irqson(void);
>  
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -706,6 +706,27 @@ void noinstr rcu_irq_exit(void)
>  	rcu_nmi_exit();
>  }
>  
> +/**
> + * rcu_irq_exit_preempt - Inform RCU that current CPU is exiting irq
> + *			  towards in kernel preemption
> + *
> + * Same as rcu_irq_exit() but has a sanity check that scheduling is safe
> + * from RCU point of view. Invoked from return from interrupt before kernel
> + * preemption.
> + */
> +void rcu_irq_exit_preempt(void)
> +{
> +	lockdep_assert_irqs_disabled();
> +	rcu_nmi_exit();
> +
> +	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nesting) <= 0,
> +			 "RCU dynticks_nesting counter underflow/zero!");
> +	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nmi_nesting) <= 0,
> +			 "RCU dynticks_nmi_nesting counter underflow/zero!");
> +	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
> +			 "RCU in extended quiescent state!");
> +}
> +
>  /*
>   * Wrapper for rcu_irq_exit() where interrupts are enabled.
>   *
>