Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     LKML <linux-kernel@vger.kernel.org>, x86@kernel.org,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Andy Lutomirski <luto@kernel.org>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Brian Gerst <brgerst@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Will Deacon <will@kernel.org>
Subject: Re: [patch V4 part 3 11/29] rcu: Provide rcu_irq_exit_preempt()
In-Reply-To: <20200514024116.GA231286@google.com>
References: <20200505134354.774943181@linutronix.de> <20200505134904.364456424@linutronix.de> <20200514024116.GA231286@google.com>
Date:   Thu, 14 May 2020 16:43:31 +0200
Message-ID: <87blmqziek.fsf@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Joel,

Joel Fernandes <joel@joelfernandes.org> writes:
> On Tue, May 05, 2020 at 03:44:05PM +0200, Thomas Gleixner wrote:
> Could you let me know which patch or part in the multi-part series is
> using it?

You found it :)
>> +void rcu_irq_exit_preempt(void)
>> +{
>> +	lockdep_assert_irqs_disabled();
>> +	rcu_nmi_exit();
>> +
>> +	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nesting) <= 0,
>> +			 "RCU dynticks_nesting counter underflow/zero!");
>
> Makes sense.
>
>> +	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nmi_nesting) <= 0,
>> +			 "RCU dynticks_nmi_nesting counter underflow/zero!");
>
> This new function will be called only from the outer-most IRQ that
> interrupted kernel mode (process context). Right? If so, a better (more
> specific) check for the second RCU_LOCKDEP_WARN above is:
>
> RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nmi_nesting) != DYNTICK_IRQ_NONIDLE,
> 			 "Bad RCU dynticks_nmi_nesting counter\n");
>
> That will make sure, it is only called from outer-most rcu_irq_exit() and
> interrupting kernel mode.

Makes sense.

> Or, if [1] is merged, then we could just combine the checks into one check.
> 	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nesting) != 1,
> 			 "Bad RCU dynticks_nmi_nesting counter\n");
>
>> +	RCU_LOCKDEP_WARN(rcu_dynticks_curr_cpu_in_eqs(),
>> +			 "RCU in extended quiescent state!");
>
> Makes sense.
>
> BTW, I wonder if a better place to do this "don't enter scheduler while RCU
> is not watching" is rcu_note_context_switch()...

I actually want to catch even the case where we don't schedule, i.e.

  if (ret_to_kernel) {
     if (interrupts_on_after_return((regs)) {
        if (IS_ENABLED(CONFIG_PREEMPTION)) {
  	   if (!preempt_count()) {
              /* Preemption is possible ... */
       	      rcu_irq_exit_preempt();
                 if (need_resched())
                    schedule_preempt_irq();

that catches any exit where preemption is possible and RCU is not
watching after rcu_irq_exit().

It does not matter whether need-resched is set here or not. Any
interrupt/exception could set it.

Yes, I'm paranoid :)

Thanks,

        tglx