Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 21 Aug 2023 15:03:10 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Wen Yang <wenyang.linux@foxmail.com>
Cc:     paul.gortmaker@windriver.com, LKML <linux-kernel@vger.kernel.org>,
        linux-rt-users <linux-rt-users@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        Frederic Weisbecker <frederic@kernel.org>
Subject: Re: [PATCH] tick/rcu: fix false positive "softirq work is pending"
 messages on RT
Message-ID: <ff5263ec-96a8-4e8e-96af-de6ab8a17750@paulmck-laptop>
Reply-To: paulmck@kernel.org
References: <20230818200757.1808398-1-paul.gortmaker@windriver.com>
 <tencent_C80D3EE7563151B306E94C2AA7BF3023F80A@qq.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <tencent_C80D3EE7563151B306E94C2AA7BF3023F80A@qq.com>
Precedence: bulk

On Mon, Aug 21, 2023 at 01:23:15AM +0800, Wen Yang wrote:
> 
> On 2023/8/19 04:07, paul.gortmaker@windriver.com wrote:
> > From: Paul Gortmaker <paul.gortmaker@windriver.com>
> > 
> > In commit 0345691b24c0 ("tick/rcu: Stop allowing RCU_SOFTIRQ in idle")
> > the new function report_idle_softirq() was created by breaking code out
> > of the existing can_stop_idle_tick() for kernels v5.18 and newer.
> > 
> > In doing so, the code essentially went from a one conditional:
> > 
> > 	if (a && b && c)
> > 		warn();
> > 
> > to a three conditional:
> > 
> > 	if (!a)
> > 		return;
> > 	if (!b)
> > 		return;
> > 	if (!c)
> > 		return;
> > 	warn();
> > 
> > However, it seems one of the conditionals didn't get a "!" removed.
> > Compare the instance of local_bh_blocked() in the old code:
> > 
> > -               if (ratelimit < 10 && !local_bh_blocked() &&
> > -                   (local_softirq_pending() & SOFTIRQ_STOP_IDLE_MASK)) {
> > -                       pr_warn("NOHZ tick-stop error: Non-RCU local softirq work is pending, handler #%02x!!!\n",
> > -                               (unsigned int) local_softirq_pending());
> > -                       ratelimit++;
> > -               }
> > 
> > ...to the usage in the new (5.18+) code:
> > 
> > +       /* On RT, softirqs handling may be waiting on some lock */
> > +       if (!local_bh_blocked())
> > +               return false;
> > 
> > It seems apparent that the "!" should be removed from the new code.
> > 
> > This issue lay dormant until another fixup for the same commit was added
> > in commit a7e282c77785 ("tick/rcu: Fix bogus ratelimit condition").
> > This commit realized the ratelimit was essentially set to zero instead
> > of ten, and hence *no* softirq pending messages would ever be issued.
> > 
> > Once this commit was backported via linux-stable, both the v6.1 and v6.4
> > preempt-rt kernels started printing out 10 instances of this at boot:
> > 
> >    NOHZ tick-stop error: local softirq work is pending, handler #80!!!
> > 
> > Just to double check my understanding of things, I confirmed that the
> > v5.18-rt did print the pending-80 messages with a cherry pick of the
> > ratelimit fix, and then confirmed no pending softirq messages were
> > printed with a revert of mainline's 034569 on a v5.18-rt baseline.
> > 
> > Finally I confirmed it fixed the issue on v6.1-rt and v6.4-rt, and
> > also didn't break anything on a defconfig of mainline master of today.
> > 
> > Fixes: 0345691b24c0 ("tick/rcu: Stop allowing RCU_SOFTIRQ in idle")
> > Cc: Wen Yang <wenyang.linux@foxmail.com>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Paul E. McKenney <paulmck@kernel.org>
> > Cc: Frederic Weisbecker <frederic@kernel.org>
> > Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> > 
> > diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> > index 2b865cb77feb..b52e1861b913 100644
> > --- a/kernel/time/tick-sched.c
> > +++ b/kernel/time/tick-sched.c
> > @@ -1050,7 +1050,7 @@ static bool report_idle_softirq(void)
> >   		return false;
> >   	/* On RT, softirqs handling may be waiting on some lock */
> > -	if (!local_bh_blocked())
> > +	if (local_bh_blocked())
> >   		return false;
> >   	pr_warn("NOHZ tick-stop error: local softirq work is pending, handler #%02x!!!\n",
> 
> Good catch!
> 
> Reviewed-by: Wen Yang <wenyang.linux@foxmail.com>

Frederic would normally take this, but he appears to be out.  So I am
(probably only temporarily) queueing this in -rcu for more testing
coverage.

							Thanx, Paul