Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 29 Jul 2019 11:58:24 +0200 (CEST)
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Peter Zijlstra <peterz@infradead.org>
cc:     Guenter Roeck <linux@roeck-us.net>, x86@kernel.org,
        Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org,
        Borislav Petkov <bp@alien8.de>
Subject: Re: sched: Unexpected reschedule of offline CPU#2!
In-Reply-To: <20190729093545.GV31381@hirez.programming.kicks-ass.net>
Message-ID: <alpine.DEB.2.21.1907291156170.1791@nanos.tec.linutronix.de>
References: <20190727164450.GA11726@roeck-us.net> <20190729093545.GV31381@hirez.programming.kicks-ass.net>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, 29 Jul 2019, Peter Zijlstra wrote:
> On Sat, Jul 27, 2019 at 09:44:50AM -0700, Guenter Roeck wrote:
> > [   61.348866] Call Trace:
> > [   61.349392]  kick_ilb+0x90/0xa0
> > [   61.349629]  trigger_load_balance+0xf0/0x5c0
> > [   61.349859]  ? check_preempt_wakeup+0x1b0/0x1b0
> > [   61.350057]  scheduler_tick+0xa7/0xd0
> 
> kick_ilb() iterates nohz.idle_cpus_mask to find itself an idle_cpu().
> 
> idle_cpus_mask() is set from nohz_balance_enter_idle() and cleared from
> nohz_balance_exit_idle(). nohz_balance_enter_idle() is called from
> __tick_nohz_idle_stop_tick() when entering nohz idle, this includes the
> cpu_is_offline() clause of the idle loop.
> 
> However, when offline, cpu_active() should also be false, and this
> function should no-op.

Ha. That reboot mess is not clearing cpu active as it's not going through
the regular cpu hotplug path. It's using reboot IPI which 'stops' the cpus
dead in their tracks after clearing cpu online....

Thanks,

	tglx