Received-SPF: pass (google.com: domain of linux-kernel+bounces-190469-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
Mime-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8
Date: Mon, 27 May 2024 21:16:57 +1000
Message-Id: <D1KDV2ZCWZF5.29STW6VA0LPX8@gmail.com>
Subject: Re: sched/isolation: tick_take_do_timer_from_boot() calls
 smp_call_function_single() with irqs disabled
From: "Nicholas Piggin" <npiggin@gmail.com>
To: "Thomas Gleixner" <tglx@linutronix.de>, "Oleg Nesterov"
 <oleg@redhat.com>, "Frederic Weisbecker" <frederic@kernel.org>
Cc: "Ingo Molnar" <mingo@redhat.com>, "Peter Zijlstra"
 <peterz@infradead.org>, "Phil Auld" <pauld@redhat.com>, "Chris von
 Recklinghausen" <crecklin@redhat.com>, <linux-kernel@vger.kernel.org>
References: <20240522151742.GA10400@redhat.com>
 <20240523132358.GA1965@redhat.com> <87h6eneeu7.ffs@tglx>
 <ZlCwKk65-eL0FrKX@pavilion.home> <20240524183700.GA17065@redhat.com>
 <87v832dfw1.ffs@tglx> <D1KB5Z1455DI.3HOVYR566ZGXN@gmail.com>
 <87ed9nd041.ffs@tglx>
In-Reply-To: <87ed9nd041.ffs@tglx>

On Mon May 27, 2024 at 8:23 PM AEST, Thomas Gleixner wrote:
> On Mon, May 27 2024 at 19:10, Nicholas Piggin wrote:
> > On Sat May 25, 2024 at 8:06 AM AEST, Thomas Gleixner wrote:
> >> along with the removal of the SMP function call voodoo programming gun=
k,
> >> a lengthy changelog and a bunch of useful comments.
> >
> > I might not have tested that path on powerpc since it should not
> > switch clockevent driver (or clocksource either I think) at least
> > on 64-bit.  Explains the smp_call_function warning if you are
> > testing on x86 :/
>
> Even on PowerPC the per CPU clockevent driver is registered during early
> boot for the boot CPU and for the APs when they are onlined. Before your
> change the boot CPU was unconditionally taking over the do_timer duty
> and never gave up on it in the NOHZ full case.
>
> The logic you added allows that the duty is taken by the first
> housekeeping CPU in the case that the boot CPU is marked NOHZ full.
>
> So yes, that function call _is_ invoked on PowerPC too if the boot CPU
> is NOHZ full. There is absolutely nothing x86 specific.

Okay, right you are. Indeed it does trigger.

> The difference is that PowerPC registers the per CPU clockevent _before_
> setting the CPU online and x86 does it afterwards.
>
> So the warning does not trigger on PowerPC because:
>
>    WARN_ON_ONCE(cpu_online(this_cpu) && irqs_disabled() && ...);

That explains it.

Thanks,
Nick