Received-SPF: pass (google.com: domain of linux-kernel+bounces-189872-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Sun, 26 May 2024 22:57:04 +0200
From: Frederic Weisbecker <frederic@kernel.org>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Oleg Nesterov <oleg@redhat.com>, Ingo Molnar <mingo@redhat.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	Peter Zijlstra <peterz@infradead.org>, Phil Auld <pauld@redhat.com>,
	Chris von Recklinghausen <crecklin@redhat.com>,
	linux-kernel@vger.kernel.org
Subject: Re: sched/isolation: tick_take_do_timer_from_boot() calls
 smp_call_function_single() with irqs disabled
Message-ID: <ZlOiIONRExNkgZ0R@pavilion.home>
References: <20240522151742.GA10400@redhat.com>
 <20240523132358.GA1965@redhat.com>
 <87h6eneeu7.ffs@tglx>
 <ZlCwKk65-eL0FrKX@pavilion.home>
 <20240524183700.GA17065@redhat.com>
 <87v832dfw1.ffs@tglx>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <87v832dfw1.ffs@tglx>

Le Sat, May 25, 2024 at 12:06:06AM +0200, Thomas Gleixner a ?crit :
> On Fri, May 24 2024 at 20:37, Oleg Nesterov wrote:
> 
> > I've already had a few beers today, I know I'll regret about this
> > email tomorrow, but I can't resist ;)
> 
> You won't regret it. :)
> 
> > On 05/24, Frederic Weisbecker wrote:
> > But again, again. tick_sched_do_timer() says
> >
> > 	* If nohz_full is enabled, this should not happen because the
> > 	* 'tick_do_timer_cpu' CPU never relinquishes.
> >
> > so I guess it is not supposed to happen?
> 
> Right. It does not happen because the kernel starts with jiffies as
> clocksource except on S390. The jiffies clocksource is not qualified to
> switch over to NOHZ mode for obvious reasons. But even on S390 which has
> a truly usable and useful clocksource the tick stays periodic to begin
> with. Why?
> 
> The NOHZ ready notification happens late in the boot process via:
> fs_initcall(clocksource_done_booting)
> 
> So by the time that happens, the secondary CPUs are up and have taken
> over the do timer duty.
> 
> [    0.600381] smp: Bringing up secondary CPUs ...
> 
> ....
> 
> [    1.917842] clocksource: Switched to clocksource kvm-clock
> [    1.918548] clocksource_done_booting: Switched to NOHZ // debug printk
> 
> This is the point where tick_nohz_activate() is called first time and
> that does:
> 
>   tick_sched_flag_set(ts, TS_FLAG_NOHZ);
> 
> So up to this point the tick is never stopped neither on housekeeping
> nor on NOHZ FULL CPUs:
> 
> tick_nohz_full_update_tick()
>   if (!tick_sched_flag_test(ts, TS_FLAG_NOHZ))
>     return;

Oh nice! I indeed completely skipped the clocksource part of the picture!

Thanks.