Received-SPF: pass (google.com: domain of linux-kernel+bounces-189370-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Sat, 25 May 2024 15:51:20 +0200
From: Oleg Nesterov <oleg@redhat.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Frederic Weisbecker <frederic@kernel.org>,
	Ingo Molnar <mingo@redhat.com>, Nicholas Piggin <npiggin@gmail.com>,
	Peter Zijlstra <peterz@infradead.org>, Phil Auld <pauld@redhat.com>,
	Chris von Recklinghausen <crecklin@redhat.com>,
	linux-kernel@vger.kernel.org
Subject: Re: sched/isolation: tick_take_do_timer_from_boot() calls
 smp_call_function_single() with irqs disabled
Message-ID: <20240525135120.GA24152@redhat.com>
References: <20240522151742.GA10400@redhat.com>
 <20240523132358.GA1965@redhat.com>
 <87h6eneeu7.ffs@tglx>
 <ZlCwKk65-eL0FrKX@pavilion.home>
 <20240524183700.GA17065@redhat.com>
 <87v832dfw1.ffs@tglx>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87v832dfw1.ffs@tglx>
User-Agent: Mutt/1.5.24 (2015-08-30)

Thomas, thanks a lot!

Let me grep a bit more to better understand your explanations.

Just one note for now.

On 05/25, Thomas Gleixner wrote:
>
> --- a/kernel/time/tick-common.c
> +++ b/kernel/time/tick-common.c
> @@ -229,11 +209,9 @@ static void tick_setup_device(struct tick_device *td,
>  			if (tick_nohz_full_cpu(cpu))
>  				tick_do_timer_boot_cpu = cpu;
>
> -		} else if (tick_do_timer_boot_cpu != -1 &&
> -						!tick_nohz_full_cpu(cpu)) {
> -			tick_take_do_timer_from_boot();
> +		} else if (tick_do_timer_boot_cpu != -1 && !tick_nohz_full_cpu(cpu)) {
> +			WRITE_ONCE(tick_do_timer_cpu, cpu);
>  			tick_do_timer_boot_cpu = -1;
> -			WARN_ON(READ_ONCE(tick_do_timer_cpu) != cpu);
>  #endif
>  		}
>
> along with the removal of the SMP function call voodoo programming gunk,

Yes,

> Changing the horribly lazy and incomprehensible '-1' to an actual
> meaningful define, e.g. TICK_DO_TIMER_NONE, would definitely help along
> with renaming the variable to tick_do_timer_nohz_full_boot_cpu.

Better yet, we can make it a boolean, we do not need cpu number. And
perhaps we can simply kill it along with #ifdef CONFIG_NO_HZ_FULL ?

	if (!td->evtdev) {
		tick_cpu = READ_ONCE(tick_do_timer_cpu);
		/*
		 * If no cpu took the do_timer update, assign it to
		 * this cpu:
		 */
		if (tick_cpu == TICK_DO_TIMER_BOOT) {
			WRITE_ONCE(tick_do_timer_cpu, cpu);
			tick_next_period = ktime_get();
			/*
			 * The boot CPU may be nohz_full, in which case the
			 * first housekeeping secondary will take do_timer
			 * from us.
			 */
		} else if (tick_nohz_full_cpu(tick_cpu) &&
			  !tick_nohz_full_cpu(cpu)) {
			WRITE_ONCE(tick_do_timer_cpu, cpu);
		}

Oleg.