Received-SPF: pass (google.com: domain of linux-kernel+bounces-48536-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Thu, 1 Feb 2024 17:33:00 +0100
From: Frederic Weisbecker <frederic@kernel.org>
To: Anna-Maria Behnsen <anna-maria@linutronix.de>
Cc: linux-kernel@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
	John Stultz <jstultz@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Eric Dumazet <edumazet@google.com>,
	"Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
	Arjan van de Ven <arjan@infradead.org>,
	"Paul E . McKenney" <paulmck@kernel.org>,
	Rik van Riel <riel@surriel.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Sebastian Siewior <bigeasy@linutronix.de>,
	Giovanni Gherdovich <ggherdovich@suse.cz>,
	Lukasz Luba <lukasz.luba@arm.com>,
	"Gautham R . Shenoy" <gautham.shenoy@amd.com>,
	Srinivas Pandruvada <srinivas.pandruvada@intel.com>,
	K Prateek Nayak <kprateek.nayak@amd.com>
Subject: Re: [PATCH v10 18/20] timers: Implement the hierarchical pull model
Message-ID: <ZbvHvKdpgqfsJp0E@localhost.localdomain>
References: <20240115143743.27827-1-anna-maria@linutronix.de>
 <20240115143743.27827-19-anna-maria@linutronix.de>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240115143743.27827-19-anna-maria@linutronix.de>

Le Mon, Jan 15, 2024 at 03:37:41PM +0100, Anna-Maria Behnsen a ?crit :
> +static bool tmigr_handle_remote_up(struct tmigr_group *group,
> +				   struct tmigr_group *child,
> +				   void *ptr)
> +{
> +	struct tmigr_remote_data *data = ptr;
> +	u64 now, next = KTIME_MAX;
> +	struct tmigr_event *evt;
> +	unsigned long jif;
> +	u8 childmask;
> +
> +	jif = data->basej;
> +	now = data->now;
> +
> +	childmask = data->childmask;
> +
> +again:
> +	/*
> +	 * Handle the group only if @childmask is the migrator or if the
> +	 * group has no migrator. Otherwise the group is active and is
> +	 * handled by its own migrator.
> +	 */
> +	if (!tmigr_check_migrator(group, childmask))
> +		return true;
> +
> +	raw_spin_lock_irq(&group->lock);
> +
> +	evt = tmigr_next_expired_groupevt(group, now);
> +
> +	if (evt) {
> +		unsigned int remote_cpu = evt->cpu;
> +
> +		raw_spin_unlock_irq(&group->lock);
> +
> +		next = tmigr_handle_remote_cpu(remote_cpu, now, jif);
> +
> +		/* check if there is another event, that needs to be handled */
> +		goto again;
> +	} else {
> +		raw_spin_unlock_irq(&group->lock);
> +	}
> +
> +	/*
> +	 * Update of childmask for the next level and keep track of the expiry
> +	 * of the first event that needs to be handled
> +	 */
> +	data->childmask = group->childmask;
> +	data->firstexp = next;

So assume we have:

            [GRP1:0]
        migrator = [GRP0:0]
        active   = [GRP0:0]
        nextevt  = TIMER3
        /                    \
    [GRP0:0]                  [GRP0:1]
 migrator = CPU0           migrator = NONE
 active   = CPU0           active   = NONE
 nextevt  = KTIME_MAX      nextevt  = TIMER3
    /         \                /         \
   0           1              2           3
  idle       idle           idle         idle (TIMER3)

Then CPU 0 goes idle:

            [GRP1:0]
        migrator = NONE
        active   = NONE
        nextevt  = TIMER3
        /                    \
    [GRP0:0]                  [GRP0:1]
 migrator = NONE           migrator = NONE
 active   = NONE           active   = NONE
 nextevt  = KTIME_MAX      nextevt  = TIMER3
    /         \                /         \
   0           1              2           3
  idle       idle           idle         idle (TIMER3)

CPU 0 is the idle migrator and its tmc->wakeup is TIMER3.
But CPU 0 has a local timer that expires before TIMER3.

When that timer interrupt fires, it raises the softirq, which
executes on IRQ tail. So CPU0 eventually calls tmigr_handle_remote()
before TIMER3 has expired.

This leads to tmigr_next_expired_groupevt() to return NULL and then
data->firstexp = KTIME_MAX and then tmc->wakeup = KTIME_MAX.

Later on, tmigr_new_timer() is called with a KTIME_MAX global
event and so tmc->wakeup stays with KTIME_MAX, ignoring TIMER3.

It looks like you need to handle the tmigr_next_expired_groupevt()
case returning NULL.

Thanks.