Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 10 Aug 2023 10:07:08 +0200
From:   Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To:     Valentin Schneider <vschneid@redhat.com>
Cc:     linux-kernel@vger.kernel.org, Ben Segall <bsegall@google.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Mel Gorman <mgorman@suse.de>,
        Peter Zijlstra <peterz@infradead.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH] sched/rt: Don't try push tasks if there are none.
Message-ID: <20230810080708.L4nV_aKI@linutronix.de>
References: <20230801152648._y603AS_@linutronix.de>
 <xhsmhv8dob13r.mognet@vschneid.remote.csb>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <xhsmhv8dob13r.mognet@vschneid.remote.csb>
Precedence: bulk

On 2023-08-09 18:02:32 [+0100], Valentin Schneider wrote:
> On 01/08/23 17:26, Sebastian Andrzej Siewior wrote:
> > I have a RT task X at a high priority and cyclictest on each CPU with
> > lower priority than X's. If X is active and each CPU wakes their own
> > cylictest thread then it ends in a longer rto_push storm.
> > A random CPU determines via balance_rt() that the CPU on which X is
> > running needs to push tasks. X has the highest priority, cyclictest is
> > next in line so there is nothing that can be done since the task with
> > the higher priority is not touched.
> >
> > tell_cpu_to_push() increments rto_loop_next and schedules
> > rto_push_irq_work_func() on X's CPU. The other CPUs also increment the
> > loop counter and do the same. Once rto_push_irq_work_func() is active it
> > does nothing because it has _no_ pushable tasks on its runqueue. Then
> > checks rto_next_cpu() and decides to queue irq_work on the local CPU
> > because another CPU requested a push by incrementing the counter.
> >
> 
> For a CPU to be in the rto_mask, it needs:
> 
>   rt_rq->rt_nr_migratory && rt_rq->rt_nr_total > 1
> 
> But if that CPU has no pushable tasks, then that means only the current
> task has p->nr_cpus_allowed > 1.
> 
> Should we change it so a CPU is only in the rto_mask iff it has pushable
> tasks? AFAICT that should not break the case where we push the current task
> away due to migration_disabled, as that still relies on the
> migration_disabled task to be in the pushable list.

Sounds good. The task with the highest priority becomes pushable if it
gets preempted (by a task with higher priority). This gets considered,
right?

Sebastian