Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Scott Wood <swood@redhat.com>
To:     Steven Rostedt <rostedt@goodmis.org>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>
Cc:     Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Rik van Riel <riel@surriel.com>,
        Mel Gorman <mgorman@suse.de>, linux-kernel@vger.kernel.org,
        linux-rt-users <linux-rt-users@vger.kernel.org>
Subject: [RFC PATCH 0/3] newidle_balance() latency mitigation
Date:   Tue, 28 Apr 2020 00:02:39 -0500
Message-Id: <20200428050242.17717-1-swood@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

These patches mitigate latency caused by newidle_balance() on large
systems, by enabling interrupts when the lock is dropped, and exiting
early at various points if an RT task is runnable on the current CPU.

When applied to an RT kernel on a 72-core machine (2 threads per core), I
saw significant reductions in latency as reported by rteval -- from
over 500us to around 160us with hyperthreading disabled, and from
over 1400us to around 380us with hyperthreading enabled.

This isn't the first time something like this has been tried:
https://lore.kernel.org/lkml/20121222003019.433916240@goodmis.org/
That attempt ended up being reverted:
https://lore.kernel.org/lkml/5122CD9C.9070702@oracle.com/

The problem in that case was the failure to keep BH disabled, and the
difficulty of fixing that when called from the post_schedule() hook.=20
This patchset uses finish_task_switch() to call newidle_balance(), which
enters in non-atomic context so we have full control over what we disable
and when.

There was a note at the end about wanting further discussion on the matte=
r --
does anyone remember if that ever happened and what the conclusion was?
Are there any other issues with enabling interrupts here and/or moving
the newidle_balance() call?

Rik van Riel (1):
  sched,rt: break out of load balancing if an RT task appears

Scott Wood (2):
  sched/fair: Call newidle_balance() from finish_task_switch()
  sched/fair: Enable interrupts when dropping lock in newidle_balance()

 kernel/sched/core.c  |  7 +++--
 kernel/sched/fair.c  | 72 +++++++++++++++++++++++---------------------
 kernel/sched/sched.h | 12 +++++---
 3 files changed, 49 insertions(+), 42 deletions(-)

--=20
2.18.2