From:   Peng Wang <rocking@linux.alibaba.com>
To:     mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com
Cc:     linux-kernel@vger.kernel.org
Subject: [PATCH] Add busy loop polling for idle SMT
Date:   Tue, 16 Nov 2021 19:51:35 +0800
Message-Id: <cover.1637062971.git.rocking@linux.alibaba.com>
Precedence: bulk

Now we have cpu_idle_force_poll which uses cpu_relax() waiting for
an arriving IPI, while sometimes busy loop on idle cpu is also
useful to provide consistent pipeline interference for hardware SMT.

When hardware SMT is enabled, the switching between idle and
busy state of one cpu will cause performance fluctuation of
other sibling cpus on the same core.

In pay-for-execution-time scenario, cloud service providers prefer
stable performance data to set stabel price for same workload.
Different execution time of the same workload caused by different
idle or busy state of sibling SMT cpus will make different bills, which
is confused for customers.

Since there is no dynamic CPU time scaling based on SMT pipeline interference,
to coordinate sibling SMT noise no matter whether they are idle or not,
busy loop in idle state can provide approximately consistent pipeline interference.

For example, a workload computing tangent and cotangent will finish in 9071ms when
sibling SMT cpus are idle, and 13299ms when sibling SMT cpus are computiing other workload.
This generate 32% performance fluctuation.

SMT idle polling makes things slower, but we can set bigger cpu quota to make up
a deficiency. This also increase power consumption by 2.2%, which is acceptable.

There may be some other possible solutions, while each has its own problem:
a) disbale hardware SMT, which means half of SMT is unused and more hardware cost.
b) busy loop in a userspace thread, but the cpu usage is confusing.

We propose this patch to discuss the performance fluctuation problem related to SMT
pipeline interference, and any comments are welcome.

Peng Wang (1):
  sched/idle: support busy loop polling on idle SMT cpus

 arch/Kconfig         | 10 ++++++
 kernel/sched/core.c  | 39 ++++++++++++++++++++++
 kernel/sched/idle.c  | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/sched.h | 18 +++++++++++
 4 files changed, 158 insertions(+)

-- 
2.9.5