IPCC statistics are used during idle load balancing. After balancing one
of the siblings of an SMT core becomes idle. The remaining busy siblings
experience increased throughput. The IPCC statistics provide a measure of
the increased throughput. Use them to select the busiest group between
otherwise identical fully_busy scheduling groups. (The avg_load is not
computed in this case and is zero for both groups).
IPCC scores are not needed to break ties with non-SMT fully_busy sched
groups. SMT sched groups always need more help.
Add a stub sched_asym_ipcc_prefer() to handle !CONFIG_IPC_CLASSES.
Cc: Ben Segall <[email protected]>
Cc: Daniel Bristot de Oliveira <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: Ionela Voinescu <[email protected]>
Cc: Joel Fernandes (Google) <[email protected]>
Cc: Len Brown <[email protected]>
Cc: Lukasz Luba <[email protected]>
Cc: Mel Gorman <[email protected]>
Cc: Perry Yuan <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>
Cc: Srinivas Pandruvada <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Tim C. Chen <[email protected]>
Cc: Valentin Schneider <[email protected]>
Cc: Zhao Liu <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Ricardo Neri <[email protected]>
---
Changes since v3:
* None
Changes since v2:
* Introduced this patch.
Changes since v1:
* N/A
---
kernel/sched/fair.c | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index fb3d793fe9ad..fcec791ede4f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9580,6 +9580,12 @@ static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs,
{
}
+static bool sched_asym_ipcc_prefer(struct sg_lb_stats *a,
+ struct sg_lb_stats *b)
+{
+ return false;
+}
+
static bool sched_asym_ipcc_pick(struct sched_group *a,
struct sched_group *b,
struct sg_lb_stats *a_stats,
@@ -9861,10 +9867,21 @@ static bool update_sd_pick_busiest(struct lb_env *env,
if (sgs->avg_load == busiest->avg_load) {
/*
* SMT sched groups need more help than non-SMT groups.
- * If @sg happens to also be SMT, either choice is good.
*/
- if (sds->busiest->flags & SD_SHARE_CPUCAPACITY)
- return false;
+ if (sds->busiest->flags & SD_SHARE_CPUCAPACITY) {
+ if (!(sg->flags & SD_SHARE_CPUCAPACITY))
+ return false;
+
+ /*
+ * Between two SMT groups, use IPCC scores to pick the
+ * one that would improve throughput the most (only
+ * asym_packing uses IPCC scores for now).
+ */
+ if (sched_ipcc_enabled() &&
+ env->sd->flags & SD_ASYM_PACKING &&
+ sched_asym_ipcc_prefer(busiest, sgs))
+ return false;
+ }
}
break;
--
2.25.1