Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp2569151imm; Mon, 10 Sep 2018 03:16:13 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbjXnMZST4mjMX/ft2l78QGBu6O86lbCcLLY2nIkYKaJzMaL/biC1RPWTF9vlITWIMsW873 X-Received: by 2002:a17:902:127:: with SMTP id 36-v6mr21484689plb.178.1536574573094; Mon, 10 Sep 2018 03:16:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536574573; cv=none; d=google.com; s=arc-20160816; b=0GmpFvrJJ10gN7oHPt4T60rcx2kVEallOI9NxbsDVCjJgNWsjjiVtNb0SV2ncbi62w xIXS8r6N21kPssizTmPnQrrA1JgW3sY3zNQpYdE2XRA2MxLN6lF0Mq3ykvuwQ+Syc9Hh B5DO8fKj6fUc6LbrYaPXgfHhTEyVjV/jBKUTxO7v7itPzySmxsRX7H9TCMcXOOHLjl3q 9mzOlZUoTkQZmPKyAtWSvZD7WRe12Hk0v6LAxTEw0jSgAlzVZ4pmY3PvWnY3pB9H8NjS 7bxapKPItE0Jx50HNXVpWEkNAqkwCJndGOCR8pLyPXsYz8q04iZ+SFAUKXp6lOaA5c4M CFkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-disposition :content-transfer-encoding:mime-version:robot-unsubscribe:robot-id :git-commit-id:subject:to:references:in-reply-to:reply-to:cc :message-id:from:date; bh=R1lgrhkav4bt+Ly899mztihZk5ssaJBLFIl+TooVHRs=; b=tigenac4s4mLWwclCmYS5YtP1NfY0FbuRa9W0FwMmuwFJip6xTW5WjhR7T9jEhBbd/ ycXLocPHUbQhhnAqnqa8ka4tR8eoCBzgBeI0g0c95Rx05uj0SgoBgRCoBg8pzBSF2oZg mSYDW5tlNFhxQWuL1LhbWRDvK2yMaK7k49h5r8OUTVh+ezaLwgX32SAxDIHXBYZPNKU5 fTU1dvyOmdU6dgpAeyzog11ihj18al5tNik2wl/Z+S09d6E/ueHTd6spAyjtF9JVnRXJ TbeLfpxr1U+fayFiobphMQZpVaa9FUckr6xJgAylMWVBE8ReWY2n97jFRPlF63j3DZ5P h75g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b10-v6si17113265pgi.416.2018.09.10.03.15.57; Mon, 10 Sep 2018 03:16:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728297AbeIJPIF (ORCPT + 99 others); Mon, 10 Sep 2018 11:08:05 -0400 Received: from terminus.zytor.com ([198.137.202.136]:45577 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727796AbeIJPIE (ORCPT ); Mon, 10 Sep 2018 11:08:04 -0400 Received: from terminus.zytor.com (localhost [127.0.0.1]) by terminus.zytor.com (8.15.2/8.15.2) with ESMTPS id w8AAEMiR1807901 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 10 Sep 2018 03:14:22 -0700 Received: (from tipbot@localhost) by terminus.zytor.com (8.15.2/8.15.2/Submit) id w8AAEM9r1807898; Mon, 10 Sep 2018 03:14:22 -0700 Date: Mon, 10 Sep 2018 03:14:22 -0700 X-Authentication-Warning: terminus.zytor.com: tipbot set sender to tipbot@zytor.com using -f From: tip-bot for Morten Rasmussen Message-ID: Cc: mingo@kernel.org, linux-kernel@vger.kernel.org, torvalds@linux-foundation.org, peterz@infradead.org, morten.rasmussen@arm.com, hpa@zytor.com, tglx@linutronix.de Reply-To: tglx@linutronix.de, hpa@zytor.com, linux-kernel@vger.kernel.org, morten.rasmussen@arm.com, peterz@infradead.org, torvalds@linux-foundation.org, mingo@kernel.org In-Reply-To: <1530699470-29808-4-git-send-email-morten.rasmussen@arm.com> References: <1530699470-29808-4-git-send-email-morten.rasmussen@arm.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:sched/core] sched/fair: Add sched_group per-CPU max capacity Git-Commit-ID: e3d6d0cb66f2351cbfd09fbae04eb9804afe9577 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DATE_IN_FUTURE_96_Q autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on terminus.zytor.com Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: e3d6d0cb66f2351cbfd09fbae04eb9804afe9577 Gitweb: https://git.kernel.org/tip/e3d6d0cb66f2351cbfd09fbae04eb9804afe9577 Author: Morten Rasmussen AuthorDate: Wed, 4 Jul 2018 11:17:41 +0100 Committer: Ingo Molnar CommitDate: Mon, 10 Sep 2018 11:05:49 +0200 sched/fair: Add sched_group per-CPU max capacity The current sg->min_capacity tracks the lowest per-CPU compute capacity available in the sched_group when rt/irq pressure is taken into account. Minimum capacity isn't the ideal metric for tracking if a sched_group needs offloading to another sched_group for some scenarios, e.g. a sched_group with multiple CPUs if only one is under heavy pressure. Tracking maximum capacity isn't perfect either but a better choice for some situations as it indicates that the sched_group definitely compute capacity constrained either due to rt/irq pressure on all CPUs or asymmetric CPU capacities (e.g. big.LITTLE). Signed-off-by: Morten Rasmussen Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: dietmar.eggemann@arm.com Cc: gaku.inami.xh@renesas.com Cc: valentin.schneider@arm.com Cc: vincent.guittot@linaro.org Link: http://lkml.kernel.org/r/1530699470-29808-4-git-send-email-morten.rasmussen@arm.com Signed-off-by: Ingo Molnar --- kernel/sched/fair.c | 24 ++++++++++++++++++++---- kernel/sched/sched.h | 1 + kernel/sched/topology.c | 2 ++ 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6e04bea5b11a..fe04315d57b3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7557,13 +7557,14 @@ static void update_cpu_capacity(struct sched_domain *sd, int cpu) cpu_rq(cpu)->cpu_capacity = capacity; sdg->sgc->capacity = capacity; sdg->sgc->min_capacity = capacity; + sdg->sgc->max_capacity = capacity; } void update_group_capacity(struct sched_domain *sd, int cpu) { struct sched_domain *child = sd->child; struct sched_group *group, *sdg = sd->groups; - unsigned long capacity, min_capacity; + unsigned long capacity, min_capacity, max_capacity; unsigned long interval; interval = msecs_to_jiffies(sd->balance_interval); @@ -7577,6 +7578,7 @@ void update_group_capacity(struct sched_domain *sd, int cpu) capacity = 0; min_capacity = ULONG_MAX; + max_capacity = 0; if (child->flags & SD_OVERLAP) { /* @@ -7607,6 +7609,7 @@ void update_group_capacity(struct sched_domain *sd, int cpu) } min_capacity = min(capacity, min_capacity); + max_capacity = max(capacity, max_capacity); } } else { /* @@ -7620,12 +7623,14 @@ void update_group_capacity(struct sched_domain *sd, int cpu) capacity += sgc->capacity; min_capacity = min(sgc->min_capacity, min_capacity); + max_capacity = max(sgc->max_capacity, max_capacity); group = group->next; } while (group != child->groups); } sdg->sgc->capacity = capacity; sdg->sgc->min_capacity = min_capacity; + sdg->sgc->max_capacity = max_capacity; } /* @@ -7721,16 +7726,27 @@ group_is_overloaded(struct lb_env *env, struct sg_lb_stats *sgs) } /* - * group_smaller_cpu_capacity: Returns true if sched_group sg has smaller + * group_smaller_min_cpu_capacity: Returns true if sched_group sg has smaller * per-CPU capacity than sched_group ref. */ static inline bool -group_smaller_cpu_capacity(struct sched_group *sg, struct sched_group *ref) +group_smaller_min_cpu_capacity(struct sched_group *sg, struct sched_group *ref) { return sg->sgc->min_capacity * capacity_margin < ref->sgc->min_capacity * 1024; } +/* + * group_smaller_max_cpu_capacity: Returns true if sched_group sg has smaller + * per-CPU capacity_orig than sched_group ref. + */ +static inline bool +group_smaller_max_cpu_capacity(struct sched_group *sg, struct sched_group *ref) +{ + return sg->sgc->max_capacity * capacity_margin < + ref->sgc->max_capacity * 1024; +} + static inline enum group_type group_classify(struct sched_group *group, struct sg_lb_stats *sgs) @@ -7876,7 +7892,7 @@ static bool update_sd_pick_busiest(struct lb_env *env, * power/energy consequences are not considered. */ if (sgs->sum_nr_running <= sgs->group_weight && - group_smaller_cpu_capacity(sds->local, sg)) + group_smaller_min_cpu_capacity(sds->local, sg)) return false; asym_packing: diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 7dbf67d147a2..fe17e0be2d7b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1197,6 +1197,7 @@ struct sched_group_capacity { */ unsigned long capacity; unsigned long min_capacity; /* Min per-CPU capacity in group */ + unsigned long max_capacity; /* Max per-CPU capacity in group */ unsigned long next_update; int imbalance; /* XXX unrelated to capacity but shared group state */ diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index b0cdf5e95bda..2536e1b938f9 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -693,6 +693,7 @@ static void init_overlap_sched_group(struct sched_domain *sd, sg_span = sched_group_span(sg); sg->sgc->capacity = SCHED_CAPACITY_SCALE * cpumask_weight(sg_span); sg->sgc->min_capacity = SCHED_CAPACITY_SCALE; + sg->sgc->max_capacity = SCHED_CAPACITY_SCALE; } static int @@ -852,6 +853,7 @@ static struct sched_group *get_group(int cpu, struct sd_data *sdd) sg->sgc->capacity = SCHED_CAPACITY_SCALE * cpumask_weight(sched_group_span(sg)); sg->sgc->min_capacity = SCHED_CAPACITY_SCALE; + sg->sgc->max_capacity = SCHED_CAPACITY_SCALE; return sg; }