Received: by 10.223.176.46 with SMTP id f43csp4499478wra; Tue, 23 Jan 2018 10:10:51 -0800 (PST) X-Google-Smtp-Source: AH8x22661NIN40rSZcGYclv4kEkA6r/AlFEVg03yKJ2L3O/MeR0d+S6PYRdpAZBdmkf/Fk0/kenb X-Received: by 10.36.117.205 with SMTP id y196mr4791502itc.51.1516731051676; Tue, 23 Jan 2018 10:10:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516731051; cv=none; d=google.com; s=arc-20160816; b=OC0kR0qjmug9TfoYjBurBLnKJamawsC4EAVuq9iIlo8J/YU0qzq9NBz+mDsqz9cIFa yeyLEAOprISUa2rPhTT2okh6xUoy0HySsR8+DBpbcVqFuFhrSLsZ4y7431gxx9Ydxpnk kABWvgk8F/ztvZEg3Dh62Rz5oqXEVAgZ6V+FeHWjnoi7UH4nf3jqo0XWUsyhcjHxzX3n d67YFY7TT9lAKKDwzaLk2Uh7WQ6eSBixZJBaI9LBsCtiLM3P9x6u6Rs82cFCU2JZ8Kgg vo9+COpAEVxuMnxryG64ssnPC3ENpTt0P2X/Pmp9VUQg/q4wRKIVnR8TBcM7B59BFcnm +JOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=lMYr5dVHBsWPlwmv60as1L/l2jsubv+rGhXwUSP2FH8=; b=mcRKuub4QXlObsDNFUxLDQzE8n6BcXta/ksOi8WcozankT3WxOPQer9Nwm8cLwmG8U YR9Dhn/4EE2Ak1N5z4fGt+QkqgF9VnCfrJLjl2T/BmPtc0TA8DFHYSy4fJD1zr8OHoJk nynK22UC7nYpa3idXO9FjuBjfe/ZxSKlz1PzLrsIAsa4n9H8ILXqCKPUNPzfKqFUfkui X56JZw6LFtp4eNuHYAzpq1lTe2zIED0RO+0yzUbXXYm0mVgWEBQwWh/4ONq7p8Dq/YlX e6CrHM3WTky4U2aZRWSSOLJJOXR3ayGEcK3jNrNwiqFl6B7QgW8jXFfSlkikm07vsDop OhBQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g12si9665071itg.2.2018.01.23.10.10.39; Tue, 23 Jan 2018 10:10:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752108AbeAWSJX (ORCPT + 99 others); Tue, 23 Jan 2018 13:09:23 -0500 Received: from foss.arm.com ([217.140.101.70]:45160 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751979AbeAWSJV (ORCPT ); Tue, 23 Jan 2018 13:09:21 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 27B6015AD; Tue, 23 Jan 2018 10:09:21 -0800 (PST) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9E5EB3F53D; Tue, 23 Jan 2018 10:09:18 -0800 (PST) From: Patrick Bellasi To: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , "Rafael J . Wysocki" , Viresh Kumar , Vincent Guittot , Paul Turner , Dietmar Eggemann , Morten Rasmussen , Juri Lelli , Todd Kjos , Joel Fernandes , Steve Muckle Subject: [PATCH v3 2/3] sched/fair: use util_est in LB and WU paths Date: Tue, 23 Jan 2018 18:08:46 +0000 Message-Id: <20180123180847.4477-3-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180123180847.4477-1-patrick.bellasi@arm.com> References: <20180123180847.4477-1-patrick.bellasi@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When the scheduler looks at the CPU utilization, the current PELT value for a CPU is returned straight away. In certain scenarios this can have undesired side effects on task placement. For example, since the task utilization is decayed at wakeup time, when a long sleeping big task is enqueued it does not add immediately a significant contribution to the target CPU. As a result we generate a race condition where other tasks can be placed on the same CPU while it is still considered relatively empty. In order to reduce this kind of race conditions, this patch introduces the required support to integrate the usage of the CPU's estimated utilization in cpu_util_wake as well as in update_sg_lb_stats. The estimated utilization of a CPU is defined to be the maximum between its PELT's utilization and the sum of the estimated utilization of the tasks currently RUNNABLE on that CPU. This allows to properly represent the spare capacity of a CPU which, for example, has just got a big task running since a long sleep period. Signed-off-by: Patrick Bellasi Reviewed-by: Dietmar Eggemann Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Viresh Kumar Cc: Paul Turner Cc: Vincent Guittot Cc: Morten Rasmussen Cc: Dietmar Eggemann Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- Changes in v3: - rebased on today's tip/sched/core (commit 07881166a892) Changes in v2: - rebase on top of v4.15-rc2 - tested that overhauled PELT code does not affect the util_est --- kernel/sched/fair.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 68 insertions(+), 6 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0bfe94f3176e..6f2e614fd79f 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6332,6 +6332,41 @@ static unsigned long cpu_util(int cpu) return (util >= capacity) ? capacity : util; } +/** + * cpu_util_est: estimated utilization for the specified CPU + * @cpu: the CPU to get the estimated utilization for + * + * The estimated utilization of a CPU is defined to be the maximum between its + * PELT's utilization and the sum of the estimated utilization of the tasks + * currently RUNNABLE on that CPU. + * + * This allows to properly represent the expected utilization of a CPU which + * has just got a big task running since a long sleep period. At the same time + * however it preserves the benefits of the "blocked utilization" in + * describing the potential for other tasks waking up on the same CPU. + * + * Return: the estimated utilization for the specified CPU + */ +static inline unsigned long cpu_util_est(int cpu) +{ + unsigned long util, util_est; + unsigned long capacity; + struct cfs_rq *cfs_rq; + + if (!sched_feat(UTIL_EST)) + return cpu_util(cpu); + + cfs_rq = &cpu_rq(cpu)->cfs; + util = cfs_rq->avg.util_avg; + util_est = cfs_rq->util_est_runnable; + util_est = max(util, util_est); + + capacity = capacity_orig_of(cpu); + util_est = min(util_est, capacity); + + return util_est; +} + static inline unsigned long task_util(struct task_struct *p) { return p->se.avg.util_avg; @@ -6348,16 +6383,43 @@ static inline unsigned long task_util_est(struct task_struct *p) */ static unsigned long cpu_util_wake(int cpu, struct task_struct *p) { - unsigned long util, capacity; + long util, util_est; /* Task has no contribution or is new */ if (cpu != task_cpu(p) || !p->se.avg.last_update_time) - return cpu_util(cpu); + return cpu_util_est(cpu); - capacity = capacity_orig_of(cpu); - util = max_t(long, cpu_rq(cpu)->cfs.avg.util_avg - task_util(p), 0); + /* Discount task's blocked util from CPU's util */ + util = cpu_util(cpu) - task_util(p); + util = max(util, 0L); - return (util >= capacity) ? capacity : util; + if (!sched_feat(UTIL_EST)) + return util; + + /* + * These are the main cases covered: + * - if *p is the only task sleeping on this CPU, then: + * cpu_util (== task_util) > util_est (== 0) + * and thus we return: + * cpu_util_wake = (cpu_util - task_util) = 0 + * + * - if other tasks are SLEEPING on the same CPU, which is just waking + * up, then: + * cpu_util >= task_util + * cpu_util > util_est (== 0) + * and thus we discount *p's blocked utilization to return: + * cpu_util_wake = (cpu_util - task_util) >= 0 + * + * - if other tasks are RUNNABLE on that CPU and + * util_est > cpu_util + * then we use util_est since it returns a more restrictive + * estimation of the spare capacity on that CPU, by just considering + * the expected utilization of tasks already runnable on that CPU. + */ + util_est = cpu_rq(cpu)->cfs.util_est_runnable; + util = max(util, util_est); + + return util; } /* @@ -7882,7 +7944,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, load = source_load(i, load_idx); sgs->group_load += load; - sgs->group_util += cpu_util(i); + sgs->group_util += cpu_util_est(i); sgs->sum_nr_running += rq->cfs.h_nr_running; nr_running = rq->nr_running; -- 2.15.1