Received: by 2002:ac0:a5a7:0:0:0:0:0 with SMTP id m36-v6csp2074434imm; Mon, 16 Jul 2018 01:30:57 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfpDZuR8FqsmiOLpohramBSysZs/k9LV0Vpj+2XM6utZX7D0OhVZprf3Mzmp+Ln3mA/Va9z X-Received: by 2002:a62:6eca:: with SMTP id j193-v6mr17208127pfc.256.1531729857218; Mon, 16 Jul 2018 01:30:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531729857; cv=none; d=google.com; s=arc-20160816; b=E8fC6U19o05rRumOhDCGZe8L4p7HnlZpNL5+0DUXYlX91dv/1p6pdNLiPQKm/reeZU aBCuxHUDaPuK+aytqakTnmmS+Bc03DrvSOJPT3vrZNVWVqpnKfRXClXxsBuN6lmTSoLu L8dYT1w6bJUpbQJkA5W1fsEFs30FM/VU0Bg0eIAPLYEhFCjyJFlSA/TgsUC8kL8Dc/pU oX8P3QNiJlFb8/+GrVEUF3OqgNJEpBtOCzVWKSwOIp8LglUH1DP0nE4svg/IBGxXLZBD +Z/5+JRSzWJwVfyzr9rco7xA2JatZwF4s/FmSh00Zifq2XiDSTrHbvG80W+2Aqtx5AWa KPzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=O37+pxPnGtx5xzFUe/doR5Xo+yqaw3rxy6ito4uEzCo=; b=rnQnoRiWgXt201r8s5sUZeg4Tr241RwIz9F4pgyt4vkGqnHFLTKpc7cscIrF6DMlCa T43ZR22JsIOQmwfdaW31ivJJ98Sd8Wueatw97uyiCJ7ViA9TNS+BAsqbWZH9Sw9giAP0 2HqpPfytrKU3DBk64D0E/pqpcCyAi0trk9XXBOpLYcJfJT3To7LQpVRi6BQ92xg7HcG8 ni5bNB2rDo28q2awPDnDMf4Rqip7iU98W6yPaPc0CEpHJzoKOk2l1YxzscbJ6oiVRlL+ yxbVenoU71N3NdWKctOP7az+IdgenEZyZTW21QExB5iIUg3MGYt+ObQxrhz/JzNGZmZs /3Dg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s1-v6si2752397pfi.369.2018.07.16.01.30.41; Mon, 16 Jul 2018 01:30:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731372AbeGPIz7 (ORCPT + 99 others); Mon, 16 Jul 2018 04:55:59 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:54370 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730828AbeGPIz6 (ORCPT ); Mon, 16 Jul 2018 04:55:58 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ABB18ED1; Mon, 16 Jul 2018 01:29:43 -0700 (PDT) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DF3B23F5A0; Mon, 16 Jul 2018 01:29:40 -0700 (PDT) From: Patrick Bellasi To: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , Tejun Heo , "Rafael J . Wysocki" , Viresh Kumar , Vincent Guittot , Paul Turner , Dietmar Eggemann , Morten Rasmussen , Juri Lelli , Todd Kjos , Joel Fernandes , Steve Muckle , Suren Baghdasaryan Subject: [PATCH v2 05/12] sched/cpufreq: uclamp: add utilization clamping for FAIR tasks Date: Mon, 16 Jul 2018 09:28:59 +0100 Message-Id: <20180716082906.6061-6-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180716082906.6061-1-patrick.bellasi@arm.com> References: <20180716082906.6061-1-patrick.bellasi@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Each time a frequency update is required via schedutil, a frequency is selected to (possibly) satisfy the utilization reported by the CFS class. However, when utilization clamping is in use, the frequency selection should consider the requirements suggested by userspace, for example, to: - boost tasks which are directly affecting the user experience by running them at least at a minimum "required" frequency - cap low priority tasks not directly affecting the user experience by running them only up to a maximum "allowed" frequency These constraints are meant to support a per-task based tuning of the frequency selection thus allowing to have a fine grained definition of performance boosting vs energy saving strategies in kernel space. Let's add the required support to clamp the utilization generated by FAIR tasks within the boundaries defined by their aggregated utilization clamp constraints. On each CPU the aggregated clamp values are obtained by considering the maximum of the {min,max}_util values for each task. This max aggregation responds to the goal of not penalizing, for example, high boosted (i.e. more important for the user-experience) CFS tasks which happens to be co-scheduled with high capped (i.e. less important for the user-experience) CFS tasks. For FAIR tasks both the utilization as well as the IOWait boost values are clamped according to the CPU aggregated utilization clamp constraints. The default values for boosting and capping are defined to be: - util_min: 0 - util_max: SCHED_CAPACITY_SCALE which means that by default no boosting/capping is enforced on FAIR tasks, and thus the frequency will be selected considering the actual utilization value of each CPU. Signed-off-by: Patrick Bellasi Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Viresh Kumar Cc: Todd Kjos Cc: Joel Fernandes Cc: Juri Lelli Cc: Dietmar Eggemann Cc: Morten Rasmussen Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- kernel/sched/cpufreq_schedutil.c | 21 ++++++++-- kernel/sched/sched.h | 71 ++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+), 3 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index c907fde01eaa..70aea6ec3c08 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -191,6 +191,7 @@ static void sugov_get_util(struct sugov_cpu *sg_cpu) static unsigned long sugov_aggregate_util(struct sugov_cpu *sg_cpu) { struct rq *rq = cpu_rq(sg_cpu->cpu); + unsigned long util_cfs; if (rt_rq_is_runnable(&rq->rt)) return sg_cpu->max; @@ -205,7 +206,9 @@ static unsigned long sugov_aggregate_util(struct sugov_cpu *sg_cpu) * util_cfs + util_dl as requested freq. However, cpufreq is not yet * ready for such an interface. So, we only do the latter for now. */ - return min(sg_cpu->max, (sg_cpu->util_dl + sg_cpu->util_cfs)); + util_cfs = uclamp_util(sg_cpu->cpu, sg_cpu->util_cfs); + + return min(sg_cpu->max, (sg_cpu->util_dl + util_cfs)); } /** @@ -252,6 +255,7 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, u64 time, unsigned int flags) { bool set_iowait_boost = flags & SCHED_CPUFREQ_IOWAIT; + unsigned int max_boost; /* Reset boost if the CPU appears to have been idle enough */ if (sg_cpu->iowait_boost && @@ -267,11 +271,22 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, u64 time, return; sg_cpu->iowait_boost_pending = true; + /* + * Boost FAIR tasks only up to the CPU clamped utilization. + * + * Since DL tasks have a much more advanced bandwidth control, it's + * safe to assume that IO boost does not apply to those tasks. + * Instead, since for RT tasks we are already going to max, we don't + * rally care about clamping the IO boost max value for them too. + */ + max_boost = sg_cpu->iowait_boost_max; + max_boost = uclamp_util(sg_cpu->cpu, max_boost); + /* Double the boost at each request */ if (sg_cpu->iowait_boost) { sg_cpu->iowait_boost <<= 1; - if (sg_cpu->iowait_boost > sg_cpu->iowait_boost_max) - sg_cpu->iowait_boost = sg_cpu->iowait_boost_max; + if (sg_cpu->iowait_boost > max_boost) + sg_cpu->iowait_boost = max_boost; return; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b471d2222410..1207add36478 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2284,6 +2284,77 @@ static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) {} #endif /* CONFIG_CPU_FREQ */ +/** + * uclamp_none: default value for a clamp + * + * This returns the default value for each clamp + * - 0 for a min utilization clamp + * - SCHED_CAPACITY_SCALE for a max utilization clamp + * + * Return: the default value for a given utilization clamp + */ +static inline unsigned int uclamp_none(int clamp_id) +{ + if (clamp_id == UCLAMP_MIN) + return 0; + return SCHED_CAPACITY_SCALE; +} + +#ifdef CONFIG_UCLAMP_TASK +/** + * uclamp_value: get the current CPU's utilization clamp value + * @cpu: the CPU to consider + * @clamp_id: the utilization clamp index (i.e. min or max utilization) + * + * The utilization clamp value for a CPU depends on its set of currently + * RUNNABLE tasks and their specific util_{min,max} constraints. + * A max aggregated value is tracked for each CPU and returned by this + * function. + * + * Return: the current value for the specified CPU and clamp index + */ +static inline unsigned int uclamp_value(unsigned int cpu, int clamp_id) +{ + struct uclamp_cpu *uc_cpu = &cpu_rq(cpu)->uclamp; + + if (uc_cpu->value[clamp_id] == UCLAMP_NONE) + return uclamp_none(clamp_id); + + return uc_cpu->value[clamp_id]; +} + +/** + * clamp_util: clamp a utilization value for a specified CPU + * @cpu: the CPU to get the clamp values from + * @util: the utilization signal to clamp + * + * Each CPU tracks util_{min,max} clamp values depending on the set of its + * currently RUNNABLE tasks. Given a utilization signal, i.e a signal in + * the [0..SCHED_CAPACITY_SCALE] range, this function returns a clamped + * utilization signal considering the current clamp values for the + * specified CPU. + * + * Return: a clamped utilization signal for a given CPU. + */ +static inline unsigned int uclamp_util(unsigned int cpu, unsigned int util) +{ + unsigned int min_util = uclamp_value(cpu, UCLAMP_MIN); + unsigned int max_util = uclamp_value(cpu, UCLAMP_MAX); + + return clamp(util, min_util, max_util); +} +#else /* CONFIG_UCLAMP_TASK */ +static inline unsigned int uclamp_value(unsigned int cpu, int clamp_id) +{ + return uclamp_none(clamp_id); +} + +static inline unsigned int uclamp_util(unsigned int cpu, unsigned int util) +{ + return util; +} +#endif /* CONFIG_UCLAMP_TASK */ + #ifdef arch_scale_freq_capacity # ifndef arch_scale_freq_invariant # define arch_scale_freq_invariant() true -- 2.17.1