Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp414399pxb; Tue, 19 Oct 2021 05:38:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxo5Q83VDKA/iLGh3xSr9nL8LcBKcq1Zg9Dv/mG6zwg7ebmTZTBMCC6TXiAPWJAnSkaa1o/ X-Received: by 2002:a17:906:9742:: with SMTP id o2mr38050865ejy.532.1634647099064; Tue, 19 Oct 2021 05:38:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634647099; cv=none; d=google.com; s=arc-20160816; b=AaCfKEEy03U7siJCnlVUhcTUl1KSy4erh1H2IrRL8yGFjY1NqjTMZ7Q/ybJ2n/EQ52 /HUbn2fcdMdyApIcDYCuYSgHvNNgio0eMdBF7scp6WOgjGR2R6ukMEiJAZJUV6jnT6LX Vs8NQpp6P1mRfvL0kuD373myzvC+upJYtpIBUZhadxqg+h3vFdx7AWB98vTUxF0MMq0i 5uizpXGMTj4UdzzcjnA2ImT4j5T9Y9VQvslcY29Bfp6czT9lQargmpmUmQkWWpt8f8z/ xfCprLritoQBhupx3RN2tOC3L/x5/pgN1vncWKdZX+i+kwISerW6uWSlkEhMKvm87QDX 7eMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=6CK6/+q8aXew5IcuMz3Fih447x496rJhBuuR5ERVtjA=; b=WR7tGzQKYX8VGXFf9spIyxa27ddMKr094A4gLIPLJK0n0hvrPO8s4ZuIMqa0MHR36O lZk19p4rOOWekmBBn7QfRYF/aVM0As3YqwWyLtkqTEcNwoG35ZhfPCEqrvCflBc22iGT LWbiLR8v8fSBrjYrytCq3YlsxRvgghgJ4N0nk3RP1mb7IIheb8fRKlhpTyGgVr+TCeDC gg8EU69q7AMTxhg5grAs4ucyHfzSF2JEQ+/Mkpo3Gqr9vyMm6wPAGTYP6FZDVuuwESWf uBlKMH2ktxFq330jSpVr68bBbTa0QhK2c6qQkgm63m+a7fiWse/v+Q5gXUGsavcCp8zs KkVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="cM1lXfg/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id c5si23716089ejb.396.2021.10.19.05.37.55; Tue, 19 Oct 2021 05:38:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="cM1lXfg/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235699AbhJSMiD (ORCPT + 99 others); Tue, 19 Oct 2021 08:38:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235692AbhJSMiC (ORCPT ); Tue, 19 Oct 2021 08:38:02 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F22A0C06161C for ; Tue, 19 Oct 2021 05:35:48 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id i12so47374171wrb.7 for ; Tue, 19 Oct 2021 05:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6CK6/+q8aXew5IcuMz3Fih447x496rJhBuuR5ERVtjA=; b=cM1lXfg/BqvlHfxKBgTeMk2o7A1TNb9l1hrJdj7WsrlhIR9Se5nK6Kej6fMlponRvJ GeViIauxTio4qYLbGPTLiJsZqOflSk+CRXoFuzpIdFk3y52A5YFjAYov6XKthYyTAxvv 35Af7SjKhni/3PllPbdQGcT0OV/gGO4rzmasLpGTdmcfCuc94y5POodStoOPCOV1tIs7 8SeteFpW6AgRvqaQubxduuJZq1eQsplvOxck8LEMGE703inHLsrRA0ubKum2LVYtbhzT +gJLJASM3tFsnFPg3t6YviL3ghYsSG9YPyvTZeipD8YwlSFzeLIgMlgZazUszwEstGCl W+2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6CK6/+q8aXew5IcuMz3Fih447x496rJhBuuR5ERVtjA=; b=npWyD793zSOzwh8uA7USWdlpuGQkDuhwiRm4nL3FP5r0s21ukAU6gvLpQcb233GT2h NKh0eNyYVOvmTwTig30Q0U+lPyXhXkGRm+kBDlR2GW9F9S3cmtZ16PqO3RZPtBBZDa/j UCedowD40xBfYjDc4PIN1vp328HsNrWORo5AO6JzL6lGpk+5rcMBjYzbyqVpdv11bG4c YJJ+LdChpJmxwfaPfl9R1sqBwUKzYwRyqeTh8SmnN+/tRfaRllMPu64I5KgplYOdSXdO jIHvRCmbVY4s8QaR0/jldssyUs4bbYdjmxh4q8SqSIXuzv8V9H6fpkPtTXBUaBf/SvfH FnKw== X-Gm-Message-State: AOAM532caVmqFhU6aoDnm92AJ3gwSrOerqV7/xMeh2/BHw6e/vFsv7Mr BwT1oN4JbTLHHej4vplVgv3Pfg== X-Received: by 2002:adf:f812:: with SMTP id s18mr6562620wrp.347.1634646947552; Tue, 19 Oct 2021 05:35:47 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:6dc7:9883:a79c:5be9]) by smtp.gmail.com with ESMTPSA id p18sm15263683wrn.41.2021.10.19.05.35.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 05:35:46 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, linux-kernel@vger.kernel.org, tim.c.chen@linux.intel.com Cc: Vincent Guittot Subject: [PATCH 3/5] sched/fair: Wait before decaying max_newidle_lb_cost Date: Tue, 19 Oct 2021 14:35:35 +0200 Message-Id: <20211019123537.17146-4-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211019123537.17146-1-vincent.guittot@linaro.org> References: <20211019123537.17146-1-vincent.guittot@linaro.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Decay max_newidle_lb_cost only when it has not been updated for a while and ensure to not decay a recently changed value. Signed-off-by: Vincent Guittot --- include/linux/sched/topology.h | 2 +- kernel/sched/fair.c | 36 +++++++++++++++++++++++++--------- kernel/sched/topology.c | 2 +- 3 files changed, 29 insertions(+), 11 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 2f9166f6dec8..c07bfa2d80f2 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -105,7 +105,7 @@ struct sched_domain { /* idle_balance() stats */ u64 max_newidle_lb_cost; - unsigned long next_decay_max_lb_cost; + unsigned long last_decay_max_lb_cost; u64 avg_scan_cost; /* select_idle_sibling */ diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c4c36865321b..e50fd751e1df 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10239,6 +10239,30 @@ void update_max_interval(void) max_load_balance_interval = HZ*num_online_cpus()/10; } +static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) +{ + if (cost > sd->max_newidle_lb_cost) { + /* + * Track max cost of a domain to make sure to not delay the + * next wakeup on the CPU. + */ + sd->max_newidle_lb_cost = cost; + sd->last_decay_max_lb_cost = jiffies; + } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { + /* + * Decay the newidle max times by ~1% per second to ensure that + * it is not outdated and the current max cost is actually + * shorter. + */ + sd->max_newidle_lb_cost = (sd->max_newidle_lb_cost * 253) / 256; + sd->last_decay_max_lb_cost = jiffies; + + return true; + } + + return false; +} + /* * It checks each scheduling domain to see if it is due to be balanced, * and initiates a balancing operation if so. @@ -10262,14 +10286,9 @@ static void rebalance_domains(struct rq *rq, enum cpu_idle_type idle) for_each_domain(cpu, sd) { /* * Decay the newidle max times here because this is a regular - * visit to all the domains. Decay ~1% per second. + * visit to all the domains. */ - if (time_after(jiffies, sd->next_decay_max_lb_cost)) { - sd->max_newidle_lb_cost = - (sd->max_newidle_lb_cost * 253) / 256; - sd->next_decay_max_lb_cost = jiffies + HZ; - need_decay = 1; - } + need_decay = update_newidle_cost(sd, 0); max_cost += sd->max_newidle_lb_cost; /* @@ -10911,8 +10930,7 @@ static int newidle_balance(struct rq *this_rq, struct rq_flags *rf) t1 = sched_clock_cpu(this_cpu); domain_cost = t1 - t0; - if (domain_cost > sd->max_newidle_lb_cost) - sd->max_newidle_lb_cost = domain_cost; + update_newidle_cost(sd, domain_cost); curr_cost += domain_cost; t0 = t1; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index e81246787560..30169c7685b6 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1568,7 +1568,7 @@ sd_init(struct sched_domain_topology_level *tl, .last_balance = jiffies, .balance_interval = sd_weight, .max_newidle_lb_cost = 0, - .next_decay_max_lb_cost = jiffies, + .last_decay_max_lb_cost = jiffies, .child = child, #ifdef CONFIG_SCHED_DEBUG .name = tl->name, -- 2.17.1