Received: by 2002:a25:1985:0:0:0:0:0 with SMTP id 127csp2650382ybz; Mon, 27 Apr 2020 01:40:58 -0700 (PDT) X-Google-Smtp-Source: APiQypKMN1UGzFpynrts7j6tADJfmUU8LCBH9X6ooA3JNeNK+Wg+BpLbxYIh0ZkcAFeZpqwhFjoX X-Received: by 2002:a17:906:4003:: with SMTP id v3mr18259140ejj.144.1587976858736; Mon, 27 Apr 2020 01:40:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587976858; cv=none; d=google.com; s=arc-20160816; b=aiRbCN3jHmayiHUPdYZK4pt/j2WSzOFXq5FjybxtL3xiRZefmsKQ/hHq9yHpdX00/F TCcj6r2zwObxYGNgJ/Ee3WWCUYHfJgFCf5WGTE58y/XWLfPB1GzGCG4fJnOJj+T3aPRd FUBs0JN+o2Ie6I5I+i+J0Dfw6vPrq89xbmerSIrhNsSU5oL4cm1ZH53TTNwmqgmEyBC4 o7Bq9y4DZnqNcIAwkum3wcUjOPPQZE4zIrHvHU3/6W7z3fiICFTRGkf7ZAvs1NcIJU+Q zG2lNauo3qtBki7Pe1xArxE0pSrjXI56d0IGWwR2aZiYHDmqgrvb+Msc9LIxhurk71qr n0yA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=ICyMwHNB6LiCGWmGiRmT/exFxeXSxMqBEf7QWs5pExc=; b=CpgomWSuT0yXvvJ+YT4gJpmxjbnN3ZUPItWhPwtc0KYti2ZY7VpKCoPtrXLpM4UVPU RC1zdsXCm64TjItceBeYBjZE1lr2nSnmWxSQ69xl2R/Cb0TqWT7SZs2fzxCzf0yBZNvx f5s6YdriUVys2rL3HrDWFpI9hcpBRD+gmMvEhN1AhjqIBl2rSotLoTYXrdSYf0Y5s1xQ IVmA5aXhmT1OyKHXDe3Z8ITPrDhuK7j8pJmugrwNLjjFe78SdA2+e/RoLewTs2nBw1ol 1FjNpJAzWXIxuBdh1fE7Gfjs9e3YJwfSPZUyAzBGfzILGIwFlc2aCqAll2KNy41qNXwr hccg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h15si7068919edv.341.2020.04.27.01.40.35; Mon, 27 Apr 2020 01:40:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726860AbgD0Ihw (ORCPT + 99 others); Mon, 27 Apr 2020 04:37:52 -0400 Received: from foss.arm.com ([217.140.110.172]:59966 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725899AbgD0Ihw (ORCPT ); Mon, 27 Apr 2020 04:37:52 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 47D6B11D4; Mon, 27 Apr 2020 01:37:51 -0700 (PDT) Received: from dell3630.arm.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id ECBFF3F68F; Mon, 27 Apr 2020 01:37:47 -0700 (PDT) From: Dietmar Eggemann To: Ingo Molnar , Peter Zijlstra , Juri Lelli Cc: Vincent Guittot , Steven Rostedt , Luca Abeni , Daniel Bristot de Oliveira , Wei Wang , Quentin Perret , Alessio Balsini , Pavan Kondeti , Patrick Bellasi , Morten Rasmussen , Valentin Schneider , Qais Yousef , linux-kernel@vger.kernel.org Subject: [PATCH v2 5/6] sched/deadline: Make DL capacity-aware Date: Mon, 27 Apr 2020 10:37:08 +0200 Message-Id: <20200427083709.30262-6-dietmar.eggemann@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200427083709.30262-1-dietmar.eggemann@arm.com> References: <20200427083709.30262-1-dietmar.eggemann@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Luca Abeni The current SCHED_DEADLINE (DL) scheduler uses a global EDF scheduling algorithm w/o considering CPU capacity or task utilization. This works well on homogeneous systems where DL tasks are guaranteed to have a bounded tardiness but presents issues on heterogeneous systems. A DL task can migrate to a CPU which does not have enough CPU capacity to correctly serve the task (e.g. a task w/ 70ms runtime and 100ms period on a CPU w/ 512 capacity). Add the DL fitness function dl_task_fits_capacity() for DL admission control on heterogeneous systems. A task fits onto a CPU if: CPU original capacity / 1024 >= task runtime / task deadline Use this function on heterogeneous systems to try to find a CPU which meets this criterion during task wakeup, push and offline migration. On homogeneous systems the original behavior of the DL admission control should be retained. Signed-off-by: Luca Abeni Signed-off-by: Dietmar Eggemann --- kernel/sched/cpudeadline.c | 14 +++++++++++++- kernel/sched/deadline.c | 18 ++++++++++++++---- kernel/sched/sched.h | 15 +++++++++++++++ 3 files changed, 42 insertions(+), 5 deletions(-) diff --git a/kernel/sched/cpudeadline.c b/kernel/sched/cpudeadline.c index 5cc4012572ec..8630f2a40a3f 100644 --- a/kernel/sched/cpudeadline.c +++ b/kernel/sched/cpudeadline.c @@ -121,7 +121,19 @@ int cpudl_find(struct cpudl *cp, struct task_struct *p, if (later_mask && cpumask_and(later_mask, cp->free_cpus, p->cpus_ptr)) { - return 1; + int cpu; + + if (!static_branch_unlikely(&sched_asym_cpucapacity)) + return 1; + + /* Ensure the capacity of the CPUs fits the task. */ + for_each_cpu(cpu, later_mask) { + if (!dl_task_fits_capacity(p, cpu)) + cpumask_clear_cpu(cpu, later_mask); + } + + if (!cpumask_empty(later_mask)) + return 1; } else { int best_cpu = cpudl_maximum(cp); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 08ab28e1cefc..575b7d88d839 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1634,6 +1634,7 @@ static int select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags) { struct task_struct *curr; + bool select_rq; struct rq *rq; if (sd_flag != SD_BALANCE_WAKE) @@ -1653,10 +1654,19 @@ select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags) * other hand, if it has a shorter deadline, we * try to make it stay here, it might be important. */ - if (unlikely(dl_task(curr)) && - (curr->nr_cpus_allowed < 2 || - !dl_entity_preempt(&p->dl, &curr->dl)) && - (p->nr_cpus_allowed > 1)) { + select_rq = unlikely(dl_task(curr)) && + (curr->nr_cpus_allowed < 2 || + !dl_entity_preempt(&p->dl, &curr->dl)) && + p->nr_cpus_allowed > 1; + + /* + * Take the capacity of the CPU into account to + * ensure it fits the requirement of the task. + */ + if (static_branch_unlikely(&sched_asym_cpucapacity)) + select_rq |= !dl_task_fits_capacity(p, cpu); + + if (select_rq) { int target = find_later_rq(p); if (target != -1 && diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 511edacc2282..ec0efd99497b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -317,6 +317,21 @@ static inline bool __dl_overflow(struct dl_bw *dl_b, unsigned long cap, cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw; } +/* + * Verify the fitness of task @p to run on @cpu taking into account the + * CPU original capacity and the runtime/deadline ratio of the task. + * + * The function will return true if the CPU original capacity of the + * @cpu scaled by SCHED_CAPACITY_SCALE >= runtime/deadline ratio of the + * task and false otherwise. + */ +static inline bool dl_task_fits_capacity(struct task_struct *p, int cpu) +{ + unsigned long cap = arch_scale_cpu_capacity(cpu); + + return cap_scale(p->dl.dl_deadline, cap) >= p->dl.dl_runtime; +} + extern void init_dl_bw(struct dl_bw *dl_b); extern int sched_dl_global_validate(void); extern void sched_dl_do_global(void); -- 2.17.1