Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp2337059imm; Fri, 7 Sep 2018 14:56:11 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaXMgDa0xMztCk6UCX6VBvl5qKbMc1DjvQ0lTweWF6/AvlZ7Eihe8sdOOnzrdYKBeQCPX/J X-Received: by 2002:a62:571b:: with SMTP id l27-v6mr10753973pfb.29.1536357371242; Fri, 07 Sep 2018 14:56:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536357371; cv=none; d=google.com; s=arc-20160816; b=ToW/QWkV5C73VQnMNhPSXoN/wUZjhUWBSHLse2uyrywf03OVXG75dQA8NSzpv08Viz UkS95cQpR5p56hYBQw5bkUr4gLjrYDwTL/XQGTcFud5QULIwRcAHXOqXTlMFBF12RTll Bibp4vwj8hUET0f/03tAnC/W/SvGrjWZSjwWOWWBelUsPwv0RAaTsc5uTDNOADRDMnzK U2mjKuet+bgyYe64sEtQmRjLjyR8jMhz5zNCmBt/uth428kJVfnZrGZgiqVv4L3iGMnK i1zKwhvaViKNAKjeyehZzFvhev5vmhjKB9C8m6/8oxaO0W20LXN+JX6d8PCiEYjf5LS4 LKHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=sJvlh8HzNYgqt17xwqY0KkguBDWUGUbjLs2d1zA4ih8=; b=MlkzJ2GbgGKR7NtMVzSo1PQVNmGucox0/79V1TOT4OMNhtu+kzAPiJGCCG0huberhD 2Nei3oDeunhw7J4C61Os/Ypw8ubSmTpaejjC4cq+0j4gkEk8eHhORShtWvE+8FbZR6tI BcHxIeGzCsg9Nlv+35H4kGVXN91Ve/SUawLMkup0vu44RxgCrJOAKh5gg0HOyubGbgk2 TqftWelkXze0vlQ0gapBuIi1v+bWvhfiUSZmS68vrPG8hgu5Rd8QwgjiHVxwOev098iV jn6FB1a43fwJnF/+42kFqJ+R+bbCh48/I9sBlw3le6L6T0ghHIcj9tJ1v0RW7mQGwOnz qAAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amazon.de header.s=amazon201209 header.b=QsbWsNdm; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.de Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d39-v6si9535255pla.41.2018.09.07.14.55.55; Fri, 07 Sep 2018 14:56:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@amazon.de header.s=amazon201209 header.b=QsbWsNdm; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730278AbeIHCYP (ORCPT + 99 others); Fri, 7 Sep 2018 22:24:15 -0400 Received: from smtp-fw-6001.amazon.com ([52.95.48.154]:60236 "EHLO smtp-fw-6001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729080AbeIHCYP (ORCPT ); Fri, 7 Sep 2018 22:24:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1536356480; x=1567892480; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sJvlh8HzNYgqt17xwqY0KkguBDWUGUbjLs2d1zA4ih8=; b=QsbWsNdmezvySfHAJrix0MIaX0VHTId2q8GD/9RdkF3IRES3hzuMyep8 Wxazt2we7KPG9IgIHdi3GsLnnn4mNzQgU+GIlYA/3wTHM3pkoslJ1wp2m blhhJ8Q2ZF/w8dIaeJVEHcZNnWUVCa5D2mNqjSIYxACPLEY8wwYFh/cDm s=; X-IronPort-AV: E=Sophos;i="5.53,343,1531785600"; d="scan'208";a="355980779" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-579b7f5b.us-west-2.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6001.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 07 Sep 2018 21:41:18 +0000 Received: from u7588a65da6b65f.ant.amazon.com (pdx2-ws-svc-lb17-vlan2.amazon.com [10.247.140.66]) by email-inbound-relay-2c-579b7f5b.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w87LfDHF130618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 7 Sep 2018 21:41:15 GMT Received: from u7588a65da6b65f.ant.amazon.com (localhost [127.0.0.1]) by u7588a65da6b65f.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTPS id w87LfBSv027102 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 7 Sep 2018 23:41:11 +0200 Received: (from jschoenh@localhost) by u7588a65da6b65f.ant.amazon.com (8.15.2/8.15.2/Submit) id w87Lf9Ge027097; Fri, 7 Sep 2018 23:41:09 +0200 From: =?UTF-8?q?Jan=20H=2E=20Sch=C3=B6nherr?= To: Ingo Molnar , Peter Zijlstra Cc: =?UTF-8?q?Jan=20H=2E=20Sch=C3=B6nherr?= , linux-kernel@vger.kernel.org Subject: [RFC 02/60] sched: Introduce set_entity_cfs() to place a SE into a certain CFS runqueue Date: Fri, 7 Sep 2018 23:39:49 +0200 Message-Id: <20180907214047.26914-3-jschoenh@amazon.de> X-Mailer: git-send-email 2.9.3.1.gcba166c.dirty In-Reply-To: <20180907214047.26914-1-jschoenh@amazon.de> References: <20180907214047.26914-1-jschoenh@amazon.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Factor out the logic to place a SE into a CFS runqueue into its own function. This consolidates various sprinkled updates of se->cfs_rq, se->parent, and se->depth at the cost of updating se->depth unnecessarily on same-group movements between CPUs. Signed-off-by: Jan H. Schönherr --- kernel/sched/fair.c | 26 ++++---------------------- kernel/sched/sched.h | 12 +++++++++--- 2 files changed, 13 insertions(+), 25 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 638fd14bb6c4..3de0158729a6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9683,14 +9683,6 @@ static void attach_entity_cfs_rq(struct sched_entity *se) { struct cfs_rq *cfs_rq = cfs_rq_of(se); -#ifdef CONFIG_FAIR_GROUP_SCHED - /* - * Since the real-depth could have been changed (only FAIR - * class maintain depth value), reset depth properly. - */ - se->depth = se->parent ? se->parent->depth + 1 : 0; -#endif - /* Synchronize entity with its cfs_rq */ update_load_avg(cfs_rq, se, sched_feat(ATTACH_AGE_LOAD) ? 0 : SKIP_AGE_LOAD); attach_entity_load_avg(cfs_rq, se, 0); @@ -9781,10 +9773,7 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) #ifdef CONFIG_FAIR_GROUP_SCHED static void task_set_group_fair(struct task_struct *p) { - struct sched_entity *se = &p->se; - set_task_rq(p, task_cpu(p)); - se->depth = se->parent ? se->parent->depth + 1 : 0; } static void task_move_group_fair(struct task_struct *p) @@ -9855,7 +9844,7 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent) goto err_free_rq; init_cfs_rq(cfs_rq); - init_tg_cfs_entry(tg, cfs_rq, se, i, parent->cfs_rq[i]->my_se); + init_tg_cfs_entry(tg, cfs_rq, se, i, parent->cfs_rq[i]); init_entity_runnable_average(se); } @@ -9912,7 +9901,7 @@ void unregister_fair_sched_group(struct task_group *tg) void init_tg_cfs_entry(struct task_group *tg, struct cfs_rq *cfs_rq, struct sched_entity *se, int cpu, - struct sched_entity *parent) + struct cfs_rq *parent) { struct rq *rq = cpu_rq(cpu); @@ -9927,18 +9916,11 @@ void init_tg_cfs_entry(struct task_group *tg, struct cfs_rq *cfs_rq, if (!se) return; - if (!parent) { - se->cfs_rq = &rq->cfs; - se->depth = 0; - } else { - se->cfs_rq = parent->my_q; - se->depth = parent->depth + 1; - } - + set_entity_cfs(se, parent); se->my_q = cfs_rq; + /* guarantee group entities always have weight */ update_load_set(&se->load, NICE_0_LOAD); - se->parent = parent; } static DEFINE_MUTEX(shares_mutex); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8435bf70a701..b4d0e8a68697 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -428,7 +428,7 @@ extern void online_fair_sched_group(struct task_group *tg); extern void unregister_fair_sched_group(struct task_group *tg); extern void init_tg_cfs_entry(struct task_group *tg, struct cfs_rq *cfs_rq, struct sched_entity *se, int cpu, - struct sched_entity *parent); + struct cfs_rq *parent); extern void init_cfs_bandwidth(struct cfs_bandwidth *cfs_b); extern void __refill_cfs_bandwidth_runtime(struct cfs_bandwidth *cfs_b); @@ -1290,6 +1290,13 @@ static inline struct task_group *task_group(struct task_struct *p) return p->sched_task_group; } +static inline void set_entity_cfs(struct sched_entity *se, struct cfs_rq *cfs_rq) +{ + se->cfs_rq = cfs_rq; + se->parent = cfs_rq->my_se; + se->depth = se->parent ? se->parent->depth + 1 : 0; +} + /* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */ static inline void set_task_rq(struct task_struct *p, unsigned int cpu) { @@ -1299,8 +1306,7 @@ static inline void set_task_rq(struct task_struct *p, unsigned int cpu) #ifdef CONFIG_FAIR_GROUP_SCHED set_task_rq_fair(&p->se, p->se.cfs_rq, tg->cfs_rq[cpu]); - p->se.cfs_rq = tg->cfs_rq[cpu]; - p->se.parent = tg->cfs_rq[cpu]->my_se; + set_entity_cfs(&p->se, tg->cfs_rq[cpu]); #endif #ifdef CONFIG_RT_GROUP_SCHED -- 2.9.3.1.gcba166c.dirty