Date: Mon, 20 Jul 2015 16:38:26 +0100
From: Morten Rasmussen <morten.rasmussen@arm.com>
To: Sai Gurrappadi <sgurrappadi@nvidia.com>
Cc: peterz@infradead.org, mingo@redhat.com, vincent.guittot@linaro.org,
        daniel.lezcano@linaro.org, Dietmar Eggemann <Dietmar.Eggemann@arm.com>,
        yuyang.du@intel.com, mturquette@baylibre.com, rjw@rjwysocki.net,
        Juri Lelli <Juri.Lelli@arm.com>, pang.xunlei@zte.com.cn,
        linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org,
        pboonstoppel@nvidia.com
Subject: Re: [RFCv5 PATCH 32/46] sched: Energy-aware wake-up task placement
Message-ID: <20150720153826.GA15088@e105550-lin.cambridge.arm.com>
References: <1436293469-25707-1-git-send-email-morten.rasmussen@arm.com>
 <1436293469-25707-33-git-send-email-morten.rasmussen@arm.com>
 <55A8480C.6080806@nvidia.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <55A8480C.6080806@nvidia.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3115
Lines: 83

On Thu, Jul 16, 2015 at 05:10:52PM -0700, Sai Gurrappadi wrote:
> Hi Morten,
> 
> On 07/07/2015 11:24 AM, Morten Rasmussen wrote:
> > ---
> 
> > +static int energy_aware_wake_cpu(struct task_struct *p, int target)
> > +{
> > +	struct sched_domain *sd;
> > +	struct sched_group *sg, *sg_target;
> > +	int target_max_cap = INT_MAX;
> > +	int target_cpu = task_cpu(p);
> > +	int i;
> > +
> > +	sd = rcu_dereference(per_cpu(sd_ea, task_cpu(p)));
> > +
> > +	if (!sd)
> > +		return target;
> > +
> > +	sg = sd->groups;
> > +	sg_target = sg;
> > +
> > +	/*
> > +	 * Find group with sufficient capacity. We only get here if no cpu is
> > +	 * overutilized. We may end up overutilizing a cpu by adding the task,
> > +	 * but that should not be any worse than select_idle_sibling().
> > +	 * load_balance() should sort it out later as we get above the tipping
> > +	 * point.
> > +	 */
> > +	do {
> > +		/* Assuming all cpus are the same in group */
> > +		int max_cap_cpu = group_first_cpu(sg);
> > +
> > +		/*
> > +		 * Assume smaller max capacity means more energy-efficient.
> > +		 * Ideally we should query the energy model for the right
> > +		 * answer but it easily ends up in an exhaustive search.
> > +		 */
> > +		if (capacity_of(max_cap_cpu) < target_max_cap &&
> > +		    task_fits_capacity(p, max_cap_cpu)) {
> > +			sg_target = sg;
> > +			target_max_cap = capacity_of(max_cap_cpu);
> > +		}
> > +	} while (sg = sg->next, sg != sd->groups);
> 
> Should be capacity_orig_of(max_cap_cpu) right? Might select a suboptimal
> sg_target if max_cap_cpu has a significant amount of IRQ/RT activity.

Right, this heuristic isn't as good as I had hoped for.
task_fits_capacity() is using capacity_of() to check if we have
available capacity after subtracting RT/IRQ activity which should be
right but I only check the first cpu. So I might discard a group due to
RT/IRQ activity on cpu the first cpu while one the sibling cpus could be
fine. Then going for lowest capacity_of() means preferring group with
most RT/IRQ activity that still has enough capacity to fit the task.

Using capacity_orig_of() we would ignore RT/IRQ activity but is likely
to be better as we can try to avoid RQ/IRQ activity later. I will use
capacity_orig_of() here instead. Thanks.

> > +
> > +	/* Find cpu with sufficient capacity */
> > +	for_each_cpu_and(i, tsk_cpus_allowed(p), sched_group_cpus(sg_target)) {
> > +		/*
> > +		 * p's blocked utilization is still accounted for on prev_cpu
> > +		 * so prev_cpu will receive a negative bias due the double
> > +		 * accouting. However, the blocked utilization may be zero.
> > +		 */
> > +		int new_usage = get_cpu_usage(i) + task_utilization(p);
> > +
> > +		if (new_usage >	capacity_orig_of(i))
> > +			continue;
> 
> Is this supposed to be capacity_of(i) instead?

Yes, we should skip cpus with too much RT/IRQ activity here. Thanks.

Morten
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/