Date: Thu, 9 Jul 2015 15:26:54 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Mike Galbraith <umgwanakikbuti@gmail.com>
Cc: Josef Bacik <jbacik@fb.com>, riel@redhat.com, mingo@redhat.com,
        linux-kernel@vger.kernel.org, morten.rasmussen@arm.com,
        kernel-team <Kernel-team@fb.com>
Subject: Re: [patch] sched: beef up wake_wide()
Message-ID: <20150709132654.GE3644@twins.programming.kicks-ass.net>
References: <1436025462.17152.37.camel@gmail.com>
 <1436080661.22930.22.camel@gmail.com>
 <1436159590.5850.27.camel@gmail.com>
 <559A91F4.7000903@fb.com>
 <1436207790.2940.30.camel@gmail.com>
 <559AD9CE.4090309@fb.com>
 <1436241678.1836.29.camel@gmail.com>
 <1436262224.1836.74.camel@gmail.com>
 <559C0700.6090009@fb.com>
 <1436336026.3767.53.camel@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1436336026.3767.53.camel@gmail.com>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2215
Lines: 61

On Wed, Jul 08, 2015 at 08:13:46AM +0200, Mike Galbraith wrote:
> 
> +/*
> + * Detect 1:N waker/wakee relationship via a switching-frequency heuristic.
> + * A waker of many should wake a different task than the one last awakened
> + * at a frequency roughly N times higher than one of its wakees.  In order
> + * to determine whether we should let the load spread vs consolodating to
> + * shared cache, we look for a minimum 'flip' frequency of llc_size in one
> + * partner, and a factor of lls_size higher frequency in the other.  With
> + * both conditions met, we can be relatively sure that we are seeing a 1:N
> + * relationship, and that load size exceeds socket size.
> + */
>  static int wake_wide(struct task_struct *p)
>  {
> +	unsigned int waker_flips = current->wakee_flips;
> +	unsigned int wakee_flips = p->wakee_flips;
>  	int factor = this_cpu_read(sd_llc_size);
>  
> +	if (waker_flips < wakee_flips)
> +		swap(waker_flips, wakee_flips);

This makes the wakee/waker names useless, the end result is more like
wakee_flips := client_flips, waker_flips := server_flips.

> +	if (wakee_flips < factor || waker_flips < wakee_flips * factor)
> +		return 0;

I don't get the first condition... why would the client ever flip? It
only talks to that one server.

> +	return 1;
>  }


> @@ -5021,14 +5015,17 @@ select_task_rq_fair(struct task_struct *
>  {
>  	struct sched_domain *tmp, *affine_sd = NULL, *sd = NULL;
>  	int cpu = smp_processor_id();
> +	int new_cpu = prev_cpu;
>  	int want_affine = 0;
>  	int sync = wake_flags & WF_SYNC;
>  
>  	rcu_read_lock();
> +	if (sd_flag & SD_BALANCE_WAKE) {
> +		want_affine = !wake_wide(p) && cpumask_test_cpu(cpu, tsk_cpus_allowed(p));
> +		if (!want_affine)
> +			goto select_idle;
> +	}

So this preserves/makes worse the bug Morten spotted, even without
want_affine we should still attempt SD_BALANCE_WAKE if set.

> +
>  	for_each_domain(cpu, tmp) {
>  		if (!(tmp->flags & SD_LOAD_BALANCE))
>  			continue;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/