Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH v3 1/7] sched: limit cpu search in select_idle_cpu
To:     Parth Shah <parth@linux.ibm.com>, linux-kernel@vger.kernel.org
Cc:     peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de,
        steven.sistare@oracle.com, dhaval.giani@oracle.com,
        daniel.lezcano@linaro.org, vincent.guittot@linaro.org,
        viresh.kumar@linaro.org, tim.c.chen@linux.intel.com,
        mgorman@techsingularity.net
References: <20190627012919.4341-1-subhra.mazumdar@oracle.com>
 <20190627012919.4341-2-subhra.mazumdar@oracle.com>
 <68baf89b-6d77-4eff-3aac-f96b72f98bae@linux.ibm.com>
From:   Subhra Mazumdar <subhra.mazumdar@oracle.com>
Message-ID: <e8dc35a9-f46a-b83d-fb7d-f8284ba712c6@oracle.com>
Date:   Fri, 28 Jun 2019 15:21:08 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.1
MIME-Version: 1.0
In-Reply-To: <68baf89b-6d77-4eff-3aac-f96b72f98bae@linux.ibm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 6/28/19 11:47 AM, Parth Shah wrote:
>
> On 6/27/19 6:59 AM, subhra mazumdar wrote:
>> Put upper and lower limit on cpu search of select_idle_cpu. The lower limit
>> is amount of cpus in a core while upper limit is twice that. This ensures
>> for any architecture we will usually search beyond a core. The upper limit
>> also helps in keeping the search cost low and constant.
>>
>> Signed-off-by: subhra mazumdar <subhra.mazumdar@oracle.com>
>> ---
>>   kernel/sched/fair.c | 15 +++++++++++----
>>   1 file changed, 11 insertions(+), 4 deletions(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index f35930f..b58f08f 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -6188,7 +6188,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t
>>   	u64 avg_cost, avg_idle;
>>   	u64 time, cost;
>>   	s64 delta;
>> -	int cpu, nr = INT_MAX;
>> +	int cpu, limit, floor, nr = INT_MAX;
>>
>>   	this_sd = rcu_dereference(*this_cpu_ptr(&sd_llc));
>>   	if (!this_sd)
>> @@ -6206,10 +6206,17 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t
>>
>>   	if (sched_feat(SIS_PROP)) {
>>   		u64 span_avg = sd->span_weight * avg_idle;
>> -		if (span_avg > 4*avg_cost)
>> +		floor = cpumask_weight(topology_sibling_cpumask(target));
>> +		if (floor < 2)
>> +			floor = 2;
>> +		limit = floor << 1;
> Is upper limit an experimental value only or it has any arch specific significance?
> Because, AFAIU, systems like POWER9 might have benefit for searching for 4-cores
> due to its different cache model. So it can be tuned for arch specific builds then.
The lower bound and upper bound were 1 core and 2 core respectively. That
is done as to search beyond one core and at the same time not to search
too much. It is heuristic that seemed to work well on all archs coupled
with the moving window mechanism. Does 4 vs 2 make any difference on your
POWER9? AFAIR it didn't on SPARC SMT8.
>
> Also variable names can be changed for better readability.
> floor -> weight_clamp_min
> limit -> weight_clamp_max
> or something similar
OK.

Thanks,
Subhra
>
>
>> +		if (span_avg > floor*avg_cost) {
>>   			nr = div_u64(span_avg, avg_cost);
>> -		else
>> -			nr = 4;
>> +			if (nr > limit)
>> +				nr = limit;
>> +		} else {
>> +			nr = floor;
>> +		}
>>   	}
>>
>>   	time = local_clock();
>>
>
> Best,
> Parth
>