Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <431faa39-4f5c-0087-7ce5-16796ca1a9e1@linux.vnet.ibm.com>
Date:   Sat, 13 May 2023 00:11:45 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.0
Subject: Re: [PATCH v4 05/12] sched/fair: Keep a fully_busy SMT sched group as
 busiest
Content-Language: en-US
To:     Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Cc:     Ricardo Neri <ricardo.neri@intel.com>,
        "Ravi V. Shankar" <ravi.v.shankar@intel.com>,
        Ben Segall <bsegall@google.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Len Brown <len.brown@intel.com>, Mel Gorman <mgorman@suse.de>,
        "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
        Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Ionela Voinescu <ionela.voinescu@arm.com>, x86@kernel.org,
        linux-kernel@vger.kernel.org,
        "Tim C . Chen" <tim.c.chen@intel.com>,
        "Peter Zijlstra (Intel)" <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        shrikanth hegde <sshegde@linux.vnet.ibm.com>,
        Srikar Dronamraju <srikar@linux.vnet.ibm.com>
References: <20230406203148.19182-1-ricardo.neri-calderon@linux.intel.com>
 <20230406203148.19182-6-ricardo.neri-calderon@linux.intel.com>
From:   Shrikanth Hegde <sshegde@linux.vnet.ibm.com>
In-Reply-To: <20230406203148.19182-6-ricardo.neri-calderon@linux.intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 4/7/23 2:01 AM, Ricardo Neri wrote:
> When comparing two fully_busy scheduling groups, keep the current busiest
> group if it represents an SMT core. Tasks in such scheduling group share
> CPU resources and need more help than tasks in a non-SMT fully_busy group.
>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Ionela Voinescu <ionela.voinescu@arm.com>
> Cc: Len Brown <len.brown@intel.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Tim C. Chen <tim.c.chen@intel.com>
> Cc: Valentin Schneider <vschneid@redhat.com>
> Cc: x86@kernel.org
> Cc: linux-kernel@vger.kernel.org
> Tested-by: Zhang Rui <rui.zhang@intel.com>
> Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> ---
> Changes since v3:
>  * None
>
> Changes since v2:
>  * Introduced this patch.
>
> Changes since v1:
>  * N/A
> ---
>  kernel/sched/fair.c | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index b151e93ec316..ea23a5163bfa 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -9566,10 +9566,22 @@ static bool update_sd_pick_busiest(struct lb_env *env,
>  		 * contention when accessing shared HW resources.
>  		 *
>  		 * XXX for now avg_load is not computed and always 0 so we
> -		 * select the 1st one.
> +		 * select the 1st one, except if @sg is composed of SMT
> +		 * siblings.
>  		 */
> -		if (sgs->avg_load <= busiest->avg_load)
> +
> +		if (sgs->avg_load < busiest->avg_load)
>  			return false;
> +
> +		if (sgs->avg_load == busiest->avg_load) {
> +			/*
> +			 * SMT sched groups need more help than non-SMT groups.
> +			 * If @sg happens to also be SMT, either choice is good.
> +			 */
> +			if (sds->busiest->flags & SD_SHARE_CPUCAPACITY)
> +				return false;
> +		}
> +
>  		break;

IIUC,

Earlier, we used to go to out_balanced if sgs->avg_load <= busiest->avg_load.
Now we go only if it is less. lets say sgs->avg_load == busiest->avg_load,
then we will return true in MC,DIE domain. This might end up traversing
multiple such group's and pick the last one as the busiest instead of
first. I guess eventually any load balance if exists will be fixed.  But
this might cause slight overhead. would it?


nit: There is typo in [2/12]  if the whole core is repeated.
+	 * CPUs. When done between cores, do it only if the whole core if the
+	 * whole core is idle.

Mentioning in this reply instead, to avoid sending another mail reply for this.


>  
>  	case group_has_spare: