Received: by 2002:a05:6a10:2726:0:0:0:0 with SMTP id ib38csp760167pxb; Tue, 5 Apr 2022 22:07:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxUtf6FR6egBNwNXE4Ppsqady4w6eY/6JbK3MmBb89o38F/gqadux+8pIxKYxX70s/bHFTb X-Received: by 2002:a05:6a00:4292:b0:4fa:7cf8:6cdb with SMTP id bx18-20020a056a00429200b004fa7cf86cdbmr7209918pfb.71.1649221657373; Tue, 05 Apr 2022 22:07:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649221657; cv=none; d=google.com; s=arc-20160816; b=Vcu/4VfkfT5F+cBr3+3jw4YJN9LD6jT0u1t2LoVe0kQ7CPsmYz8Wv95ryzmNEI70Vq T3ZGXu7bBrtFWpxMP7u396qL/Ee0jhF0rDkFSEwaDjn0nxcdybJpCgh0a1ve+W0BR56o uLUlduNZ/18aGg8ThfS3nI36UwpI9dliuifkbnTjKTLQymH5KWJ+HmqU3mDC/Bhu9s+F FHU7SUEhlBXaA4mdeOkf6SC4UzuKV8BpreEeaevT4D4236LnewgTOFTwPZ6P2oG3/yUq OSOYVVIBcdFg0LLpb9PD0i42FJ243NXqqFC3FLEUUrz+ec+aKv6sorUrG2th7xqWFpZr 8uPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=TiTm/Ni/jSCOB4SVu5tVgT/H2PKyGTRMr4RiXIzklDE=; b=VOenmp8dudBpYCRoyE9/DRs9ZHsS2P4d1v+PVg3cKucGS0d6RdpmC+VG+PZ0FT9Cil UNue/tFsqMasMk521Lj3tDQg6G27dK0Kwty9oAHq77FPpwHdage7uLdlz156X10Q0+gc 9pQXkDnT9GuaIj7mBUOkjQxjmxDeUIgs7oXg3UGOO9chyslAnEaRHDwa8HjAqpmeNtdH o8UDCzqyL3NWPvSp2MTLmPiKK43/3rFWbN6W4naQnm+HbJmHHR+UjpGkpr/wD69RA9ik hD8Csq6fquRJL+NzeshaWfWWjuwiVCUeZpRgZWsTfuZsZLf34193/aJpAH2Tboci+PyV Bpjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=xsRUJl8I; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id x189-20020a6363c6000000b003995a4ce0b9si4361743pgb.498.2022.04.05.22.07.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Apr 2022 22:07:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=xsRUJl8I; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9971C191437; Tue, 5 Apr 2022 20:31:08 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381406AbiDELpV (ORCPT + 99 others); Tue, 5 Apr 2022 07:45:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244854AbiDEIwn (ORCPT ); Tue, 5 Apr 2022 04:52:43 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C892237D5; Tue, 5 Apr 2022 01:44:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CB70860FFC; Tue, 5 Apr 2022 08:44:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DD01CC385A1; Tue, 5 Apr 2022 08:44:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1649148293; bh=Fw5yvEs8qrDdncDvk2Na5/mnNyK7AlzuICs41EhKw+s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=xsRUJl8IIipRNtBHXRaVcul1S29hLkypBroLr8PWXEx1jHQdj3XWvSM5oMgH6/g6+ yYxgu+cjvAWCHbDpK5/CJdr9xszV4Q8xSFCr7Mw40LNdS+fc2YvPEycJ1O36LkkdZW ERfUm88Ka1mf0QtUVqqr1Ji5Ij4U1M0k0cDyaFrI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Mel Gorman , "Peter Zijlstra (Intel)" , "Gautham R. Shenoy" , Sasha Levin Subject: [PATCH 5.16 0266/1017] sched/fair: Improve consistency of allowed NUMA balance calculations Date: Tue, 5 Apr 2022 09:19:39 +0200 Message-Id: <20220405070402.159610553@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220405070354.155796697@linuxfoundation.org> References: <20220405070354.155796697@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mel Gorman [ Upstream commit 2cfb7a1b031b0e816af7a6ee0c6ab83b0acdf05a ] There are inconsistencies when determining if a NUMA imbalance is allowed that should be corrected. o allow_numa_imbalance changes types and is not always examining the destination group so both the type should be corrected as well as the naming. o find_idlest_group uses the sched_domain's weight instead of the group weight which is different to find_busiest_group o find_busiest_group uses the source group instead of the destination which is different to task_numa_find_cpu o Both find_idlest_group and find_busiest_group should account for the number of running tasks if a move was allowed to be consistent with task_numa_find_cpu Fixes: 7d2b5dd0bcc4 ("sched/numa: Allow a floating imbalance between NUMA nodes") Signed-off-by: Mel Gorman Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Gautham R. Shenoy Link: https://lore.kernel.org/r/20220208094334.16379-2-mgorman@techsingularity.net Signed-off-by: Sasha Levin --- kernel/sched/fair.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 069e01772d92..9637766e220d 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9062,9 +9062,10 @@ static bool update_pick_idlest(struct sched_group *idlest, * This is an approximation as the number of running tasks may not be * related to the number of busy CPUs due to sched_setaffinity. */ -static inline bool allow_numa_imbalance(int dst_running, int dst_weight) +static inline bool +allow_numa_imbalance(unsigned int running, unsigned int weight) { - return (dst_running < (dst_weight >> 2)); + return (running < (weight >> 2)); } /* @@ -9198,12 +9199,13 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p, int this_cpu) return idlest; #endif /* - * Otherwise, keep the task on this node to stay close - * its wakeup source and improve locality. If there is - * a real need of migration, periodic load balance will - * take care of it. + * Otherwise, keep the task close to the wakeup source + * and improve locality if the number of running tasks + * would remain below threshold where an imbalance is + * allowed. If there is a real need of migration, + * periodic load balance will take care of it. */ - if (allow_numa_imbalance(local_sgs.sum_nr_running, sd->span_weight)) + if (allow_numa_imbalance(local_sgs.sum_nr_running + 1, local_sgs.group_weight)) return NULL; } @@ -9409,7 +9411,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s /* Consider allowing a small imbalance between NUMA groups */ if (env->sd->flags & SD_NUMA) { env->imbalance = adjust_numa_imbalance(env->imbalance, - busiest->sum_nr_running, busiest->group_weight); + local->sum_nr_running + 1, local->group_weight); } return; -- 2.34.1