Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Darren Hart <darren@os.amperecomputing.com>
To:     LKML <linux-kernel@vger.kernel.org>,
        Linux Arm <linux-arm-kernel@lists.infradead.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc:     Sudeep Holla <sudeep.holla@arm.com>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Barry Song <song.bao.hua@hisilicon.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        "D . Scott Phillips" <scott@os.amperecomputing.com>,
        Ilkka Koskinen <ilkka@os.amperecomputing.com>,
        Carl Worth <carl@os.amperecomputing.com>,
        stable@vger.kernel.org
Subject: [PATCH v4] topology: make core_mask include at least cluster_siblings
Date:   Mon,  4 Apr 2022 16:40:37 -0700
Message-Id: <3d58dc946a4fa1cc696d05baad1cf05ae686a86d.1649115057.git.darren@os.amperecomputing.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?u9FYvcF4OVivJMCF6SdXqoGC4B8LzKlrAXrqy53m5pHe4VjjGUORJ96ywqp0?=
 =?us-ascii?Q?92NtH/SezYFjXHhQfmFsobtTOAdaxxxSZ7fjB9Yukc8c0hw/wm4w6ZtZeOlJ?=
 =?us-ascii?Q?KE4p19kUyv0Vb0E8UZqxNGtEkfLFQkQYv89zEhOBHrFIuf4E1gb0JsWsk123?=
 =?us-ascii?Q?FrhU/mu8Nfm562I+0YYku7+quZEkL+yJ88E1sCJWWvH6ME9ca0vX0i8QuQy5?=
 =?us-ascii?Q?Wu2OVw5f7baTuIKFvgrsse7gJW9ph6nt7KOnjyoOKPrdiUn0VZPYu9Ew2GS8?=
 =?us-ascii?Q?tdp5HnOsC+a0TbNlS+KkZL3+nPAu38Y4RiRqtpqdLU2wgRPeIpgqhyUhHu1b?=
 =?us-ascii?Q?G5jO6v/Sb+g6euYRY0kj86ydFaUq8glztiE7ujZx6W0vLT7RUZg9E+kyuptv?=
 =?us-ascii?Q?6qr3kg1mDcek3hjartWLRTKS8i2+4m1auU86i86+6gs8o9TgGhim2VyHGH6A?=
 =?us-ascii?Q?A7/Ph1RAoVDspCJ+xh4If1xCo+VInyvjycV9gzq7jH0kScWKKYBQvFlgEhwq?=
 =?us-ascii?Q?MVDSTOlbjTSO7zdUaD92+At7seSWjxdpv7u12LoZVDxOpRwRmm43as7mGIRd?=
 =?us-ascii?Q?RU0SUMpmZ+/KD5y9PpEv7Jci9zsD7rUUFjBZm6MYvx35av9CYk2RPVN7KFqb?=
 =?us-ascii?Q?u630GSOWAnxDQthEz6BMO7HPbq2Dha1cUN0AmBFgfJAhTH99aMWcxKCKNTZN?=
 =?us-ascii?Q?sWzymci6J5TV0V39BkAALbCk3tR3GTPzo/oPeKvU99UIUWmQiw5fff/f7BLi?=
 =?us-ascii?Q?It2XKpU5FdwqDwXub8IijKsWK4hTkcbL0iYz+hfz0jNZPgvq1GcJZhbi8fYM?=
 =?us-ascii?Q?pNxM6mlEaKpJjLsGEVxQb36zler6tanCk+CTdjGChKONS1GIeNaBSRM8A3Bt?=
 =?us-ascii?Q?Rk9PF2MWXwN75COsMIPqOyyXZuB3Aj2p4twMZhmfzszYOIzAc6t/9ubkH9Xz?=
 =?us-ascii?Q?8MwCbi2De3GejBOQULwp71PxbHIlgbRgFmJ59Ed+qrYc/RdiJqC53RT6yJfz?=
 =?us-ascii?Q?vX8SDLkKxLmyOo5cUEypTkxUCZGfmWfeEprp5xdu10NktV7oZ27NS54c+g+q?=
 =?us-ascii?Q?H++flMKIdEwbPgNb/Lv43bx+CZWPROma9yb5fypJNOK44lWgH85gZS3lVa6E?=
 =?us-ascii?Q?2vp89+j7k43Sjm7qWHfnYrxeHFGi7VMXHQNxvtsc+zzuS3V/nhL/bja0URow?=
 =?us-ascii?Q?itdMwENdxtCaUkRnXVDj16h9A2CAvLqGYMpWtVym8ZEaqfokfoOw+cSgUDat?=
 =?us-ascii?Q?OY3CzNwOBj3Bvs6gzH6FL/WaB1UXIMyBCFfXVhL2ABv4Gzulv5xMC3sQ1fuE?=
 =?us-ascii?Q?k+HI+hZrpUdR4q41Y9UrM8nyrtZzFTWnd1npZKQuEXr3S8EoUIK/o66RKo3a?=
 =?us-ascii?Q?u8eiFusQq98HaMe39WNjLOEym6pmo4782Wjh6F6q/ymc9v+RmbzuFHVsO8Uf?=
 =?us-ascii?Q?lVoIFBLTan820pHuCrHPZ2zWYt5hrdP1xQUd/vd3rIk8qX2iexGXSABHmSk+?=
 =?us-ascii?Q?fnbk05sPwoi2odm+5REq+icAsjwT2E3rCWVXRXCU7uEyZS9b5oTAQKDaizEy?=
 =?us-ascii?Q?eeHP/zPqMth6yZ9BtqatHWu8e4u+fuUv29Bss1BhF50m2McrqCnhU6HTPbXH?=
 =?us-ascii?Q?OQPaeaXYHcCiBt8z//iE2hzLy2psECdFahWM6/f6IjDyePTtDKze4dwog4hX?=
 =?us-ascii?Q?NjceRMp/Oz2k8rVE3kfPmvs14sdSGOhVaWmVdTjtoEXTBOuGnYXt7EDkSOWj?=
 =?us-ascii?Q?Dv9/elOP/cOIdwIshD3LU7ukgdEKoP4VO2DRBOSR19qg1o7F4WqY?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7dd126a3-40c3-48b6-203f-08da1694bc28
X-MS-Exchange-CrossTenant-AuthSource: SA0PR01MB6329.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Apr 2022 23:42:09.7031
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IHSxCvSOAKReUhC9CghMureSWfof0gidUUF2ZMV4JwLYK3Uj6vKlcMZJE9XTf8gtDOcYGSLNFDLs3pko+z0sCZt/PcVGYVOYIQ3UooY99B9Ja8d0Wufm+vEbiuOsudOI
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR01MB4924
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Ampere Altra defines CPU clusters in the ACPI PPTT. They share a Snoop
Control Unit, but have no shared CPU-side last level cache.

cpu_coregroup_mask() will return a cpumask with weight 1, while
cpu_clustergroup_mask() will return a cpumask with weight 2.

As a result, build_sched_domain() will BUG() once per CPU with:

BUG: arch topology borken
the CLS domain not a subset of the MC domain

The MC level cpumask is then extended to that of the CLS child, and is
later removed entirely as redundant. This sched domain topology is an
improvement over previous topologies, or those built without
SCHED_CLUSTER, particularly for certain latency sensitive workloads.
With the current scheduler model and heuristics, this is a desirable
default topology for Ampere Altra and Altra Max system.

Rather than create a custom sched domains topology structure and
introduce new logic in arch/arm64 to detect these systems, update the
core_mask so coregroup is never a subset of clustergroup, extending it
to cluster_siblings if necessary. Only do this if CONFIG_SCHED_CLUSTER
is enabled to avoid also changing the topology (MC) when
CONFIG_SCHED_CLUSTER is disabled.

This has the added benefit over a custom topology of working for both
symmetric and asymmetric topologies. It does not address systems where
the CLUSTER topology is above a populated MC topology, but these are not
considered today and can be addressed separately if and when they
appear.

The final sched domain topology for a 2 socket Ampere Altra system is
unchanged with or without CONFIG_SCHED_CLUSTER, and the BUG is avoided:

For CPU0:

CONFIG_SCHED_CLUSTER=y
CLS  [0-1]
DIE  [0-79]
NUMA [0-159]

CONFIG_SCHED_CLUSTER is not set
DIE  [0-79]
NUMA [0-159]

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Sudeep Holla <sudeep.holla@arm.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Barry Song <song.bao.hua@hisilicon.com>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: D. Scott Phillips <scott@os.amperecomputing.com>
Cc: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Cc: Carl Worth <carl@os.amperecomputing.com>
Cc: <stable@vger.kernel.org> # 5.16.x
Suggested-by: Barry Song <song.bao.hua@hisilicon.com>
Signed-off-by: Darren Hart <darren@os.amperecomputing.com>
---
v1: Drop MC level if coregroup weight == 1
v2: New sd topo in arch/arm64/kernel/smp.c
v3: No new topo, extend core_mask to cluster_siblings
v4: Rebase on 5.18-rc1 for GregKH to pull. Add IS_ENABLED(CONFIG_SCHED_CLUSTER).

 drivers/base/arch_topology.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 1d6636ebaac5..5497c5ab7318 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -667,6 +667,15 @@ const struct cpumask *cpu_coregroup_mask(int cpu)
 			core_mask = &cpu_topology[cpu].llc_sibling;
 	}
 
+	/*
+	 * For systems with no shared cpu-side LLC but with clusters defined,
+	 * extend core_mask to cluster_siblings. The sched domain builder will
+	 * then remove MC as redundant with CLS if SCHED_CLUSTER is enabled.
+	 */
+	if (IS_ENABLED(CONFIG_SCHED_CLUSTER) &&
+	    cpumask_subset(core_mask, &cpu_topology[cpu].cluster_sibling))
+		core_mask = &cpu_topology[cpu].cluster_sibling;
+
 	return core_mask;
 }
 
-- 
2.31.1