Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
From:   Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>
To:     Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc:     Aleksandar Rikalo <arikalo@gmail.com>,
        Chao-ying Fu <cfu@wavecomp.com>,
        Daniel Lezcano <daniel.lezcano@linaro.org>,
        Dragan Mladjenovic <dragan.mladjenovic@syrmia.com>,
        Geert Uytterhoeven <geert@linux-m68k.org>,
        Greg Ungerer <gerg@kernel.org>,
        Hauke Mehrtens <hauke@hauke-m.de>,
        Ilya Lipnitskiy <ilya.lipnitskiy@gmail.com>,
        Jiaxun Yang <jiaxun.yang@flygoat.com>,
        linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org,
        Marc Zyngier <maz@kernel.org>,
        Paul Burton <paulburton@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Serge Semin <fancer.lancer@gmail.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Tiezhu Yang <yangtiezhu@loongson.cn>
Subject: [PATCH v3 04/11] irqchip: mips-gic: Support multi-cluster in for_each_online_cpu_gic()
Date:   Tue, 31 Oct 2023 17:49:20 +0100
Message-Id: <20231031164927.371663-5-aleksandar.rikalo@syrmia.com>
In-Reply-To: <20231031164927.371663-1-aleksandar.rikalo@syrmia.com>
References: <20231031164927.371663-1-aleksandar.rikalo@syrmia.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?x0R3kE8UgxS3Fi6zSkA2SdKRmmW3AzyGwX9TII5HLgW1QUM06SzVW6LhKToe?=
 =?us-ascii?Q?oIgbCtsjwLcEjVOJgemzFlc3cFTx1Xmt6EJ/11zkBDJhHWHeExOZ4brI0qCH?=
 =?us-ascii?Q?UbMJBtjgAtOA06oAsRVJRBZXiuy4uF0JzxUw7k12902niRM+hS9ZdeUjyvQZ?=
 =?us-ascii?Q?uevoHKodX0skFGz18gEEadDQDDDgKvdFGxW/yo7yEY34LUFpCWjCzwEx9SoX?=
 =?us-ascii?Q?TgwQJ/VnQX+WxTg7Rs5moA/Gyg2i0JYqjdEIo3ab/HLOe3YI+nxDWjG9wXqL?=
 =?us-ascii?Q?zdSFD9gYFgVahxX7O4setuXZp8WXE2Y659blgijQkaIIgbWeWcBszyGHpdV2?=
 =?us-ascii?Q?yXpxNGCXECUp/CY2yTr91uYPo+06l/sHrABefzq6UTWBiOWjuUMei3H1kr8I?=
 =?us-ascii?Q?A8YFpqQBrUBV1EgcZfmeY5Jd9F4o7ZZBTjq4yOcOFg4kJcBa+45XyvJzCXBI?=
 =?us-ascii?Q?FoLmjuJzEqn12pqcTp9tWnd7L+ApBtOdd02U6rlbb4x6LsdjK4EegsAiEdAL?=
 =?us-ascii?Q?sLfT8XRsmog0LuHEzkm0T7oie1+znbBeG7ONfgX6CfD+X7fHgLKYwbjAuGwG?=
 =?us-ascii?Q?VmpONmCE9mrXonLTsw3z3YSK/vxEv4tmV6tWAa/4ONDPgWxS07xc2vLeVV9K?=
 =?us-ascii?Q?zcj2A0SLWGCt8/V+JuuTk5v9Dqc0o3HlvFPhciSU1Ts/Y5MJBuXphcj1vGZd?=
 =?us-ascii?Q?WGA8r/5rq0YjFbLnLdrAfci9Tmpj3/alAHFrj5bJNIGwiMV/RAqx2mIXP0P3?=
 =?us-ascii?Q?FxmvvZbziSn37FCx4uu4paWpyxXmkixq4eT+GmyHx/g42LT8FUnvkvqP+O8M?=
 =?us-ascii?Q?LfqUiOlmOX2YEHeegCOFX97PdqCgs2lVnJfQH5aT2A/32ZLL60Rp1vz6Q/Uy?=
 =?us-ascii?Q?ApmTq2YO3M3Tcbp7c1+2PSIP/AdWYv/Xpb1/7Yidp+NDB8N2dkvqT2T7c0PG?=
 =?us-ascii?Q?nhcIG4fNN8onVdHVJRl65WlK7ZtDCI+xFzxpm2cmvWnDARvvibyhNYKM6Hxt?=
 =?us-ascii?Q?+TFQ4aYusd6/KZDS7j7sDUM6ag3hB8RNZXz5NfxLrcAtghl6dUylHWZ9tX76?=
 =?us-ascii?Q?a9NV32f1Q+rLE8Tzy5+ONLgUlOnfISyjdF9arHhxq5eNWBe3KPZgW46sKeRa?=
 =?us-ascii?Q?3JN4xV2w3ph/WJckoJqERaSpmjUA+kDIAHOXZuhNnFIcVOZ8uw31p9+dZfVT?=
 =?us-ascii?Q?VauvcQprhNu84xq2vQFan2ta60SaSDdaRZiKpd/QEwkF5jJFliO/Vex64aDY?=
 =?us-ascii?Q?vRYKpHUonbeK2p0vXPusTJlbwj84U8eFxFJ1hntRiQ5z1io6iz2GSQTCG/gN?=
 =?us-ascii?Q?1QxlO7gKuElbqzwyCB+qa1ZEXVxp+5OArqqZTT/WKL0acPiCY1rHzq3Ub3+B?=
 =?us-ascii?Q?W03MXABo6wPV/R0hIzkrvjcvOsXB+BXLbMPhmPb7A0tjCqG1qtmQlekWraen?=
 =?us-ascii?Q?FYpxaYY/i+ZhaCzAXmzDlDLXRyjD9E/UaE8K6xs6bgK+kK8xVc6KgJISBTEa?=
 =?us-ascii?Q?qea7L8yPHs/r+yBQ2pIEbQ5TMIvRa1PZboW3z7YRM/fDTQbqLbH8JB3GlB14?=
 =?us-ascii?Q?DUHPL/s5iUNZEYB88qWBRCi2HFM6+aiMIwEDQ18kXUQJk3PDGSMNPqP0QS4x?=
 =?us-ascii?Q?4w=3D=3D?=
X-OriginatorOrg: syrmia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eb3e7f4a-3b39-4b8b-b8c6-08dbda3161bb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR0302MB3486.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 16:49:43.2572
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 19214a73-c1ab-4e19-8f59-14bdcb09a66e
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8chkTFbEuC5q2LC+w3S30s5jf7gpb1FxPlF8zoZEKhPUsSJNfebhcMJzV3H2g/ut0Suu8In7BULBbEYHJ2BQfjjI3baOQKbBk+FzShRHlTk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7187
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 31 Oct 2023 09:53:25 -0700 (PDT)

From: Paul Burton <paulburton@kernel.org>

Introduce support for multi-cluster GIC register access in
__gic_with_next_online_cpu(), and therefore in its user
for_each_online_cpu_gic(). We access registers in remote clusters
using the CM's GCR_CL_REDIRECT register, and so here we delegate
to mips_cm_lock_other() in order to configure this access.

With this done, users of gic_with_each_online_cpu() gain support
for multi-cluster with no further changes.

Signed-off-by: Paul Burton <paulburton@kernel.org>
Signed-off-by: Chao-ying Fu <cfu@wavecomp.com>
Signed-off-by: Dragan Mladjenovic <dragan.mladjenovic@syrmia.com>
Signed-off-by: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>
---
 drivers/irqchip/irq-mips-gic.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index 9e7182150b5c..317ccc2593d1 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -70,6 +70,20 @@ static int __gic_with_next_online_cpu(int prev)
 {
 	unsigned int cpu;
 
+	/*
+	 * Unlock access to the previous CPU's GIC local register block.
+	 *
+	 * Delegate to the CM locking code in the multi-cluster case, since
+	 * other clusters can only be accessed using GCR_CL_REDIRECT.
+	 *
+	 * In the single cluster case we don't need to do anything; the caller
+	 * is responsible for maintaining gic_lock & nothing should be
+	 * expecting any particular value of GIC_VL_OTHER so we can leave it
+	 * as-is.
+	 */
+	if ((prev != -1) && mips_cps_multicluster_cpus())
+		mips_cm_unlock_other();
+
 	/* Discover the next online CPU */
 	cpu = cpumask_next(prev, cpu_online_mask);
 
@@ -80,10 +94,16 @@ static int __gic_with_next_online_cpu(int prev)
 	/*
 	 * Lock access to the next CPU's GIC local register block.
 	 *
+	 * Delegate to the CM locking code in the multi-cluster case, since
+	 * other clusters can only be accessed using GCR_CL_REDIRECT.
+	 *
 	 * In the single cluster case we simply set GIC_VL_OTHER. The caller
 	 * holds gic_lock so nothing can clobber the value we write.
 	 */
-	write_gic_vl_other(mips_cm_vp_id(cpu));
+	if (mips_cps_multicluster_cpus())
+		mips_cm_lock_other_cpu(cpu, CM_GCR_Cx_OTHER_BLOCK_LOCAL);
+	else
+		write_gic_vl_other(mips_cm_vp_id(cpu));
 
 	return cpu;
 }
@@ -105,6 +125,9 @@ static inline void __lockdep_assert_held(raw_spinlock_t *gic_lock)
  *
  * The caller must hold gic_lock throughout the loop, such that GIC_VL_OTHER
  * cannot be clobbered.
+ *
+ * Please note that if you start a loop this way, it must be completed to the
+ * end, otherwise, GIC local register block will remain locked.
  */
 #define for_each_online_cpu_gic(cpu, gic_lock)		\
 	for (__lockdep_assert_held(gic_lock),		\
-- 
2.25.1