Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Vladimir Oltean <vladimir.oltean@nxp.com>
To:     linux-kernel@vger.kernel.org
Cc:     Lee Jones <lee.jones@linaro.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Marc Zyngier <maz@kernel.org>,
        Rasmus Villemoes <linux@rasmusvillemoes.dk>,
        Arnd Bergmann <arnd@arndb.de>,
        Hou Zhiqiang <Zhiqiang.Hou@nxp.com>,
        Biwen Li <biwen.li@nxp.com>,
        Sean Anderson <sean.anderson@seco.com>
Subject: [PATCH v4] irqchip/ls-extirq: fix invalid wait context by avoiding to use regmap
Date:   Thu, 28 Jul 2022 17:42:54 +0300
Message-Id: <20220728144254.175385-1-vladimir.oltean@nxp.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?r84BFUkss21CYI8sVsbRuTNKEB8gpgUy0E+iK5mf6413D6VjZ/FAe77P4hI6?=
 =?us-ascii?Q?5MBoGmikwbNSPoOPqlPcEAX2m+t1uMKLSq5NpZt7Qpikk3qrjpFx5eLc83OT?=
 =?us-ascii?Q?L2F/9YCYW9D7xLXJngC18N14xrlxkUH7nF6DwJ+r0Bh8W8P57BzyElyPITw+?=
 =?us-ascii?Q?6l03+uR1IKGyInRlFOLQi74Hh7E0rMDUfrZyhuiESfUgYNeInRZnBSx7LCnx?=
 =?us-ascii?Q?fSHCrB9cIvQHIEF/g5CCuvM/ncHQPxWquxYid2aNhzt67kQst71+PwlKLZIj?=
 =?us-ascii?Q?66DjiJLkdFUQm9hvi6AVAwfavsJHKvkYVj8KbUAE1yywDIlPUY31ijhAMPxN?=
 =?us-ascii?Q?uFTDRUhaP+orgbA8EMvZcghjHKESwUJ7rUdCi2pqvtpsMVuuYi7tn02N36+3?=
 =?us-ascii?Q?LiOV4VnEERKtL6mM6aFdsm0pUDT0ttXDiGNuBBcZjqYPraHTJeDwm1fg8Xcr?=
 =?us-ascii?Q?h0evXEHJ/l7nm1Qee3oaoKEX4xlO+NvwMgO4Pf+Zio1qgrnDcRAqanoacqWW?=
 =?us-ascii?Q?WkxTPISSGEM2zXSRzffWSnqwVvG6M0ye+YwuC8qjlEe8Zg1yrPZlLpKLF70D?=
 =?us-ascii?Q?VuRaNhWPybLoMm4KyTI/93T8e85fHm59bwv50pqRTkvGh7oyFKeY/nGQXAfP?=
 =?us-ascii?Q?h3yUPqgjbtPlMoV2amQHpjnQJivYb66unMCd1oQgiHrah7O3zeS3dEx3TFV4?=
 =?us-ascii?Q?IQkS7VYuSd688OjwIHiUqIpvnxiWYQSsANhzUopt/VmbtD++fW4u2ug0N0qc?=
 =?us-ascii?Q?pC/cGLU2kl4gj99ZNFD1zbVM/o4hVzQQpS4gbtLn2EEn1re9bqrhbkDV4AoN?=
 =?us-ascii?Q?i/nM7YPtI/d56ZvAV12GZI5A8+hT9XWuZXRiCl3XPLLa1gGaJSBAGdbb0Z6l?=
 =?us-ascii?Q?qUT+95RYkdA35olmlvA/moJUnHYT8XWMJFNGS9A9HQBY0NtW2bR9HJc0i0FO?=
 =?us-ascii?Q?SzK9V+tfAjoRZsOboSrET8PwttXrnoHs9DdskZoHuGlBMwqjKTPcU46wVsCV?=
 =?us-ascii?Q?6k5avEY7qbdb/qVPULowNdrVdphTMqH/K5rk8m2Fr1XInEVAMICecuY3VhUD?=
 =?us-ascii?Q?TXx7J9pXjYdl6x3k88ZrwkHGvCVvv/wAKbII2X9uwrTwqc8Sftqc24CiXhAc?=
 =?us-ascii?Q?QikxA986l1qi0svTmpuBwG86+yNTtcrGBpDg6/RZdGWbxiYZhexzu/hLC3RL?=
 =?us-ascii?Q?fbQ044JTx3gqugAHnqzh8sLRTIL/RbtHAwpTSmvG9mgSp406agzpMmkP0/xH?=
 =?us-ascii?Q?12Ssrf/+QN6X/vSQA7eV1cm3OSdpPEiRLDavuddOXhsMFvncCuhlCxZ2E09j?=
 =?us-ascii?Q?ztFJaJr1go9JIW62UCDWOPJMXiPj4XQhXgh+G+k7KSpW59RD5bfhFOrjf3YG?=
 =?us-ascii?Q?P70egLFGLu6OzIz3XhiyPZVsuVhJTeQn/QUi2uEnsSgxrf08Fn3QWQKx1W79?=
 =?us-ascii?Q?DfS0L4p02fKGSzqhOYoYMGCj/NZLlp1jtDPJeGZVI67lEswatjxoSdaMfQOM?=
 =?us-ascii?Q?HWxr/yYWKWziOQnMecg0/yGUIQ2L3mgyiyepd5mFUfQ3K9rOisPE2i4Kh3MV?=
 =?us-ascii?Q?J5ECOvNz93WAwPlEjlPP3p7Hbk2P6DJcAzNelOhLo5Zg9VIfywIoilIDbdau?=
 =?us-ascii?Q?Nw=3D=3D?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43291f4a-2e53-40b5-9d8f-08da70a77a2e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5136.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jul 2022 14:43:04.0102
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RMiQWEr2dK+Xs0IpP3l4KjhPrLhDO3xptF6xAkBEjc3G70yhE757+aJNYrBlExhfA/I6nwlWWSRwAX2N+oz4tw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6307
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,SPF_HELO_PASS,
        T_SPF_PERMERROR autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The irqchip->irq_set_type method is called by __irq_set_trigger() under
the desc->lock raw spinlock.

The ls-extirq implementation, ls_extirq_irq_set_type(), uses an MMIO
regmap created by of_syscon_register(), which uses plain spinlocks
(the kind that are sleepable on RT).

Therefore, this is an invalid locking scheme for which we get a kernel
splat stating just that ("[ BUG: Invalid wait context ]"), because the
context in which the plain spinlock may sleep is atomic due to the raw
spinlock. We need to go raw spinlocks all the way.

Make this driver ioremap its INTPCR register on its own, and stop
relying on syscon to provide a regmap. Since the regmap we got from
syscon belonged to the parent and the newly ioremapped region belongs
just to us, the offset to the INTPCR register is now 0, because of the
address translation that takes place through the device tree.

One complication, due to the fact that this driver uses IRQCHIP_DECLARE
rather than traditional platform devices with probe and remove methods,
is that we cannot use devres, so we need to implement a full-blown
cleanup procedure on the error path.

Fixes: 0dcd9f872769 ("irqchip: Add support for Layerscape external interrupt lines")
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
v3->v4:
- fix the dumb decision of dropping spinlocks altogether in the extirq
  driver, and add a comment as to why they're there
- drop priv->read() and priv->write() and open-code the read-modify-write
  inside a new ls_extirq_intpcr_rmw() helper.

v2->v3:
- stop using regmap, do the rmw manually using function pointers for BE/LE
- adapt comment style to the subsystem
- use of_device_is_big_endian
- reword commit message

v1->v2:
- create a separate regmap for the ls-extirq driver rather than relying
  on the one provided by syscon or modifying that.

For reference, v1 is at:
https://lore.kernel.org/lkml/20210825205041.927788-3-vladimir.oltean@nxp.com/
and v2 is at:
https://lore.kernel.org/lkml/20220722204019.969272-1-vladimir.oltean@nxp.com/
and v3 is at:
https://lore.kernel.org/lkml/20220727160915.3648155-1-vladimir.oltean@nxp.com/

For extra reviewer convenience, the ls-extirq appears in the following
SoC device trees:
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi#L289
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi#L249
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi#L319
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi#L325
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi#L682
https://elixir.bootlin.com/linux/v5.18.13/source/arch/arm/boot/dts/ls1021a.dtsi#L182

Patch tested on LX2160A and LS1021A.

 drivers/irqchip/irq-ls-extirq.c | 87 ++++++++++++++++++++++++---------
 1 file changed, 63 insertions(+), 24 deletions(-)

diff --git a/drivers/irqchip/irq-ls-extirq.c b/drivers/irqchip/irq-ls-extirq.c
index 853b3972dbe7..d8d48b1f7c29 100644
--- a/drivers/irqchip/irq-ls-extirq.c
+++ b/drivers/irqchip/irq-ls-extirq.c
@@ -6,8 +6,7 @@
 #include <linux/irqchip.h>
 #include <linux/irqdomain.h>
 #include <linux/of.h>
-#include <linux/mfd/syscon.h>
-#include <linux/regmap.h>
+#include <linux/of_address.h>
 #include <linux/slab.h>
 
 #include <dt-bindings/interrupt-controller/arm-gic.h>
@@ -16,13 +15,41 @@
 #define LS1021A_SCFGREVCR 0x200
 
 struct ls_extirq_data {
-	struct regmap		*syscon;
-	u32			intpcr;
+	void __iomem		*intpcr;
+	raw_spinlock_t		lock;
+	bool			big_endian;
 	bool			is_ls1021a_or_ls1043a;
 	u32			nirq;
 	struct irq_fwspec	map[MAXIRQ];
 };
 
+static void ls_extirq_intpcr_rmw(struct ls_extirq_data *priv, u32 mask,
+				 u32 value)
+{
+	u32 intpcr;
+
+	/*
+	 * Serialize concurrent calls to ls_extirq_set_type() from multiple
+	 * IRQ descriptors, making sure the read-modify-write is atomic.
+	 */
+	raw_spin_lock(&priv->lock);
+
+	if (priv->big_endian)
+		intpcr = ioread32be(priv->intpcr);
+	else
+		intpcr = ioread32(priv->intpcr);
+
+	intpcr &= ~mask;
+	intpcr |= value;
+
+	if (priv->big_endian)
+		iowrite32be(intpcr, priv->intpcr);
+	else
+		iowrite32(intpcr, priv->intpcr);
+
+	raw_spin_unlock(&priv->lock);
+}
+
 static int
 ls_extirq_set_type(struct irq_data *data, unsigned int type)
 {
@@ -51,7 +78,8 @@ ls_extirq_set_type(struct irq_data *data, unsigned int type)
 	default:
 		return -EINVAL;
 	}
-	regmap_update_bits(priv->syscon, priv->intpcr, mask, value);
+
+	ls_extirq_intpcr_rmw(priv, mask, value);
 
 	return irq_chip_set_type_parent(data, type);
 }
@@ -143,7 +171,6 @@ ls_extirq_parse_map(struct ls_extirq_data *priv, struct device_node *node)
 static int __init
 ls_extirq_of_init(struct device_node *node, struct device_node *parent)
 {
-
 	struct irq_domain *domain, *parent_domain;
 	struct ls_extirq_data *priv;
 	int ret;
@@ -151,40 +178,52 @@ ls_extirq_of_init(struct device_node *node, struct device_node *parent)
 	parent_domain = irq_find_host(parent);
 	if (!parent_domain) {
 		pr_err("Cannot find parent domain\n");
-		return -ENODEV;
+		ret = -ENODEV;
+		goto err_irq_find_host;
 	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
-	if (!priv)
-		return -ENOMEM;
-
-	priv->syscon = syscon_node_to_regmap(node->parent);
-	if (IS_ERR(priv->syscon)) {
-		ret = PTR_ERR(priv->syscon);
-		pr_err("Failed to lookup parent regmap\n");
-		goto out;
+	if (!priv) {
+		ret = -ENOMEM;
+		goto err_alloc_priv;
 	}
-	ret = of_property_read_u32(node, "reg", &priv->intpcr);
-	if (ret) {
-		pr_err("Missing INTPCR offset value\n");
-		goto out;
+
+	/*
+	 * All extirq OF nodes are under a scfg/syscon node with
+	 * the 'ranges' property
+	 */
+	priv->intpcr = of_iomap(node, 0);
+	if (!priv->intpcr) {
+		pr_err("Cannot ioremap OF node %pOF\n", node);
+		ret = -ENOMEM;
+		goto err_iomap;
 	}
 
 	ret = ls_extirq_parse_map(priv, node);
 	if (ret)
-		goto out;
+		goto err_parse_map;
 
+	priv->big_endian = of_device_is_big_endian(parent);
 	priv->is_ls1021a_or_ls1043a = of_device_is_compatible(node, "fsl,ls1021a-extirq") ||
 				      of_device_is_compatible(node, "fsl,ls1043a-extirq");
+	raw_spin_lock_init(&priv->lock);
 
 	domain = irq_domain_add_hierarchy(parent_domain, 0, priv->nirq, node,
 					  &extirq_domain_ops, priv);
-	if (!domain)
+	if (!domain) {
 		ret = -ENOMEM;
+		goto err_add_hierarchy;
+	}
 
-out:
-	if (ret)
-		kfree(priv);
+	return 0;
+
+err_add_hierarchy:
+err_parse_map:
+	iounmap(priv->intpcr);
+err_iomap:
+	kfree(priv);
+err_alloc_priv:
+err_irq_find_host:
 	return ret;
 }
 
-- 
2.34.1