Hi Thomas, Marc,
This is V2 of previous series[1] of minor cleanup on GIC(v3) and genirq.
Changelog:
v1 -> v2
- patch[1/5]: Address the issue by Marc
- 1000000 -> USEC_PER_SEC;
- Vertical alignment of parameters;
- Update commit message;
- patch[2/5]: Address the issue by Marc
- gic_irq -> irqd_to_hwirq replacement;
- Convert the type of struct irq_data::hwirq to irq_hw_number_t;
- Some coding style improvements;
- Add Suggested-by from Marc;
- patch[3/5]: No change
- patch[4/5]:
- Add Acked-by from Marc;
- patch[5/5]: New
- As suggested by Marc, implement a dedicated API for irq_desc init
Dawei Li (5):
irqchip/gic-v3: Implement read polling with dedicated API
irqchip/gic(v3): Replace gic_irq() with irqd_to_hwirq()
genirq: Remove unneeded forward declaration
genirq: Initialize resend_node hlist for all irq_desc
genirq: Introduce generic init_desc()
drivers/irqchip/irq-gic-v3.c | 55 ++++++++---------
drivers/irqchip/irq-gic.c | 27 ++++-----
include/linux/irq.h | 2 +-
include/linux/irqhandler.h | 2 +-
kernel/irq/irqdesc.c | 112 ++++++++++++++++++++---------------
5 files changed, 102 insertions(+), 96 deletions(-)
Thanks,
Dawei
[1] v1:
https://lore.kernel.org/all/[email protected]/
--
2.27.0
Kernel provide read*_poll_* API family to support looping based polling
code pattern like below:
while (...)
{
val = op(addr);
condition = cond(val);
if (condition)
break;
/* Maybe some timeout handling stuff */
cpu_relax();
udelay();
}
As such, use readl_relaxed_poll_timeout_atomic() to implement atomic
register polling logic in gic-v3.
It's worth noting that this conversion would be impossilbe without
support of commit 7349a69cf312 ("iopoll: Do not use timekeeping in
read_poll_timeout_atomic()"), which remove time keeping code from
read_poll_timeout_atomic(), reason below:
Compared to other 'ordinary' device driver, IRQ chip driver is kinda
special, whose initialization(via init_IRQ()) happens pretty early,
even before timekeeping_init(). As a result, calling time keeping code
in irq chip init code is bogus.
Signed-off-by: Dawei Li <[email protected]>
---
drivers/irqchip/irq-gic-v3.c | 36 +++++++++++++++++-------------------
1 file changed, 17 insertions(+), 19 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 98b0329b7154..020a67195b16 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -19,6 +19,7 @@
#include <linux/percpu.h>
#include <linux/refcount.h>
#include <linux/slab.h>
+#include <linux/iopoll.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic-common.h>
@@ -251,17 +252,16 @@ static inline void __iomem *gic_dist_base(struct irq_data *d)
static void gic_do_wait_for_rwp(void __iomem *base, u32 bit)
{
- u32 count = 1000000; /* 1s! */
+ u32 val;
+ int ret;
- while (readl_relaxed(base + GICD_CTLR) & bit) {
- count--;
- if (!count) {
- pr_err_ratelimited("RWP timeout, gone fishing\n");
- return;
- }
- cpu_relax();
- udelay(1);
- }
+ ret = readl_relaxed_poll_timeout_atomic(base + GICD_CTLR,
+ val,
+ !(val & bit),
+ 1,
+ USEC_PER_SEC);
+ if (ret == -ETIMEDOUT)
+ pr_err_ratelimited("RWP timeout, gone fishing\n");
}
/* Wait for completion of a distributor change */
@@ -279,8 +279,8 @@ static void gic_redist_wait_for_rwp(void)
static void gic_enable_redist(bool enable)
{
void __iomem *rbase;
- u32 count = 1000000; /* 1s! */
u32 val;
+ int ret;
if (gic_data.flags & FLAGS_WORKAROUND_GICR_WAKER_MSM8996)
return;
@@ -301,14 +301,12 @@ static void gic_enable_redist(bool enable)
return; /* No PM support in this redistributor */
}
- while (--count) {
- val = readl_relaxed(rbase + GICR_WAKER);
- if (enable ^ (bool)(val & GICR_WAKER_ChildrenAsleep))
- break;
- cpu_relax();
- udelay(1);
- }
- if (!count)
+ ret = readl_relaxed_poll_timeout_atomic(rbase + GICR_WAKER,
+ val,
+ enable ^ (bool)(val & GICR_WAKER_ChildrenAsleep),
+ 1,
+ USEC_PER_SEC);
+ if (ret == -ETIMEDOUT)
pr_err_ratelimited("redistributor failed to %s...\n",
enable ? "wakeup" : "sleep");
}
--
2.27.0
For !CONFIG_SPARSE_IRQ kernel, early_irq_init() is supposed to
initialize all the desc entries in system, desc->resend_node
included.
Thus, initialize desc->resend_node for all irq_desc entries, rather
than irq_desc[0] only, which is the current implementation is about.
Fixes: bc06a9e08742 ("genirq: Use hlist for managing resend handlers")
Cc: [email protected]
Acked-by: Marc Zyngier <[email protected]>
Signed-off-by: Dawei Li <[email protected]>
---
kernel/irq/irqdesc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 27ca1c866f29..371eb1711d34 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -600,7 +600,7 @@ int __init early_irq_init(void)
mutex_init(&desc[i].request_mutex);
init_waitqueue_head(&desc[i].wait_for_threads);
desc_set_defaults(i, &desc[i], node, NULL, NULL);
- irq_resend_init(desc);
+ irq_resend_init(&desc[i]);
}
return arch_early_irq_init();
}
--
2.27.0
Protoype of irq_flow_handler_t is independent of irq_data, so
remove unneeded forward declaration.
Signed-off-by: Dawei Li <[email protected]>
---
include/linux/irqhandler.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/irqhandler.h b/include/linux/irqhandler.h
index c30f454a9518..72dd1eb3a0e7 100644
--- a/include/linux/irqhandler.h
+++ b/include/linux/irqhandler.h
@@ -8,7 +8,7 @@
*/
struct irq_desc;
-struct irq_data;
+
typedef void (*irq_flow_handler_t)(struct irq_desc *desc);
#endif
--
2.27.0
GIC & GIC-v3 share same gic_irq() implementations, both of which serve
exact same purpose of irqd_to_hwirq(). irqd_to_hwirq() is a generic and
top level API of interrupt subsystem, it's independent of any chip
implementation.
As such, replace gic_irq() with irqd_to_hwirq() in gic(v3) code.
Also, convert the type of struct irq_data::hwirq to irq_hw_number_t
explicitly.
Suggested-by: Marc Zyngier <[email protected]>
Signed-off-by: Dawei Li <[email protected]>
---
drivers/irqchip/irq-gic-v3.c | 19 +++++++------------
drivers/irqchip/irq-gic.c | 27 ++++++++++++---------------
include/linux/irq.h | 2 +-
3 files changed, 20 insertions(+), 28 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 020a67195b16..b6cc23e393e3 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -181,11 +181,6 @@ static enum gic_intid_range get_intid_range(struct irq_data *d)
return __get_intid_range(d->hwirq);
}
-static inline unsigned int gic_irq(struct irq_data *d)
-{
- return d->hwirq;
-}
-
static inline bool gic_irq_in_rdist(struct irq_data *d)
{
switch (get_intid_range(d)) {
@@ -546,7 +541,7 @@ static int gic_irq_nmi_setup(struct irq_data *d)
* A secondary irq_chip should be in charge of LPI request,
* it should not be possible to get there
*/
- if (WARN_ON(gic_irq(d) >= 8192))
+ if (WARN_ON(irqd_to_hwirq(d) >= 8192))
return -EINVAL;
/* desc lock should already be held */
@@ -586,7 +581,7 @@ static void gic_irq_nmi_teardown(struct irq_data *d)
* A secondary irq_chip should be in charge of LPI request,
* it should not be possible to get there
*/
- if (WARN_ON(gic_irq(d) >= 8192))
+ if (WARN_ON(irqd_to_hwirq(d) >= 8192))
return;
/* desc lock should already be held */
@@ -624,7 +619,7 @@ static bool gic_arm64_erratum_2941627_needed(struct irq_data *d)
static void gic_eoi_irq(struct irq_data *d)
{
- write_gicreg(gic_irq(d), ICC_EOIR1_EL1);
+ write_gicreg(irqd_to_hwirq(d), ICC_EOIR1_EL1);
isb();
if (gic_arm64_erratum_2941627_needed(d)) {
@@ -644,19 +639,19 @@ static void gic_eoimode1_eoi_irq(struct irq_data *d)
* No need to deactivate an LPI, or an interrupt that
* is is getting forwarded to a vcpu.
*/
- if (gic_irq(d) >= 8192 || irqd_is_forwarded_to_vcpu(d))
+ if (irqd_to_hwirq(d) >= 8192 || irqd_is_forwarded_to_vcpu(d))
return;
if (!gic_arm64_erratum_2941627_needed(d))
- gic_write_dir(gic_irq(d));
+ gic_write_dir(irqd_to_hwirq(d));
else
gic_poke_irq(d, GICD_ICACTIVER);
}
static int gic_set_type(struct irq_data *d, unsigned int type)
{
+ irq_hw_number_t irq = irqd_to_hwirq(d);
enum gic_intid_range range;
- unsigned int irq = gic_irq(d);
void __iomem *base;
u32 offset, index;
int ret;
@@ -682,7 +677,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
ret = gic_configure_irq(index, type, base + offset, NULL);
if (ret && (range == PPI_RANGE || range == EPPI_RANGE)) {
/* Misconfigured PPIs are usually not fatal */
- pr_warn("GIC: PPI INTID%d is secure or misconfigured\n", irq);
+ pr_warn("GIC: PPI INTID%ld is secure or misconfigured\n", irq);
ret = 0;
}
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 412196a7dad5..98aa383e39db 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -162,11 +162,6 @@ static inline void __iomem *gic_cpu_base(struct irq_data *d)
return gic_data_cpu_base(gic_data);
}
-static inline unsigned int gic_irq(struct irq_data *d)
-{
- return d->hwirq;
-}
-
static inline bool cascading_gic_irq(struct irq_data *d)
{
void *data = irq_data_get_irq_handler_data(d);
@@ -183,14 +178,16 @@ static inline bool cascading_gic_irq(struct irq_data *d)
*/
static void gic_poke_irq(struct irq_data *d, u32 offset)
{
- u32 mask = 1 << (gic_irq(d) % 32);
- writel_relaxed(mask, gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4);
+ u32 mask = 1 << (irqd_to_hwirq(d) % 32);
+
+ writel_relaxed(mask, gic_dist_base(d) + offset + (irqd_to_hwirq(d) / 32) * 4);
}
static int gic_peek_irq(struct irq_data *d, u32 offset)
{
- u32 mask = 1 << (gic_irq(d) % 32);
- return !!(readl_relaxed(gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4) & mask);
+ u32 mask = 1 << (irqd_to_hwirq(d) % 32);
+
+ return !!(readl_relaxed(gic_dist_base(d) + offset + (irqd_to_hwirq(d) / 32) * 4) & mask);
}
static void gic_mask_irq(struct irq_data *d)
@@ -220,7 +217,7 @@ static void gic_unmask_irq(struct irq_data *d)
static void gic_eoi_irq(struct irq_data *d)
{
- u32 hwirq = gic_irq(d);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
if (hwirq < 16)
hwirq = this_cpu_read(sgi_intid);
@@ -230,7 +227,7 @@ static void gic_eoi_irq(struct irq_data *d)
static void gic_eoimode1_eoi_irq(struct irq_data *d)
{
- u32 hwirq = gic_irq(d);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
/* Do not deactivate an IRQ forwarded to a vcpu. */
if (irqd_is_forwarded_to_vcpu(d))
@@ -293,8 +290,8 @@ static int gic_irq_get_irqchip_state(struct irq_data *d,
static int gic_set_type(struct irq_data *d, unsigned int type)
{
+ irq_hw_number_t gicirq = irqd_to_hwirq(d);
void __iomem *base = gic_dist_base(d);
- unsigned int gicirq = gic_irq(d);
int ret;
/* Interrupt configuration for SGIs can't be changed */
@@ -309,7 +306,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
ret = gic_configure_irq(gicirq, type, base + GIC_DIST_CONFIG, NULL);
if (ret && gicirq < 32) {
/* Misconfigured PPIs are usually not fatal */
- pr_warn("GIC: PPI%d is secure or misconfigured\n", gicirq - 16);
+ pr_warn("GIC: PPI%ld is secure or misconfigured\n", gicirq - 16);
ret = 0;
}
@@ -319,7 +316,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
static int gic_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)
{
/* Only interrupts on the primary GIC can be forwarded to a vcpu. */
- if (cascading_gic_irq(d) || gic_irq(d) < 16)
+ if (cascading_gic_irq(d) || irqd_to_hwirq(d) < 16)
return -EINVAL;
if (vcpu)
@@ -796,7 +793,7 @@ static void rmw_writeb(u8 bval, void __iomem *addr)
static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
bool force)
{
- void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + gic_irq(d);
+ void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + irqd_to_hwirq(d);
struct gic_chip_data *gic = irq_data_get_irq_chip_data(d);
unsigned int cpu;
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 90081afa10ce..97baa937ab5b 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -179,7 +179,7 @@ struct irq_common_data {
struct irq_data {
u32 mask;
unsigned int irq;
- unsigned long hwirq;
+ irq_hw_number_t hwirq;
struct irq_common_data *common;
struct irq_chip *chip;
struct irq_domain *domain;
--
2.27.0
Those two functions below share much duplicated codes for
initialization for irq_desc:
- alloc_desc() CONFIG_SPARSE_IRQ
- early_irq_init() !CONFIG_SPARSE_IRQ
Implement a dedicated function to fully initialize irq_desc.
Also, make free_masks() global to implement exception handling in
early_irq_init.
Suggested-by: Marc Zyngier <[email protected]>
Signed-off-by: Dawei Li <[email protected]>
---
kernel/irq/irqdesc.c | 112 ++++++++++++++++++++++++-------------------
1 file changed, 64 insertions(+), 48 deletions(-)
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 371eb1711d34..4c6b32318ce3 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -92,11 +92,23 @@ static void desc_smp_init(struct irq_desc *desc, int node,
#endif
}
+static void free_masks(struct irq_desc *desc)
+{
+#ifdef CONFIG_GENERIC_PENDING_IRQ
+ free_cpumask_var(desc->pending_mask);
+#endif
+ free_cpumask_var(desc->irq_common_data.affinity);
+#ifdef CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK
+ free_cpumask_var(desc->irq_common_data.effective_affinity);
+#endif
+}
+
#else
static inline int
alloc_masks(struct irq_desc *desc, int node) { return 0; }
static inline void
desc_smp_init(struct irq_desc *desc, int node, const struct cpumask *affinity) { }
+static inline void free_masks(struct irq_desc *desc) { }
#endif
static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node,
@@ -165,6 +177,39 @@ static void delete_irq_desc(unsigned int irq)
mas_erase(&mas);
}
+#ifdef CONFIG_SPARSE_IRQ
+static const struct kobj_type irq_kobj_type;
+#endif
+
+static int init_desc(struct irq_desc *desc, int irq, int node,
+ unsigned int flags,
+ const struct cpumask *affinity,
+ struct module *owner)
+{
+ desc->kstat_irqs = alloc_percpu(unsigned int);
+ if (!desc->kstat_irqs)
+ return -ENOMEM;
+
+ if (alloc_masks(desc, node)) {
+ free_percpu(desc->kstat_irqs);
+ return -ENOMEM;
+ }
+
+ raw_spin_lock_init(&desc->lock);
+ lockdep_set_class(&desc->lock, &irq_desc_lock_class);
+ mutex_init(&desc->request_mutex);
+ init_waitqueue_head(&desc->wait_for_threads);
+ desc_set_defaults(irq, desc, node, affinity, owner);
+ irqd_set(&desc->irq_data, flags);
+ irq_resend_init(desc);
+#ifdef CONFIG_SPARSE_IRQ
+ kobject_init(&desc->kobj, &irq_kobj_type);
+ init_rcu_head(&desc->rcu);
+#endif
+
+ return 0;
+}
+
#ifdef CONFIG_SPARSE_IRQ
static void irq_kobj_release(struct kobject *kobj);
@@ -384,21 +429,6 @@ struct irq_desc *irq_to_desc(unsigned int irq)
EXPORT_SYMBOL_GPL(irq_to_desc);
#endif
-#ifdef CONFIG_SMP
-static void free_masks(struct irq_desc *desc)
-{
-#ifdef CONFIG_GENERIC_PENDING_IRQ
- free_cpumask_var(desc->pending_mask);
-#endif
- free_cpumask_var(desc->irq_common_data.affinity);
-#ifdef CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK
- free_cpumask_var(desc->irq_common_data.effective_affinity);
-#endif
-}
-#else
-static inline void free_masks(struct irq_desc *desc) { }
-#endif
-
void irq_lock_sparse(void)
{
mutex_lock(&sparse_irq_lock);
@@ -414,36 +444,19 @@ static struct irq_desc *alloc_desc(int irq, int node, unsigned int flags,
struct module *owner)
{
struct irq_desc *desc;
+ int ret;
desc = kzalloc_node(sizeof(*desc), GFP_KERNEL, node);
if (!desc)
return NULL;
- /* allocate based on nr_cpu_ids */
- desc->kstat_irqs = alloc_percpu(unsigned int);
- if (!desc->kstat_irqs)
- goto err_desc;
-
- if (alloc_masks(desc, node))
- goto err_kstat;
- raw_spin_lock_init(&desc->lock);
- lockdep_set_class(&desc->lock, &irq_desc_lock_class);
- mutex_init(&desc->request_mutex);
- init_rcu_head(&desc->rcu);
- init_waitqueue_head(&desc->wait_for_threads);
-
- desc_set_defaults(irq, desc, node, affinity, owner);
- irqd_set(&desc->irq_data, flags);
- kobject_init(&desc->kobj, &irq_kobj_type);
- irq_resend_init(desc);
+ ret = init_desc(desc, irq, node, flags, affinity, owner);
+ if (unlikely(ret)) {
+ kfree(desc);
+ return NULL;
+ }
return desc;
-
-err_kstat:
- free_percpu(desc->kstat_irqs);
-err_desc:
- kfree(desc);
- return NULL;
}
static void irq_kobj_release(struct kobject *kobj)
@@ -583,26 +596,29 @@ struct irq_desc irq_desc[NR_IRQS] __cacheline_aligned_in_smp = {
int __init early_irq_init(void)
{
int count, i, node = first_online_node;
- struct irq_desc *desc;
+ int ret;
init_irq_default_affinity();
printk(KERN_INFO "NR_IRQS: %d\n", NR_IRQS);
- desc = irq_desc;
count = ARRAY_SIZE(irq_desc);
for (i = 0; i < count; i++) {
- desc[i].kstat_irqs = alloc_percpu(unsigned int);
- alloc_masks(&desc[i], node);
- raw_spin_lock_init(&desc[i].lock);
- lockdep_set_class(&desc[i].lock, &irq_desc_lock_class);
- mutex_init(&desc[i].request_mutex);
- init_waitqueue_head(&desc[i].wait_for_threads);
- desc_set_defaults(i, &desc[i], node, NULL, NULL);
- irq_resend_init(&desc[i]);
+ ret = init_desc(irq_desc + i, i, node, 0, NULL, NULL);
+ if (unlikely(ret))
+ goto __free_desc_res;
}
+
return arch_early_irq_init();
+
+__free_desc_res:
+ while (--i >= 0) {
+ free_masks(irq_desc + i);
+ free_percpu(irq_desc[i].kstat_irqs);
+ }
+
+ return ret;
}
struct irq_desc *irq_to_desc(unsigned int irq)
--
2.27.0
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 9676635685fe348003a29948d9726e5d9e4b4a6e
Gitweb: https://git.kernel.org/tip/9676635685fe348003a29948d9726e5d9e4b4a6e
Author: Dawei Li <[email protected]>
AuthorDate: Mon, 22 Jan 2024 16:57:14 +08:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Wed, 24 Jan 2024 16:02:48 +01:00
genirq: Remove unneeded forward declaration
The protoype of irq_flow_handler_t is independent of irq_data, so remove
unneeded forward declaration.
Signed-off-by: Dawei Li <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
include/linux/irqhandler.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/irqhandler.h b/include/linux/irqhandler.h
index c30f454..72dd1eb 100644
--- a/include/linux/irqhandler.h
+++ b/include/linux/irqhandler.h
@@ -8,7 +8,7 @@
*/
struct irq_desc;
-struct irq_data;
+
typedef void (*irq_flow_handler_t)(struct irq_desc *desc);
#endif
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 22653244a9fed06f2f864b44808a85bf5c4e3ef2
Gitweb: https://git.kernel.org/tip/22653244a9fed06f2f864b44808a85bf5c4e3ef2
Author: Dawei Li <[email protected]>
AuthorDate: Mon, 22 Jan 2024 16:57:16 +08:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Wed, 24 Jan 2024 16:02:48 +01:00
genirq: Deduplicate interrupt descriptor initialization
alloc_desc() and early_irq_init() contain duplicated code to initialize
interrupt descriptors.
Replace that with a helper function.
Suggested-by: Marc Zyngier <[email protected]>
Signed-off-by: Dawei Li <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
kernel/irq/irqdesc.c | 112 ++++++++++++++++++++++++------------------
1 file changed, 64 insertions(+), 48 deletions(-)
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 371eb17..4c6b323 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -92,11 +92,23 @@ static void desc_smp_init(struct irq_desc *desc, int node,
#endif
}
+static void free_masks(struct irq_desc *desc)
+{
+#ifdef CONFIG_GENERIC_PENDING_IRQ
+ free_cpumask_var(desc->pending_mask);
+#endif
+ free_cpumask_var(desc->irq_common_data.affinity);
+#ifdef CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK
+ free_cpumask_var(desc->irq_common_data.effective_affinity);
+#endif
+}
+
#else
static inline int
alloc_masks(struct irq_desc *desc, int node) { return 0; }
static inline void
desc_smp_init(struct irq_desc *desc, int node, const struct cpumask *affinity) { }
+static inline void free_masks(struct irq_desc *desc) { }
#endif
static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node,
@@ -166,6 +178,39 @@ static void delete_irq_desc(unsigned int irq)
}
#ifdef CONFIG_SPARSE_IRQ
+static const struct kobj_type irq_kobj_type;
+#endif
+
+static int init_desc(struct irq_desc *desc, int irq, int node,
+ unsigned int flags,
+ const struct cpumask *affinity,
+ struct module *owner)
+{
+ desc->kstat_irqs = alloc_percpu(unsigned int);
+ if (!desc->kstat_irqs)
+ return -ENOMEM;
+
+ if (alloc_masks(desc, node)) {
+ free_percpu(desc->kstat_irqs);
+ return -ENOMEM;
+ }
+
+ raw_spin_lock_init(&desc->lock);
+ lockdep_set_class(&desc->lock, &irq_desc_lock_class);
+ mutex_init(&desc->request_mutex);
+ init_waitqueue_head(&desc->wait_for_threads);
+ desc_set_defaults(irq, desc, node, affinity, owner);
+ irqd_set(&desc->irq_data, flags);
+ irq_resend_init(desc);
+#ifdef CONFIG_SPARSE_IRQ
+ kobject_init(&desc->kobj, &irq_kobj_type);
+ init_rcu_head(&desc->rcu);
+#endif
+
+ return 0;
+}
+
+#ifdef CONFIG_SPARSE_IRQ
static void irq_kobj_release(struct kobject *kobj);
@@ -384,21 +429,6 @@ struct irq_desc *irq_to_desc(unsigned int irq)
EXPORT_SYMBOL_GPL(irq_to_desc);
#endif
-#ifdef CONFIG_SMP
-static void free_masks(struct irq_desc *desc)
-{
-#ifdef CONFIG_GENERIC_PENDING_IRQ
- free_cpumask_var(desc->pending_mask);
-#endif
- free_cpumask_var(desc->irq_common_data.affinity);
-#ifdef CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK
- free_cpumask_var(desc->irq_common_data.effective_affinity);
-#endif
-}
-#else
-static inline void free_masks(struct irq_desc *desc) { }
-#endif
-
void irq_lock_sparse(void)
{
mutex_lock(&sparse_irq_lock);
@@ -414,36 +444,19 @@ static struct irq_desc *alloc_desc(int irq, int node, unsigned int flags,
struct module *owner)
{
struct irq_desc *desc;
+ int ret;
desc = kzalloc_node(sizeof(*desc), GFP_KERNEL, node);
if (!desc)
return NULL;
- /* allocate based on nr_cpu_ids */
- desc->kstat_irqs = alloc_percpu(unsigned int);
- if (!desc->kstat_irqs)
- goto err_desc;
-
- if (alloc_masks(desc, node))
- goto err_kstat;
- raw_spin_lock_init(&desc->lock);
- lockdep_set_class(&desc->lock, &irq_desc_lock_class);
- mutex_init(&desc->request_mutex);
- init_rcu_head(&desc->rcu);
- init_waitqueue_head(&desc->wait_for_threads);
-
- desc_set_defaults(irq, desc, node, affinity, owner);
- irqd_set(&desc->irq_data, flags);
- kobject_init(&desc->kobj, &irq_kobj_type);
- irq_resend_init(desc);
+ ret = init_desc(desc, irq, node, flags, affinity, owner);
+ if (unlikely(ret)) {
+ kfree(desc);
+ return NULL;
+ }
return desc;
-
-err_kstat:
- free_percpu(desc->kstat_irqs);
-err_desc:
- kfree(desc);
- return NULL;
}
static void irq_kobj_release(struct kobject *kobj)
@@ -583,26 +596,29 @@ struct irq_desc irq_desc[NR_IRQS] __cacheline_aligned_in_smp = {
int __init early_irq_init(void)
{
int count, i, node = first_online_node;
- struct irq_desc *desc;
+ int ret;
init_irq_default_affinity();
printk(KERN_INFO "NR_IRQS: %d\n", NR_IRQS);
- desc = irq_desc;
count = ARRAY_SIZE(irq_desc);
for (i = 0; i < count; i++) {
- desc[i].kstat_irqs = alloc_percpu(unsigned int);
- alloc_masks(&desc[i], node);
- raw_spin_lock_init(&desc[i].lock);
- lockdep_set_class(&desc[i].lock, &irq_desc_lock_class);
- mutex_init(&desc[i].request_mutex);
- init_waitqueue_head(&desc[i].wait_for_threads);
- desc_set_defaults(i, &desc[i], node, NULL, NULL);
- irq_resend_init(&desc[i]);
+ ret = init_desc(irq_desc + i, i, node, 0, NULL, NULL);
+ if (unlikely(ret))
+ goto __free_desc_res;
}
+
return arch_early_irq_init();
+
+__free_desc_res:
+ while (--i >= 0) {
+ free_masks(irq_desc + i);
+ free_percpu(irq_desc[i].kstat_irqs);
+ }
+
+ return ret;
}
struct irq_desc *irq_to_desc(unsigned int irq)
The following commit has been merged into the irq/core branch of tip:
Commit-ID: a0c446dc4d9365a24d81f2ee024bdde46e40365f
Gitweb: https://git.kernel.org/tip/a0c446dc4d9365a24d81f2ee024bdde46e40365f
Author: Dawei Li <[email protected]>
AuthorDate: Mon, 22 Jan 2024 16:57:12 +08:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Wed, 24 Jan 2024 16:02:48 +01:00
irqchip/gic-v3: Use readl_relaxed_poll_timeout_atomic()
Replace the open coded register polling loop with
readl_relaxed_poll_timeout_atomic() which provides the same functionality.
Signed-off-by: Dawei Li <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
drivers/irqchip/irq-gic-v3.c | 32 +++++++++++++-------------------
1 file changed, 13 insertions(+), 19 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 98b0329..65cbf37 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -19,6 +19,7 @@
#include <linux/percpu.h>
#include <linux/refcount.h>
#include <linux/slab.h>
+#include <linux/iopoll.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic-common.h>
@@ -251,17 +252,13 @@ static inline void __iomem *gic_dist_base(struct irq_data *d)
static void gic_do_wait_for_rwp(void __iomem *base, u32 bit)
{
- u32 count = 1000000; /* 1s! */
+ u32 val;
+ int ret;
- while (readl_relaxed(base + GICD_CTLR) & bit) {
- count--;
- if (!count) {
- pr_err_ratelimited("RWP timeout, gone fishing\n");
- return;
- }
- cpu_relax();
- udelay(1);
- }
+ ret = readl_relaxed_poll_timeout_atomic(base + GICD_CTLR, val, !(val & bit),
+ 1, USEC_PER_SEC);
+ if (ret == -ETIMEDOUT)
+ pr_err_ratelimited("RWP timeout, gone fishing\n");
}
/* Wait for completion of a distributor change */
@@ -279,8 +276,8 @@ static void gic_redist_wait_for_rwp(void)
static void gic_enable_redist(bool enable)
{
void __iomem *rbase;
- u32 count = 1000000; /* 1s! */
u32 val;
+ int ret;
if (gic_data.flags & FLAGS_WORKAROUND_GICR_WAKER_MSM8996)
return;
@@ -301,16 +298,13 @@ static void gic_enable_redist(bool enable)
return; /* No PM support in this redistributor */
}
- while (--count) {
- val = readl_relaxed(rbase + GICR_WAKER);
- if (enable ^ (bool)(val & GICR_WAKER_ChildrenAsleep))
- break;
- cpu_relax();
- udelay(1);
- }
- if (!count)
+ ret = readl_relaxed_poll_timeout_atomic(rbase + GICR_WAKER, val,
+ enable ^ (bool)(val & GICR_WAKER_ChildrenAsleep),
+ 1, USEC_PER_SEC);
+ if (ret == -ETIMEDOUT) {
pr_err_ratelimited("redistributor failed to %s...\n",
enable ? "wakeup" : "sleep");
+ }
}
/*
The following commit has been merged into the irq/core branch of tip:
Commit-ID: d22083a5f09b2066728a91f3abb71284451247b1
Gitweb: https://git.kernel.org/tip/d22083a5f09b2066728a91f3abb71284451247b1
Author: Dawei Li <[email protected]>
AuthorDate: Mon, 22 Jan 2024 16:57:13 +08:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Wed, 24 Jan 2024 16:02:48 +01:00
irqchip/gic(v3): Replace gic_irq() with irqd_to_hwirq()
GIC & GIC-v3 share same gic_irq() implementations, both of which serve
exact same purpose as irqd_to_hwirq(). irqd_to_hwirq() is a generic and
top level API of the interrupt subsystem, it's independent of any chip
implementation.
Replace gic_irq() with irqd_to_hwirq() and convert struct irq_data::hwirq
to irq_hw_number_t explicitly.
Suggested-by: Marc Zyngier <[email protected]>
Signed-off-by: Dawei Li <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
drivers/irqchip/irq-gic-v3.c | 19 +++++++------------
drivers/irqchip/irq-gic.c | 27 ++++++++++++---------------
include/linux/irq.h | 2 +-
3 files changed, 20 insertions(+), 28 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 65cbf37..20a75f0 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -181,11 +181,6 @@ static enum gic_intid_range get_intid_range(struct irq_data *d)
return __get_intid_range(d->hwirq);
}
-static inline unsigned int gic_irq(struct irq_data *d)
-{
- return d->hwirq;
-}
-
static inline bool gic_irq_in_rdist(struct irq_data *d)
{
switch (get_intid_range(d)) {
@@ -542,7 +537,7 @@ static int gic_irq_nmi_setup(struct irq_data *d)
* A secondary irq_chip should be in charge of LPI request,
* it should not be possible to get there
*/
- if (WARN_ON(gic_irq(d) >= 8192))
+ if (WARN_ON(irqd_to_hwirq(d) >= 8192))
return -EINVAL;
/* desc lock should already be held */
@@ -582,7 +577,7 @@ static void gic_irq_nmi_teardown(struct irq_data *d)
* A secondary irq_chip should be in charge of LPI request,
* it should not be possible to get there
*/
- if (WARN_ON(gic_irq(d) >= 8192))
+ if (WARN_ON(irqd_to_hwirq(d) >= 8192))
return;
/* desc lock should already be held */
@@ -620,7 +615,7 @@ static bool gic_arm64_erratum_2941627_needed(struct irq_data *d)
static void gic_eoi_irq(struct irq_data *d)
{
- write_gicreg(gic_irq(d), ICC_EOIR1_EL1);
+ write_gicreg(irqd_to_hwirq(d), ICC_EOIR1_EL1);
isb();
if (gic_arm64_erratum_2941627_needed(d)) {
@@ -640,19 +635,19 @@ static void gic_eoimode1_eoi_irq(struct irq_data *d)
* No need to deactivate an LPI, or an interrupt that
* is is getting forwarded to a vcpu.
*/
- if (gic_irq(d) >= 8192 || irqd_is_forwarded_to_vcpu(d))
+ if (irqd_to_hwirq(d) >= 8192 || irqd_is_forwarded_to_vcpu(d))
return;
if (!gic_arm64_erratum_2941627_needed(d))
- gic_write_dir(gic_irq(d));
+ gic_write_dir(irqd_to_hwirq(d));
else
gic_poke_irq(d, GICD_ICACTIVER);
}
static int gic_set_type(struct irq_data *d, unsigned int type)
{
+ irq_hw_number_t irq = irqd_to_hwirq(d);
enum gic_intid_range range;
- unsigned int irq = gic_irq(d);
void __iomem *base;
u32 offset, index;
int ret;
@@ -678,7 +673,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
ret = gic_configure_irq(index, type, base + offset, NULL);
if (ret && (range == PPI_RANGE || range == EPPI_RANGE)) {
/* Misconfigured PPIs are usually not fatal */
- pr_warn("GIC: PPI INTID%d is secure or misconfigured\n", irq);
+ pr_warn("GIC: PPI INTID%ld is secure or misconfigured\n", irq);
ret = 0;
}
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 412196a..98aa383 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -162,11 +162,6 @@ static inline void __iomem *gic_cpu_base(struct irq_data *d)
return gic_data_cpu_base(gic_data);
}
-static inline unsigned int gic_irq(struct irq_data *d)
-{
- return d->hwirq;
-}
-
static inline bool cascading_gic_irq(struct irq_data *d)
{
void *data = irq_data_get_irq_handler_data(d);
@@ -183,14 +178,16 @@ static inline bool cascading_gic_irq(struct irq_data *d)
*/
static void gic_poke_irq(struct irq_data *d, u32 offset)
{
- u32 mask = 1 << (gic_irq(d) % 32);
- writel_relaxed(mask, gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4);
+ u32 mask = 1 << (irqd_to_hwirq(d) % 32);
+
+ writel_relaxed(mask, gic_dist_base(d) + offset + (irqd_to_hwirq(d) / 32) * 4);
}
static int gic_peek_irq(struct irq_data *d, u32 offset)
{
- u32 mask = 1 << (gic_irq(d) % 32);
- return !!(readl_relaxed(gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4) & mask);
+ u32 mask = 1 << (irqd_to_hwirq(d) % 32);
+
+ return !!(readl_relaxed(gic_dist_base(d) + offset + (irqd_to_hwirq(d) / 32) * 4) & mask);
}
static void gic_mask_irq(struct irq_data *d)
@@ -220,7 +217,7 @@ static void gic_unmask_irq(struct irq_data *d)
static void gic_eoi_irq(struct irq_data *d)
{
- u32 hwirq = gic_irq(d);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
if (hwirq < 16)
hwirq = this_cpu_read(sgi_intid);
@@ -230,7 +227,7 @@ static void gic_eoi_irq(struct irq_data *d)
static void gic_eoimode1_eoi_irq(struct irq_data *d)
{
- u32 hwirq = gic_irq(d);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
/* Do not deactivate an IRQ forwarded to a vcpu. */
if (irqd_is_forwarded_to_vcpu(d))
@@ -293,8 +290,8 @@ static int gic_irq_get_irqchip_state(struct irq_data *d,
static int gic_set_type(struct irq_data *d, unsigned int type)
{
+ irq_hw_number_t gicirq = irqd_to_hwirq(d);
void __iomem *base = gic_dist_base(d);
- unsigned int gicirq = gic_irq(d);
int ret;
/* Interrupt configuration for SGIs can't be changed */
@@ -309,7 +306,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
ret = gic_configure_irq(gicirq, type, base + GIC_DIST_CONFIG, NULL);
if (ret && gicirq < 32) {
/* Misconfigured PPIs are usually not fatal */
- pr_warn("GIC: PPI%d is secure or misconfigured\n", gicirq - 16);
+ pr_warn("GIC: PPI%ld is secure or misconfigured\n", gicirq - 16);
ret = 0;
}
@@ -319,7 +316,7 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
static int gic_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)
{
/* Only interrupts on the primary GIC can be forwarded to a vcpu. */
- if (cascading_gic_irq(d) || gic_irq(d) < 16)
+ if (cascading_gic_irq(d) || irqd_to_hwirq(d) < 16)
return -EINVAL;
if (vcpu)
@@ -796,7 +793,7 @@ static void rmw_writeb(u8 bval, void __iomem *addr)
static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
bool force)
{
- void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + gic_irq(d);
+ void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + irqd_to_hwirq(d);
struct gic_chip_data *gic = irq_data_get_irq_chip_data(d);
unsigned int cpu;
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 90081af..97baa93 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -179,7 +179,7 @@ struct irq_common_data {
struct irq_data {
u32 mask;
unsigned int irq;
- unsigned long hwirq;
+ irq_hw_number_t hwirq;
struct irq_common_data *common;
struct irq_chip *chip;
struct irq_domain *domain;
The following commit has been merged into the irq/core branch of tip:
Commit-ID: b184c8c2889ceef0a137c7d0567ef9fe3d92276e
Gitweb: https://git.kernel.org/tip/b184c8c2889ceef0a137c7d0567ef9fe3d92276e
Author: Dawei Li <[email protected]>
AuthorDate: Mon, 22 Jan 2024 16:57:15 +08:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Wed, 24 Jan 2024 14:15:41 +01:00
genirq: Initialize resend_node hlist for all interrupt descriptors
For a CONFIG_SPARSE_IRQ=n kernel, early_irq_init() is supposed to
initialize all interrupt descriptors.
It does except for irq_desc::resend_node, which ia only initialized for the
first descriptor.
Use the indexed decriptor and not the base pointer to address that.
Fixes: bc06a9e08742 ("genirq: Use hlist for managing resend handlers")
Signed-off-by: Dawei Li <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/[email protected]
---
kernel/irq/irqdesc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 27ca1c8..371eb17 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -600,7 +600,7 @@ int __init early_irq_init(void)
mutex_init(&desc[i].request_mutex);
init_waitqueue_head(&desc[i].wait_for_threads);
desc_set_defaults(i, &desc[i], node, NULL, NULL);
- irq_resend_init(desc);
+ irq_resend_init(&desc[i]);
}
return arch_early_irq_init();
}