2016-03-17 14:19:36

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 00/15] Add support for Tegra210 AGIC

The Tegra210 AGIC interrupt controller is a 2nd level interrupt controller
located in a separate power domain to the main GIC interrupt controller.
It can route interrupts to the main CPU cluster or an Audio DSP slave.
This series only support routing interrupts to the main CPU cluster.

Ideally we would like to re-use the existing ARM GIC driver because the
AGIC is a GIC-400. However, in order to do so this requires adding
runtime power management support for irqchips and several significant
changes to the exisiting GIC driver for power management reasons.

Originally, this series was sent out as an RFC [0] and hopefully, I have
addressed all the comments and issues reported. I was/am tempted to
split this series into two series, one to add runtime-pm support for
irqchips (patches 1-8) and one to add support for the AGIC to the GIC
driver (patches 9-15). If this would be more manageable and easier, I
am happy to do so.

I have included a fix for OMAP (needs testing) that was reported by
Grygorii. I know it may seem odd to include in a series adding an
interrupt controller for Tegra, but oh well ...

This is work well so far on Tegra, but would love to get some more
testing on other platforms.

[0] http://marc.info/?l=linux-kernel&m=145034948920651&w=2

Jon Hunter (15):
ARM: tegra: Correct interrupt type for ARM TWD
ARM: OMAP: Correct interrupt type for ARM TWD
irqchip/gic: Don't unnecessarily write the IRQ configuration
irqchip/gic: WARN if setting the interrupt type fails
irqchip: Mask the non-type/sense bits when translating an IRQ
irqdomain: Ensure type settings match for an existing mapping
irqdomain: Don't set type when mapping an IRQ
genirq: Add runtime power management support for IRQ chips
irqchip/gic: Don't initialise chip if mapping IO space fails
irqchip/gic: Remove static irq_chip definition for eoimode1
irqchip/gic: Return an error if GIC initialisation fails
irqchip/gic: Pass GIC pointer to save/restore functions
irqchip/gic: Prepare for adding platform driver
dt-bindings: arm-gic: Drop 'clock-names' from binding document
irqchip/gic: Add support for tegra AGIC interrupt controller

.../bindings/interrupt-controller/arm,gic.txt | 12 +-
arch/arm/boot/dts/omap4.dtsi | 2 +-
arch/arm/boot/dts/tegra20.dtsi | 2 +-
arch/arm/boot/dts/tegra30.dtsi | 2 +-
drivers/irqchip/Kconfig | 1 +
drivers/irqchip/irq-crossbar.c | 2 +-
drivers/irqchip/irq-gic-common.c | 19 +-
drivers/irqchip/irq-gic-common.h | 2 +-
drivers/irqchip/irq-gic-v3.c | 6 +-
drivers/irqchip/irq-gic.c | 437 ++++++++++++++++-----
drivers/irqchip/irq-hip04.c | 5 +-
drivers/irqchip/irq-tegra.c | 2 +-
include/linux/irq.h | 5 +
include/linux/irqdomain.h | 3 +
kernel/irq/chip.c | 52 +++
kernel/irq/internals.h | 1 +
kernel/irq/irqdomain.c | 77 +++-
kernel/irq/manage.c | 21 +-
kernel/irq/pm.c | 3 +
19 files changed, 492 insertions(+), 162 deletions(-)

--
2.1.4


2016-03-17 14:19:39

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 01/15] ARM: tegra: Correct interrupt type for ARM TWD

The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
the ARM GIC documentation, whether the type for PPIs can be set is
IMPLEMENTATION DEFINED. For Tegra20/30 devices the PPI type cannot be
set and so when we attempt to set the type for the ARM TWD interrupt it
fails. This has done unnoticed because it fails silently and because we
cannot re-configure the type it has had no impact. Nevertheless fix the
type for the TWD interrupt so that it matches the hardware configuration.

Signed-off-by: Jon Hunter <[email protected]>

---
Ideally, we would not be attempting to set the type for an interrupt
where it cannot be programmed but this would require changes to the
device-tree bindings for the GIC. This series adds a WARNING to catch
any of these silent failures.
---
arch/arm/boot/dts/tegra20.dtsi | 2 +-
arch/arm/boot/dts/tegra30.dtsi | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/boot/dts/tegra20.dtsi b/arch/arm/boot/dts/tegra20.dtsi
index 8fb61b93c226..2207c08e3fa3 100644
--- a/arch/arm/boot/dts/tegra20.dtsi
+++ b/arch/arm/boot/dts/tegra20.dtsi
@@ -145,7 +145,7 @@
interrupt-parent = <&intc>;
reg = <0x50040600 0x20>;
interrupts = <GIC_PPI 13
- (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_HIGH)>;
+ (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_EDGE_RISING)>;
clocks = <&tegra_car TEGRA20_CLK_TWD>;
};

diff --git a/arch/arm/boot/dts/tegra30.dtsi b/arch/arm/boot/dts/tegra30.dtsi
index c6edc8cea34e..5030065cbdfe 100644
--- a/arch/arm/boot/dts/tegra30.dtsi
+++ b/arch/arm/boot/dts/tegra30.dtsi
@@ -230,7 +230,7 @@
reg = <0x50040600 0x20>;
interrupt-parent = <&intc>;
interrupts = <GIC_PPI 13
- (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
+ (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_EDGE_RISING)>;
clocks = <&tegra_car TEGRA30_CLK_TWD>;
};

--
2.1.4

2016-03-17 14:19:49

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 03/15] irqchip/gic: Don't unnecessarily write the IRQ configuration

If the interrupt configuration matches the current configuration, then
don't bother writing the configuration again.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic-common.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index f174ce0ca361..ffff5a45f1e3 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -50,13 +50,17 @@ int gic_configure_irq(unsigned int irq, unsigned int type,
else if (type & IRQ_TYPE_EDGE_BOTH)
val |= confmask;

+ /* If the current configuration is the same, then we are done */
+ if (val == oldval)
+ return 0;
+
/*
* Write back the new configuration, and possibly re-enable
- * the interrupt. If we tried to write a new configuration and failed,
+ * the interrupt. If we fail to write a new configuration,
* return an error.
*/
writel_relaxed(val, base + GIC_DIST_CONFIG + confoff);
- if (readl_relaxed(base + GIC_DIST_CONFIG + confoff) != val && val != oldval)
+ if (readl_relaxed(base + GIC_DIST_CONFIG + confoff) != val)
ret = -EINVAL;

if (sync_access)
--
2.1.4

2016-03-17 14:19:58

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 05/15] irqchip: Mask the non-type/sense bits when translating an IRQ

The firmware parameter that contains the IRQ sense bits may also contain
other data. When return the IRQ type, bits outside of these sense bits
should be masked. If these bits are not masked and
irq_create_fwspec_mapping() is called to map an IRQ, then the comparison
of the type returned from irq_domain_translate() will never match
that returned by irq_get_trigger_type() (because this function masks the
none sense bits) and so we will always call irq_set_irq_type() to program
the type even if it was not really necessary.

Currently, the downside to this is unnecessarily re-programmming the type
but nevertheless this should be avoided.

The Tegra LIC, TI Crossbar and GIC-V3 irqchips all have client instances
(from reviewing the device-tree sources) where bits outside the IRQ sense
bits are set, but do not mask these bits. Therefore, ensure these bits
are masked for these irqchips.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-crossbar.c | 2 +-
drivers/irqchip/irq-gic-v3.c | 2 +-
drivers/irqchip/irq-tegra.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-crossbar.c b/drivers/irqchip/irq-crossbar.c
index 75573fa431ba..1eef56a89b1f 100644
--- a/drivers/irqchip/irq-crossbar.c
+++ b/drivers/irqchip/irq-crossbar.c
@@ -183,7 +183,7 @@ static int crossbar_domain_translate(struct irq_domain *d,
return -EINVAL;

*hwirq = fwspec->param[1];
- *type = fwspec->param[2];
+ *type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
return 0;
}

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index c569c466fa31..972335b32eae 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -777,7 +777,7 @@ static int gic_irq_domain_translate(struct irq_domain *d,
return -EINVAL;

*hwirq = fwspec->param[0];
- *type = fwspec->param[1];
+ *type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
return 0;
}

diff --git a/drivers/irqchip/irq-tegra.c b/drivers/irqchip/irq-tegra.c
index 121ec301372e..7ceaff099072 100644
--- a/drivers/irqchip/irq-tegra.c
+++ b/drivers/irqchip/irq-tegra.c
@@ -235,7 +235,7 @@ static int tegra_ictlr_domain_translate(struct irq_domain *d,
return -EINVAL;

*hwirq = fwspec->param[1];
- *type = fwspec->param[2];
+ *type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
return 0;
}

--
2.1.4

2016-03-17 14:20:07

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 07/15] irqdomain: Don't set type when mapping an IRQ

Some IRQ chips, such as GPIO controllers or secondary level interrupt
controllers, may require require additional runtime power management
control to ensure they are accessible. For such IRQ chips, it makes sense
to enable the IRQ chip when interrupts are requested and disabled them
again once all interrupts have been freed.

When mapping an IRQ, the IRQ type settings are read and then programmed.
The mapping of the IRQ happens before the IRQ is requested and so the
programming of the type settings occurs before the IRQ is requested. This
is a problem for IRQ chips that require additional power management
control because they may not be accessible yet. Therefore, when mapping
the IRQ, don't program the type settings, just save them and then program
these saved settings when the IRQ is requested (so long as if they are not
overridden via the call to request the IRQ).

Add a stub function for irq_domain_free_irqs() to avoid any compilation
errors when CONFIG_IRQ_DOMAIN_HIERARCHY is not selected.

Signed-off-by: Jon Hunter <[email protected]>
---
include/linux/irqdomain.h | 3 +++
kernel/irq/irqdomain.c | 18 ++++++++++++++----
kernel/irq/manage.c | 7 +++++++
3 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index 2aed04396210..fc66876d1965 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -440,6 +440,9 @@ static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
return -1;
}

+static inline void irq_domain_free_irqs(unsigned int virq,
+ unsigned int nr_irqs) { }
+
static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
{
return false;
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 0ea285baa619..332bee2205e0 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -576,6 +576,7 @@ static void of_phandle_args_to_fwspec(struct of_phandle_args *irq_data,
unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
{
struct irq_domain *domain;
+ struct irq_data *irq_data;
irq_hw_number_t hwirq;
unsigned int cur_type = IRQ_TYPE_NONE;
unsigned int type = IRQ_TYPE_NONE;
@@ -638,10 +639,19 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
return 0;
}

- /* Set type if specified and different than the current one */
- if (type != IRQ_TYPE_NONE &&
- type != irq_get_trigger_type(virq))
- irq_set_irq_type(virq, type);
+ irq_data = irq_get_irq_data(virq);
+ if (!irq_data) {
+ if (irq_domain_is_hierarchy(domain))
+ irq_domain_free_irqs(virq, 1);
+ else
+ irq_dispose_mapping(virq);
+
+ return 0;
+ }
+
+ /* Store trigger type */
+ irqd_set_trigger_type(irq_data, type);
+
return virq;
}
EXPORT_SYMBOL_GPL(irq_create_fwspec_mapping);
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index cc1cc641d653..b2a93a37f772 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -1117,6 +1117,13 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
new->irq = irq;

/*
+ * If the trigger type is not specified by the caller,
+ * then use the default for this interrupt.
+ */
+ if (!(new->flags & IRQF_TRIGGER_MASK))
+ new->flags |= irqd_get_trigger_type(&desc->irq_data);
+
+ /*
* Check whether the interrupt nests into another interrupt
* thread.
*/
--
2.1.4

2016-03-17 14:20:14

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 09/15] irqchip/gic: Don't initialise chip if mapping IO space fails

If we fail to map the address space for the GIC distributor or CPU
interface, then don't attempt to initialise the chip, just WARN and
return.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 6c555a2c5315..a4a13ef35c1b 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -1203,10 +1203,14 @@ gic_of_init(struct device_node *node, struct device_node *parent)
return -ENODEV;

dist_base = of_iomap(node, 0);
- WARN(!dist_base, "unable to map gic dist registers\n");
+ if (WARN(!dist_base, "unable to map gic dist registers\n"))
+ return -ENOMEM;

cpu_base = of_iomap(node, 1);
- WARN(!cpu_base, "unable to map gic cpu registers\n");
+ if (WARN(!cpu_base, "unable to map gic cpu registers\n")) {
+ iounmap(dist_base);
+ return -ENOMEM;
+ }

/*
* Disable split EOI/Deactivate if either HYP is not available
--
2.1.4

2016-03-17 14:20:22

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 10/15] irqchip/gic: Remove static irq_chip definition for eoimode1

There are only 3 differences (not including the name) in the definitions
of the gic_chip and gic_eoimode1_chip structures. Instead of statically
defining the gic_eoimode1_chip structure, remove it and populate the
eoimode1 functions dynamically for the appropriate GIC irqchips.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic.c | 22 ++++++----------------
1 file changed, 6 insertions(+), 16 deletions(-)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index a4a13ef35c1b..b0a781f8c450 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -393,20 +393,6 @@ static struct irq_chip gic_chip = {
IRQCHIP_MASK_ON_SUSPEND,
};

-static struct irq_chip gic_eoimode1_chip = {
- .name = "GICv2",
- .irq_mask = gic_eoimode1_mask_irq,
- .irq_unmask = gic_unmask_irq,
- .irq_eoi = gic_eoimode1_eoi_irq,
- .irq_set_type = gic_set_type,
- .irq_get_irqchip_state = gic_irq_get_irqchip_state,
- .irq_set_irqchip_state = gic_irq_set_irqchip_state,
- .irq_set_vcpu_affinity = gic_irq_set_vcpu_affinity,
- .flags = IRQCHIP_SET_TYPE_MASKED |
- IRQCHIP_SKIP_SET_WAKE |
- IRQCHIP_MASK_ON_SUSPEND,
-};
-
void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
{
BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
@@ -1028,10 +1014,14 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
gic = &gic_data[gic_nr];

/* Initialize irq_chip */
+ gic->chip = gic_chip;
+
if (static_key_true(&supports_deactivate) && gic_nr == 0) {
- gic->chip = gic_eoimode1_chip;
+ gic->chip.irq_mask = gic_eoimode1_mask_irq;
+ gic->chip.irq_eoi = gic_eoimode1_eoi_irq;
+ gic->chip.irq_set_vcpu_affinity = gic_irq_set_vcpu_affinity;
+ gic->chip.name = kasprintf(GFP_KERNEL, "GICv2");
} else {
- gic->chip = gic_chip;
gic->chip.name = kasprintf(GFP_KERNEL, "GIC-%d", gic_nr);
}

--
2.1.4

2016-03-17 14:20:34

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 13/15] irqchip/gic: Prepare for adding platform driver

To support GIC chips located in power-domains outside of the CPU subsystem
it is necessary to add a platform driver for these chips, so that the
probing of the chip can be deferred if resources, such as a power-domain,
is not yet available.

To re-use the code that initialises the GIC (found in __gic_init_bases()),
from within the platform driver, it is necessary to move the code from the
__init section so that it is always present and not removed. Unfortunately,
it is not possible to simply drop the __init from the function declaration
for __gic_init_bases() because it contains calls to set_smp_cross_call()
and set_handle_irq() which are both located in the __init section.
Fortunately, these calls are only required for the root controller and
because the platform driver will only support non-root controllers that
can be initialised later in the boot process, we can move these calls to
another function. Move the bulk of the code from __gic_init_bases() to a
new function called gic_init_bases() which is not located in the __init
section and can be used by the platform driver. Update __gic_init_bases()
to call gic_init_bases() and if necessary, set_smp_cross_call() and
set_handle_irq().

The function, gic_init_bases(), references the GIC via a pointer to the
GIC chip data structure instead of an index so that it can be used by the
platform driver and statically declared GICs. This means that the name
must be passed to gic_init_bases() as well, because the name will not be
passed upon an index for platform devices.

Drop the __init section from the gic_dist_config(), gic_dist_init() and
gic_pm_init() so these can be re-used by the platform driver as well.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic-common.c | 4 +-
drivers/irqchip/irq-gic.c | 80 +++++++++++++++++++++++++++-------------
2 files changed, 57 insertions(+), 27 deletions(-)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 423a345d7b75..646fc0f4a3bc 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -64,8 +64,8 @@ void gic_configure_irq(unsigned int irq, unsigned int type,
sync_access();
}

-void __init gic_dist_config(void __iomem *base, int gic_irqs,
- void (*sync_access)(void))
+void gic_dist_config(void __iomem *base, int gic_irqs,
+ void (*sync_access)(void))
{
unsigned int i;

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index de9f1caee648..7cc5380db298 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -438,7 +438,7 @@ static void gic_cpu_if_up(struct gic_chip_data *gic)
}


-static void __init gic_dist_init(struct gic_chip_data *gic)
+static void gic_dist_init(struct gic_chip_data *gic)
{
unsigned int i;
u32 cpumask;
@@ -711,7 +711,7 @@ static struct notifier_block gic_notifier_block = {
.notifier_call = gic_notifier,
};

-static void __init gic_pm_init(struct gic_chip_data *gic)
+static void gic_pm_init(struct gic_chip_data *gic)
{
gic->saved_ppi_enable = __alloc_percpu(DIV_ROUND_UP(32, 32) * 4,
sizeof(u32));
@@ -729,7 +729,7 @@ static void __init gic_pm_init(struct gic_chip_data *gic)
cpu_pm_register_notifier(&gic_notifier_block);
}
#else
-static void __init gic_pm_init(struct gic_chip_data *gic)
+static void gic_pm_init(struct gic_chip_data *gic)
{
}
#endif
@@ -1003,34 +1003,31 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
.unmap = gic_irq_domain_unmap,
};

-static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
- void __iomem *dist_base, void __iomem *cpu_base,
- u32 percpu_offset, struct fwnode_handle *handle)
+static int gic_init_bases(struct gic_chip_data *gic, int irq_start,
+ void __iomem *dist_base, void __iomem *cpu_base,
+ u32 percpu_offset, struct fwnode_handle *handle,
+ const char *name)
{
irq_hw_number_t hwirq_base;
- struct gic_chip_data *gic;
int gic_irqs, irq_base, i, ret;

- BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
+ if (WARN_ON(!gic || gic->domain))
+ return -EINVAL;

gic_check_cpu_features();

- gic = &gic_data[gic_nr];
-
/* Initialize irq_chip */
gic->chip = gic_chip;
+ gic->chip.name = name;

- if (static_key_true(&supports_deactivate) && gic_nr == 0) {
+ if (static_key_true(&supports_deactivate) && gic == &gic_data[0]) {
gic->chip.irq_mask = gic_eoimode1_mask_irq;
gic->chip.irq_eoi = gic_eoimode1_eoi_irq;
gic->chip.irq_set_vcpu_affinity = gic_irq_set_vcpu_affinity;
- gic->chip.name = kasprintf(GFP_KERNEL, "GICv2");
- } else {
- gic->chip.name = kasprintf(GFP_KERNEL, "GIC-%d", gic_nr);
}

#ifdef CONFIG_SMP
- if (gic_nr == 0)
+ if (gic == &gic_data[0])
gic->chip.irq_set_affinity = gic_set_affinity;
#endif

@@ -1084,7 +1081,7 @@ static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
* For primary GICs, skip over SGIs.
* For secondary GICs, skip over PPIs, too.
*/
- if (gic_nr == 0 && (irq_start & 31) > 0) {
+ if (gic == &gic_data[0] && (irq_start & 31) > 0) {
hwirq_base = 16;
if (irq_start != -1)
irq_start = (irq_start & ~31) + 16;
@@ -1111,7 +1108,7 @@ static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
goto error;
}

- if (gic_nr == 0) {
+ if (gic == &gic_data[0]) {
/*
* Initialize the CPU interface map to all CPUs.
* It will be refined as each CPU probes its ID.
@@ -1119,13 +1116,6 @@ static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
*/
for (i = 0; i < NR_GIC_CPU_IF; i++)
gic_cpu_map[i] = 0xff;
-#ifdef CONFIG_SMP
- set_smp_cross_call(gic_raise_softirq);
- register_cpu_notifier(&gic_cpu_notifier);
-#endif
- set_handle_irq(gic_handle_irq);
- if (static_key_true(&supports_deactivate))
- pr_info("GIC: Using split EOI/Deactivate mode\n");
}

gic_dist_init(gic);
@@ -1140,7 +1130,47 @@ error:
free_percpu(gic->cpu_base.percpu_base);
}

- kfree(gic->chip.name);
+ return ret;
+}
+
+static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
+ void __iomem *dist_base,
+ void __iomem *cpu_base,
+ u32 percpu_offset,
+ struct fwnode_handle *handle)
+{
+ struct gic_chip_data *gic;
+ char *name;
+ int ret;
+
+ if (WARN_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR))
+ return -EINVAL;
+
+ gic = &gic_data[gic_nr];
+
+ if (static_key_true(&supports_deactivate) && gic_nr == 0)
+ name = kasprintf(GFP_KERNEL, "GICv2");
+ else
+ name = kasprintf(GFP_KERNEL, "GIC-%d", gic_nr);
+
+ ret = gic_init_bases(gic, irq_start, dist_base, cpu_base,
+ percpu_offset, handle, name);
+ if (ret) {
+ kfree(gic->chip.name);
+ return ret;
+ }
+
+ if (gic_nr == 0) {
+ if (IS_ENABLED(CONFIG_SMP)) {
+ set_smp_cross_call(gic_raise_softirq);
+ register_cpu_notifier(&gic_cpu_notifier);
+ }
+
+ set_handle_irq(gic_handle_irq);
+
+ if (static_key_true(&supports_deactivate))
+ pr_info("GIC: Using split EOI/Deactivate mode\n");
+ }

return ret;
}
--
2.1.4

2016-03-17 14:20:31

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 11/15] irqchip/gic: Return an error if GIC initialisation fails

If the GIC initialisation fails, then currently we do not return an error
or clean-up afterwards. Although for root controllers, this failure may be
fatal anyway, for secondary controllers, it may not be fatal and so return
an error on failure and clean-up.

For non-banked GIC controllers, make sure that we free any memory
allocated if we fail to initialise the IRQ domain. Please note that
free_percpu() only frees memory if the pointer passed to it is not NULL
and so it is unnecessary to check if both pointers are valid or not.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic.c | 57 ++++++++++++++++++++++++++++++++++-------------
1 file changed, 41 insertions(+), 16 deletions(-)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index b0a781f8c450..42a1412b5186 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -999,13 +999,13 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
.unmap = gic_irq_domain_unmap,
};

-static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
+static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
void __iomem *dist_base, void __iomem *cpu_base,
u32 percpu_offset, struct fwnode_handle *handle)
{
irq_hw_number_t hwirq_base;
struct gic_chip_data *gic;
- int gic_irqs, irq_base, i;
+ int gic_irqs, irq_base, i, ret;

BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);

@@ -1030,17 +1030,16 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
gic->chip.irq_set_affinity = gic_set_affinity;
#endif

-#ifdef CONFIG_GIC_NON_BANKED
- if (percpu_offset) { /* Frankein-GIC without banked registers... */
+ if (IS_ENABLED(CONFIG_GIC_NON_BANKED) && percpu_offset) {
+ /* Frankein-GIC without banked registers... */
unsigned int cpu;

gic->dist_base.percpu_base = alloc_percpu(void __iomem *);
gic->cpu_base.percpu_base = alloc_percpu(void __iomem *);
if (WARN_ON(!gic->dist_base.percpu_base ||
!gic->cpu_base.percpu_base)) {
- free_percpu(gic->dist_base.percpu_base);
- free_percpu(gic->cpu_base.percpu_base);
- return;
+ ret = -ENOMEM;
+ goto error;
}

for_each_possible_cpu(cpu) {
@@ -1052,9 +1051,8 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
}

gic_set_base_accessor(gic, gic_get_percpu_base);
- } else
-#endif
- { /* Normal, sane GIC... */
+ } else {
+ /* Normal, sane GIC... */
WARN(percpu_offset,
"GIC_NON_BANKED not enabled, ignoring %08x offset!",
percpu_offset);
@@ -1104,8 +1102,10 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
hwirq_base, &gic_irq_domain_ops, gic);
}

- if (WARN_ON(!gic->domain))
- return;
+ if (WARN_ON(!gic->domain)) {
+ ret = -ENODEV;
+ goto error;
+ }

if (gic_nr == 0) {
/*
@@ -1127,6 +1127,18 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
gic_dist_init(gic);
gic_cpu_init(gic);
gic_pm_init(gic);
+
+ return 0;
+
+error:
+ if (IS_ENABLED(CONFIG_GIC_NON_BANKED) && percpu_offset) {
+ free_percpu(gic->dist_base.percpu_base);
+ free_percpu(gic->cpu_base.percpu_base);
+ }
+
+ kfree(gic->chip.name);
+
+ return ret;
}

void __init gic_init(unsigned int gic_nr, int irq_start,
@@ -1187,7 +1199,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
void __iomem *cpu_base;
void __iomem *dist_base;
u32 percpu_offset;
- int irq;
+ int irq, ret;

if (WARN_ON(!node))
return -ENODEV;
@@ -1212,8 +1224,14 @@ gic_of_init(struct device_node *node, struct device_node *parent)
if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
percpu_offset = 0;

- __gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset,
+ ret = __gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset,
&node->fwnode);
+ if (ret) {
+ iounmap(dist_base);
+ iounmap(cpu_base);
+ return ret;
+ }
+
if (!gic_cnt)
gic_init_physaddr(node);

@@ -1302,7 +1320,7 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,
struct acpi_madt_generic_distributor *dist;
void __iomem *cpu_base, *dist_base;
struct fwnode_handle *domain_handle;
- int count;
+ int count, ret;

/* Collect CPU base addresses */
count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
@@ -1345,7 +1363,14 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,
return -ENOMEM;
}

- __gic_init_bases(0, -1, dist_base, cpu_base, 0, domain_handle);
+ ret = __gic_init_bases(0, -1, dist_base, cpu_base, 0, domain_handle);
+ if (ret) {
+ pr_err("Failed to initialise GIC\n");
+ irq_domain_free_fwnode(domain_handle);
+ iounmap(cpu_base);
+ iounmap(dist_base);
+ return ret;
+ }

acpi_set_irq_model(ACPI_IRQ_MODEL_GIC, domain_handle);

--
2.1.4

2016-03-17 14:20:42

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 15/15] irqchip/gic: Add support for tegra AGIC interrupt controller

Add a driver for the Tegra-AGIC interrupt controller which is compatible
with the ARM GIC-400 interrupt controller.

The Tegra AGIC (Audio GIC) is part of the Audio Processing Engine (APE) on
Tegra210 and can route interrupts to either the GIC for the CPU subsystem
or the Audio DSP (ADSP) within the APE. The AGIC uses CPU interface 0 to
route interrupts to the CPU GIC and CPU interface 1 to route interrupts to
the ADSP.

The APE is located within its own power domain on the chip and so the
AGIC needs to manage both the power domain and its clocks. Commit
afbbd2338176 ("irqchip/gic: Document optional Clock and Power Domain
properties") adding clock and power-domain properties to the GIC binding
and so the aim would be to make use of these to handle power management
(however, this is very much dependent upon adding support for generic
PM domains for Tegra which is still a work-in-progress).

With the AGIC being located in a different power domain to the main CPU
cluster this means that:
1. The interrupt controller cannot be registered via IRQCHIP_DECLARE()
because it needs to be registered as a platform device so that the
generic PM domain core will ensure that the power domain is available
before probing.
2. The interrupt controller cannot be suspended/restored based upon
changes in the CPU power state and needs to use runtime-pm instead.

The GIC platform driver has been implemented by making the following
changes to the core GIC driver:
1. Remove the dependency on CONFIG_CPU_PM from PM specific variables and
functions so that they can be used by the platform driver even when
CONFIG_CPU_PM is not selected.
2. Move the code that maps the GIC registers and parses the device-tree
blob into a new function called gic_of_setup() that can be used by
both the platform driver as well as the existing driver.
3. Add and register platform driver for the GIC. The platform driver
uses the PM_CLK framework for managing the clocks used by the GIC
and so select CONFIG_PM_CLK.

Finally, a couple other notes on the implementation are:
1. Currently the GIC platform driver only supports non-root GICs and
assumes that the GIC has a parent interrupt. It is assumed that
root interrupt controllers need to be initialised early.
2. There is no specific suspend handling for platform devices because
non-wakeup interrupts will be disabled by the kernel during late
suspend.

Signed-off-by: Jon Hunter <[email protected]>
---
.../bindings/interrupt-controller/arm,gic.txt | 1 +
drivers/irqchip/Kconfig | 1 +
drivers/irqchip/irq-gic.c | 210 ++++++++++++++++++---
3 files changed, 189 insertions(+), 23 deletions(-)

diff --git a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
index c471d1a7a8ea..9d84a86d0934 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+++ b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
@@ -21,6 +21,7 @@ Main node required properties:
"arm,pl390"
"arm,tc11mp-gic"
"brcm,brahma-b15-gic"
+ "nvidia,tegra210-agic"
"qcom,msm-8660-qgic"
"qcom,msm-qgic2"
- interrupt-controller : Identifies the node as an interrupt controller
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 7e8c441ff2de..ececa3cb6c0a 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -7,6 +7,7 @@ config ARM_GIC
select IRQ_DOMAIN
select IRQ_DOMAIN_HIERARCHY
select MULTI_IRQ_HANDLER
+ select PM_CLK

config ARM_GIC_MAX_NR
int
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 7cc5380db298..9e7cf7abf757 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -26,6 +26,7 @@
#include <linux/module.h>
#include <linux/list.h>
#include <linux/smp.h>
+#include <linux/clk.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
#include <linux/cpumask.h>
@@ -37,6 +38,9 @@
#include <linux/irqdomain.h>
#include <linux/interrupt.h>
#include <linux/percpu.h>
+#include <linux/platform_device.h>
+#include <linux/pm_clock.h>
+#include <linux/pm_runtime.h>
#include <linux/slab.h>
#include <linux/irqchip.h>
#include <linux/irqchip/chained_irq.h>
@@ -72,7 +76,6 @@ struct gic_chip_data {
struct irq_chip chip;
union gic_base dist_base;
union gic_base cpu_base;
-#ifdef CONFIG_CPU_PM
u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
u32 saved_spi_active[DIV_ROUND_UP(1020, 32)];
u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
@@ -80,7 +83,6 @@ struct gic_chip_data {
u32 __percpu *saved_ppi_enable;
u32 __percpu *saved_ppi_active;
u32 __percpu *saved_ppi_conf;
-#endif
struct irq_domain *domain;
unsigned int gic_irqs;
#ifdef CONFIG_GIC_NON_BANKED
@@ -512,7 +514,6 @@ int gic_cpu_if_down(unsigned int gic_nr)
return 0;
}

-#ifdef CONFIG_CPU_PM
/*
* Saves the GIC distributor registers during suspend or idle. Must be called
* with interrupts disabled but before powering down the GIC. After calling
@@ -728,11 +729,6 @@ static void gic_pm_init(struct gic_chip_data *gic)
if (gic == &gic_data[0])
cpu_pm_register_notifier(&gic_notifier_block);
}
-#else
-static void gic_pm_init(struct gic_chip_data *gic)
-{
-}
-#endif

#ifdef CONFIG_SMP
static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
@@ -1227,27 +1223,42 @@ static bool gic_check_eoimode(struct device_node *node, void __iomem **base)
return true;
}

-int __init
-gic_of_init(struct device_node *node, struct device_node *parent)
+static int gic_of_setup(struct device_node *node, void __iomem **dist_base,
+ void __iomem **cpu_base, u32 *percpu_offset)
{
- void __iomem *cpu_base;
- void __iomem *dist_base;
- u32 percpu_offset;
- int irq, ret;
-
if (WARN_ON(!node))
return -ENODEV;

- dist_base = of_iomap(node, 0);
- if (WARN(!dist_base, "unable to map gic dist registers\n"))
+ *dist_base = of_iomap(node, 0);
+ if (WARN(!*dist_base, "unable to map gic dist registers\n"))
return -ENOMEM;

- cpu_base = of_iomap(node, 1);
- if (WARN(!cpu_base, "unable to map gic cpu registers\n")) {
- iounmap(dist_base);
+ *cpu_base = of_iomap(node, 1);
+ if (WARN(!*cpu_base, "unable to map gic cpu registers\n")) {
+ iounmap(*dist_base);
return -ENOMEM;
}

+ if (of_property_read_u32(node, "cpu-offset", percpu_offset))
+ *percpu_offset = 0;
+
+ return 0;
+}
+
+int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+ void __iomem *cpu_base;
+ void __iomem *dist_base;
+ u32 percpu_offset;
+ int irq, ret;
+
+ if (WARN_ON(gic_cnt >= CONFIG_ARM_GIC_MAX_NR))
+ return -EINVAL;
+
+ ret = gic_of_setup(node, &dist_base, &cpu_base, &percpu_offset);
+ if (ret)
+ return ret;
+
/*
* Disable split EOI/Deactivate if either HYP is not available
* or the CPU interface is too small.
@@ -1255,9 +1266,6 @@ gic_of_init(struct device_node *node, struct device_node *parent)
if (gic_cnt == 0 && !gic_check_eoimode(node, &cpu_base))
static_key_slow_dec(&supports_deactivate);

- if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
- percpu_offset = 0;
-
ret = __gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset,
&node->fwnode);
if (ret) {
@@ -1290,6 +1298,162 @@ IRQCHIP_DECLARE(msm_8660_qgic, "qcom,msm-8660-qgic", gic_of_init);
IRQCHIP_DECLARE(msm_qgic2, "qcom,msm-qgic2", gic_of_init);
IRQCHIP_DECLARE(pl390, "arm,pl390", gic_of_init);

+static int gic_runtime_resume(struct device *dev)
+{
+ struct gic_chip_data *gic = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_clk_resume(dev);
+ if (ret)
+ return ret;
+
+ gic_dist_restore(gic);
+ gic_cpu_restore(gic);
+
+ return 0;
+}
+
+static int gic_runtime_suspend(struct device *dev)
+{
+ struct gic_chip_data *gic = dev_get_drvdata(dev);
+
+ gic_dist_save(gic);
+ gic_cpu_save(gic);
+
+ return pm_clk_suspend(dev);
+}
+
+static int gic_get_clocks(struct device *dev)
+{
+ struct clk *clk;
+ unsigned int i, count;
+ int ret;
+
+ if (!dev || !dev->of_node)
+ return -EINVAL;
+
+ count = of_count_phandle_with_args(dev->of_node, "clocks",
+ "#clock-cells");
+ if (count == 0)
+ return -ENODEV;
+
+ ret = pm_clk_create(dev);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < count; i++) {
+ clk = of_clk_get(dev->of_node, i);
+ if (IS_ERR(clk)) {
+ dev_err(dev, "failed to get clock at index %d\n", i);
+ ret = PTR_ERR(clk);
+ goto error;
+ }
+
+ ret = pm_clk_add_clk(dev, clk);
+ if (ret) {
+ dev_err(dev, "failed to add clock at index %d\n", i);
+ clk_put(clk);
+ goto error;
+ }
+ }
+
+ return 0;
+
+error:
+ pm_clk_destroy(dev);
+
+ return ret;
+}
+
+static int gic_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct gic_chip_data *gic;
+ void __iomem *dist_base;
+ void __iomem *cpu_base;
+ u32 percpu_offset;
+ int ret, irq;
+
+ if (dev->of_node == NULL)
+ return -EINVAL;
+
+ gic = devm_kzalloc(dev, sizeof(*gic), GFP_KERNEL);
+ if (!gic)
+ return -ENOMEM;
+
+ ret = gic_get_clocks(dev);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, gic);
+
+ pm_runtime_enable(dev);
+
+ ret = pm_runtime_get_sync(dev);
+ if (ret < 0)
+ goto rpm_disable;
+
+ irq = irq_of_parse_and_map(dev->of_node, 0);
+ if (!irq) {
+ ret = -EINVAL;
+ goto rpm_put;
+ }
+
+ ret = gic_of_setup(dev->of_node, &dist_base, &cpu_base, &percpu_offset);
+ if (ret)
+ goto irq_dispose;
+
+ ret = gic_init_bases(gic, -1, dist_base, cpu_base,
+ percpu_offset, &dev->of_node->fwnode,
+ dev->of_node->name);
+ if (ret)
+ goto gic_unmap;
+
+ gic->chip.parent = dev;
+
+ irq_set_chained_handler_and_data(irq, gic_handle_cascade_irq, gic);
+
+ pm_runtime_put(dev);
+
+ dev_info(dev, "GIC IRQ controller registered\n");
+
+ return 0;
+
+gic_unmap:
+ iounmap(dist_base);
+ iounmap(cpu_base);
+irq_dispose:
+ irq_dispose_mapping(irq);
+rpm_put:
+ pm_runtime_put_sync(dev);
+rpm_disable:
+ pm_runtime_disable(dev);
+ pm_clk_destroy(dev);
+
+ return ret;
+}
+
+static const struct dev_pm_ops gic_pm_ops = {
+ SET_RUNTIME_PM_OPS(gic_runtime_suspend,
+ gic_runtime_resume, NULL)
+};
+
+static const struct of_device_id gic_match[] = {
+ { .compatible = "nvidia,tegra210-agic", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, gic_match);
+
+static struct platform_driver gic_driver = {
+ .probe = gic_probe,
+ .driver = {
+ .name = "gic",
+ .of_match_table = gic_match,
+ .pm = &gic_pm_ops,
+ }
+};
+
+builtin_platform_driver(gic_driver);
#endif

#ifdef CONFIG_ACPI
--
2.1.4

2016-03-17 14:21:22

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
Domain properties") documented optional clock and power-dmoain properties
for the ARM GIC. Currently, there are no users of these and for the
Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
functional clock and interface clock, that need to be enabled.

To allow flexibility, drop the 'clock-names' from the GIC binding and
just provide a list of clocks which the driver can parse. It is assumed
that any clocks that are listed, need to be enabled in order to access
the GIC.

Signed-off-by: Jon Hunter <[email protected]>

---

Please note that I am not sure if this will be popular, but I am trying
to come up with a generic way to handle multiple clocks that may be
required for accessing a GIC.

.../devicetree/bindings/interrupt-controller/arm,gic.txt | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
index 793c20ff8fcc..c471d1a7a8ea 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+++ b/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
@@ -61,15 +61,8 @@ Optional
regions, used when the GIC doesn't have banked registers. The offset is
cpu-offset * cpu-nr.

-- clocks : List of phandle and clock-specific pairs, one for each entry
- in clock-names.
-- clock-names : List of names for the GIC clock input(s). Valid clock names
- depend on the GIC variant:
- "ic_clk" (for "arm,arm11mp-gic")
- "PERIPHCLKEN" (for "arm,cortex-a15-gic")
- "PERIPHCLK", "PERIPHCLKEN" (for "arm,cortex-a9-gic")
- "clk" (for "arm,gic-400")
- "gclk" (for "arm,pl390")
+- clocks : List of phandle and clock-specific pairs required for
+ accessing the GIC.

- power-domains : A phandle and PM domain specifier as defined by bindings of
the power controller specified by phandle, used when the GIC
--
2.1.4

2016-03-17 14:21:46

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 12/15] irqchip/gic: Pass GIC pointer to save/restore functions

Instead of passing the GIC index to the save/restore functions pass a
pointer to the GIC chip data. This will allow these save/restore
functions to be re-used by a platform driver where the GIC chip data
structure is allocated dynamically and so there is no applicable index
for identifying the GIC.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic.c | 74 +++++++++++++++++++++++++----------------------
1 file changed, 39 insertions(+), 35 deletions(-)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 42a1412b5186..de9f1caee648 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -519,34 +519,35 @@ int gic_cpu_if_down(unsigned int gic_nr)
* this function, no interrupts will be delivered by the GIC, and another
* platform-specific wakeup source must be enabled.
*/
-static void gic_dist_save(unsigned int gic_nr)
+static void gic_dist_save(struct gic_chip_data *gic)
{
unsigned int gic_irqs;
void __iomem *dist_base;
int i;

- BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
+ if (WARN_ON(!gic))
+ return;

- gic_irqs = gic_data[gic_nr].gic_irqs;
- dist_base = gic_data_dist_base(&gic_data[gic_nr]);
+ gic_irqs = gic->gic_irqs;
+ dist_base = gic_data_dist_base(gic);

if (!dist_base)
return;

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 16); i++)
- gic_data[gic_nr].saved_spi_conf[i] =
+ gic->saved_spi_conf[i] =
readl_relaxed(dist_base + GIC_DIST_CONFIG + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
- gic_data[gic_nr].saved_spi_target[i] =
+ gic->saved_spi_target[i] =
readl_relaxed(dist_base + GIC_DIST_TARGET + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++)
- gic_data[gic_nr].saved_spi_enable[i] =
+ gic->saved_spi_enable[i] =
readl_relaxed(dist_base + GIC_DIST_ENABLE_SET + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++)
- gic_data[gic_nr].saved_spi_active[i] =
+ gic->saved_spi_active[i] =
readl_relaxed(dist_base + GIC_DIST_ACTIVE_SET + i * 4);
}

@@ -557,16 +558,17 @@ static void gic_dist_save(unsigned int gic_nr)
* handled normally, but any edge interrupts that occured will not be seen by
* the GIC and need to be handled by the platform-specific wakeup source.
*/
-static void gic_dist_restore(unsigned int gic_nr)
+static void gic_dist_restore(struct gic_chip_data *gic)
{
unsigned int gic_irqs;
unsigned int i;
void __iomem *dist_base;

- BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
+ if (WARN_ON(!gic))
+ return;

- gic_irqs = gic_data[gic_nr].gic_irqs;
- dist_base = gic_data_dist_base(&gic_data[gic_nr]);
+ gic_irqs = gic->gic_irqs;
+ dist_base = gic_data_dist_base(gic);

if (!dist_base)
return;
@@ -574,7 +576,7 @@ static void gic_dist_restore(unsigned int gic_nr)
writel_relaxed(GICD_DISABLE, dist_base + GIC_DIST_CTRL);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 16); i++)
- writel_relaxed(gic_data[gic_nr].saved_spi_conf[i],
+ writel_relaxed(gic->saved_spi_conf[i],
dist_base + GIC_DIST_CONFIG + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
@@ -582,85 +584,87 @@ static void gic_dist_restore(unsigned int gic_nr)
dist_base + GIC_DIST_PRI + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
- writel_relaxed(gic_data[gic_nr].saved_spi_target[i],
+ writel_relaxed(gic->saved_spi_target[i],
dist_base + GIC_DIST_TARGET + i * 4);

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++) {
writel_relaxed(GICD_INT_EN_CLR_X32,
dist_base + GIC_DIST_ENABLE_CLEAR + i * 4);
- writel_relaxed(gic_data[gic_nr].saved_spi_enable[i],
+ writel_relaxed(gic->saved_spi_enable[i],
dist_base + GIC_DIST_ENABLE_SET + i * 4);
}

for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++) {
writel_relaxed(GICD_INT_EN_CLR_X32,
dist_base + GIC_DIST_ACTIVE_CLEAR + i * 4);
- writel_relaxed(gic_data[gic_nr].saved_spi_active[i],
+ writel_relaxed(gic->saved_spi_active[i],
dist_base + GIC_DIST_ACTIVE_SET + i * 4);
}

writel_relaxed(GICD_ENABLE, dist_base + GIC_DIST_CTRL);
}

-static void gic_cpu_save(unsigned int gic_nr)
+static void gic_cpu_save(struct gic_chip_data *gic)
{
int i;
u32 *ptr;
void __iomem *dist_base;
void __iomem *cpu_base;

- BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
+ if (WARN_ON(!gic))
+ return;

- dist_base = gic_data_dist_base(&gic_data[gic_nr]);
- cpu_base = gic_data_cpu_base(&gic_data[gic_nr]);
+ dist_base = gic_data_dist_base(gic);
+ cpu_base = gic_data_cpu_base(gic);

if (!dist_base || !cpu_base)
return;

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_enable);
+ ptr = raw_cpu_ptr(gic->saved_ppi_enable);
for (i = 0; i < DIV_ROUND_UP(32, 32); i++)
ptr[i] = readl_relaxed(dist_base + GIC_DIST_ENABLE_SET + i * 4);

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_active);
+ ptr = raw_cpu_ptr(gic->saved_ppi_active);
for (i = 0; i < DIV_ROUND_UP(32, 32); i++)
ptr[i] = readl_relaxed(dist_base + GIC_DIST_ACTIVE_SET + i * 4);

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_conf);
+ ptr = raw_cpu_ptr(gic->saved_ppi_conf);
for (i = 0; i < DIV_ROUND_UP(32, 16); i++)
ptr[i] = readl_relaxed(dist_base + GIC_DIST_CONFIG + i * 4);

}

-static void gic_cpu_restore(unsigned int gic_nr)
+static void gic_cpu_restore(struct gic_chip_data *gic)
{
int i;
u32 *ptr;
void __iomem *dist_base;
void __iomem *cpu_base;

- BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
+ if (WARN_ON(!gic))
+ return;

- dist_base = gic_data_dist_base(&gic_data[gic_nr]);
- cpu_base = gic_data_cpu_base(&gic_data[gic_nr]);
+ dist_base = gic_data_dist_base(gic);
+ cpu_base = gic_data_cpu_base(gic);

if (!dist_base || !cpu_base)
return;

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_enable);
+ ptr = raw_cpu_ptr(gic->saved_ppi_enable);
for (i = 0; i < DIV_ROUND_UP(32, 32); i++) {
writel_relaxed(GICD_INT_EN_CLR_X32,
dist_base + GIC_DIST_ENABLE_CLEAR + i * 4);
writel_relaxed(ptr[i], dist_base + GIC_DIST_ENABLE_SET + i * 4);
}

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_active);
+ ptr = raw_cpu_ptr(gic->saved_ppi_active);
for (i = 0; i < DIV_ROUND_UP(32, 32); i++) {
writel_relaxed(GICD_INT_EN_CLR_X32,
dist_base + GIC_DIST_ACTIVE_CLEAR + i * 4);
writel_relaxed(ptr[i], dist_base + GIC_DIST_ACTIVE_SET + i * 4);
}

- ptr = raw_cpu_ptr(gic_data[gic_nr].saved_ppi_conf);
+ ptr = raw_cpu_ptr(gic->saved_ppi_conf);
for (i = 0; i < DIV_ROUND_UP(32, 16); i++)
writel_relaxed(ptr[i], dist_base + GIC_DIST_CONFIG + i * 4);

@@ -669,7 +673,7 @@ static void gic_cpu_restore(unsigned int gic_nr)
dist_base + GIC_DIST_PRI + i * 4);

writel_relaxed(GICC_INT_PRI_THRESHOLD, cpu_base + GIC_CPU_PRIMASK);
- gic_cpu_if_up(&gic_data[gic_nr]);
+ gic_cpu_if_up(gic);
}

static int gic_notifier(struct notifier_block *self, unsigned long cmd, void *v)
@@ -684,18 +688,18 @@ static int gic_notifier(struct notifier_block *self, unsigned long cmd, void *v)
#endif
switch (cmd) {
case CPU_PM_ENTER:
- gic_cpu_save(i);
+ gic_cpu_save(&gic_data[i]);
break;
case CPU_PM_ENTER_FAILED:
case CPU_PM_EXIT:
- gic_cpu_restore(i);
+ gic_cpu_restore(&gic_data[i]);
break;
case CPU_CLUSTER_PM_ENTER:
- gic_dist_save(i);
+ gic_dist_save(&gic_data[i]);
break;
case CPU_CLUSTER_PM_ENTER_FAILED:
case CPU_CLUSTER_PM_EXIT:
- gic_dist_restore(i);
+ gic_dist_restore(&gic_data[i]);
break;
}
}
--
2.1.4

2016-03-17 14:22:32

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Some IRQ chips may be located in a power domain outside of the CPU
subsystem and hence will require device specific runtime power
management. In order to support such IRQ chips, add a pointer for a
device structure to the irq_chip structure, and if this pointer is
populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
configuration, then the pm_runtime_get/put APIs for this chip will be
called when an IRQ is requested/freed, respectively.

When entering system suspend and each interrupt is disabled if there is
no wake-up set for that interrupt. For an IRQ chip that utilises runtime
power management, print a warning message for each active interrupt that
has no wake-up set because these interrupts may be unnecessarily keeping
the IRQ chip enabled during system suspend.

Signed-off-by: Jon Hunter <[email protected]>
---
include/linux/irq.h | 5 +++++
kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
kernel/irq/internals.h | 1 +
kernel/irq/manage.c | 14 +++++++++++---
kernel/irq/pm.c | 3 +++
5 files changed, 72 insertions(+), 3 deletions(-)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index c4de62348ff2..82f36390048d 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
/**
* struct irq_chip - hardware interrupt chip descriptor
*
+ * @parent: pointer to associated device
* @name: name for /proc/interrupts
* @irq_startup: start up the interrupt (defaults to ->enable if NULL)
* @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
@@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
* @flags: chip specific flags
*/
struct irq_chip {
+ struct device *parent;
const char *name;
unsigned int (*irq_startup)(struct irq_data *data);
void (*irq_shutdown)(struct irq_data *data);
@@ -488,6 +490,9 @@ extern void handle_bad_irq(struct irq_desc *desc);
extern void handle_nested_irq(unsigned int irq);

extern int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg);
+extern int irq_chip_pm_get(struct irq_data *data);
+extern int irq_chip_pm_put(struct irq_data *data);
+extern bool irq_chip_pm_suspended(struct irq_data *data);
#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
extern void irq_chip_enable_parent(struct irq_data *data);
extern void irq_chip_disable_parent(struct irq_data *data);
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 2f9f2b0e79f2..c575b700e88a 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -1093,3 +1093,55 @@ int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)

return 0;
}
+
+/**
+ * irq_chip_pm_get - Enable power for an IRQ chip
+ * @data: Pointer to interrupt specific data
+ *
+ * Enable the power to the IRQ chip referenced by the interrupt data
+ * structure.
+ */
+int irq_chip_pm_get(struct irq_data *data)
+{
+ int retval = 0;
+
+ if (IS_ENABLED(CONFIG_PM) && data->chip->parent)
+ retval = pm_runtime_get_sync(data->chip->parent);
+
+ return (retval < 0) ? retval : 0;
+}
+
+/**
+ * irq_chip_pm_put - Disable power for an IRQ chip
+ * @data: Pointer to interrupt specific data
+ *
+ * Disable the power to the IRQ chip referenced by the interrupt data
+ * structure, belongs. Note that power will only be disabled, once this
+ * function has been called for all IRQs that have called irq_chip_pm_get().
+ */
+int irq_chip_pm_put(struct irq_data *data)
+{
+ int retval = 0;
+
+ if (IS_ENABLED(CONFIG_PM) && data->chip->parent)
+ retval = pm_runtime_put(data->chip->parent);
+
+ return (retval < 0) ? retval : 0;
+}
+
+/**
+ * irq_chip_pm_suspended - Power status for an IRQ chip
+ * @data: Pointer to interrupt specific data
+ *
+ * Return the runtime power status for an IRQ chip referenced by the
+ * interrupt data structure.
+ */
+bool irq_chip_pm_suspended(struct irq_data *data)
+{
+ bool status = true;
+
+ if (IS_ENABLED(CONFIG_PM) && data->chip->parent)
+ status = pm_runtime_status_suspended(data->chip->parent);
+
+ return status;
+}
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index 09be2c903c6d..d5edcdc9382a 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -7,6 +7,7 @@
*/
#include <linux/irqdesc.h>
#include <linux/kernel_stat.h>
+#include <linux/pm_runtime.h>

#ifdef CONFIG_SPARSE_IRQ
# define IRQ_BITMAP_BITS (NR_IRQS + 8196)
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index b2a93a37f772..65878e7c7c82 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -1114,6 +1114,10 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
if (!try_module_get(desc->owner))
return -ENODEV;

+ ret = irq_chip_pm_get(&desc->irq_data);
+ if (ret < 0)
+ goto out_mput;
+
new->irq = irq;

/*
@@ -1131,7 +1135,7 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
if (nested) {
if (!new->thread_fn) {
ret = -EINVAL;
- goto out_mput;
+ goto out_pm;
}
/*
* Replace the primary handler which was provided from
@@ -1143,7 +1147,7 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
if (irq_settings_can_thread(desc)) {
ret = irq_setup_forced_threading(new);
if (ret)
- goto out_mput;
+ goto out_pm;
}
}

@@ -1155,7 +1159,7 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
if (new->thread_fn && !nested) {
ret = setup_irq_thread(new, irq, false);
if (ret)
- goto out_mput;
+ goto out_pm;
if (new->secondary) {
ret = setup_irq_thread(new->secondary, irq, true);
if (ret)
@@ -1397,6 +1401,8 @@ out_thread:
kthread_stop(t);
put_task_struct(t);
}
+out_pm:
+ irq_chip_pm_put(&desc->irq_data);
out_mput:
module_put(desc->owner);
return ret;
@@ -1513,6 +1519,7 @@ static struct irqaction *__free_irq(unsigned int irq, void *dev_id)
}
}

+ irq_chip_pm_put(&desc->irq_data);
module_put(desc->owner);
kfree(action->secondary);
return action;
@@ -1829,6 +1836,7 @@ static struct irqaction *__free_percpu_irq(unsigned int irq, void __percpu *dev_

unregister_handler_proc(irq, action);

+ irq_chip_pm_put(&desc->irq_data);
module_put(desc->owner);
return action;

diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
index cea1de0161f1..ab436119084f 100644
--- a/kernel/irq/pm.c
+++ b/kernel/irq/pm.c
@@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
* suspend_device_irqs().
*/
return true;
+ } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
+ pr_warn("irq %d has no wakeup set and has not been freed!\n",
+ desc->irq_data.irq);
}

desc->istate |= IRQS_SUSPENDED;
--
2.1.4

2016-03-17 14:23:01

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

Setting the interrupt type for private peripheral interrupts (PPIs) may
not be supported by a given GIC because it is IMPLEMENTATION DEFINED
whether this is allowed. There is no way to know if setting the type is
supported for a given GIC and so the value written is read back to
verify it matches the desired configuration. If it does not match then
an error is return.

There are cases where the interrupt configuration read from firmware
(such as a device-tree blob), has been incorrect and hence
gic_configure_irq() has returned an error. This error has gone
undetected because the error code returned was ignored but the interrupt
still worked fine because the configuration for the interrupt could not
be overwritten.

Given that this has done undetected and we should only fail to set the
type for PPIs whose configuration cannot be changed anyway, don't return
an error and simply WARN if this fails. This will allows us to fix up any
places in the kernel where we should be checking the return status and
maintain back compatibility with firmware images that may have incorrect
interrupt configurations.

Signed-off-by: Jon Hunter <[email protected]>
---
drivers/irqchip/irq-gic-common.c | 13 ++++---------
drivers/irqchip/irq-gic-common.h | 2 +-
drivers/irqchip/irq-gic-v3.c | 4 +++-
drivers/irqchip/irq-gic.c | 4 +++-
drivers/irqchip/irq-hip04.c | 5 ++---
5 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index ffff5a45f1e3..423a345d7b75 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -32,13 +32,12 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
}
}

-int gic_configure_irq(unsigned int irq, unsigned int type,
+void gic_configure_irq(unsigned int irq, unsigned int type,
void __iomem *base, void (*sync_access)(void))
{
u32 confmask = 0x2 << ((irq % 16) * 2);
u32 confoff = (irq / 16) * 4;
u32 val, oldval;
- int ret = 0;

/*
* Read current configuration register, and insert the config
@@ -52,21 +51,17 @@ int gic_configure_irq(unsigned int irq, unsigned int type,

/* If the current configuration is the same, then we are done */
if (val == oldval)
- return 0;
+ return;

/*
* Write back the new configuration, and possibly re-enable
- * the interrupt. If we fail to write a new configuration,
- * return an error.
+ * the interrupt. WARN if we fail to write a new configuration.
*/
writel_relaxed(val, base + GIC_DIST_CONFIG + confoff);
- if (readl_relaxed(base + GIC_DIST_CONFIG + confoff) != val)
- ret = -EINVAL;
+ WARN_ON(readl_relaxed(base + GIC_DIST_CONFIG + confoff) != val);

if (sync_access)
sync_access();
-
- return ret;
}

void __init gic_dist_config(void __iomem *base, int gic_irqs,
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index fff697db8e22..73dee3bc6bba 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -27,7 +27,7 @@ struct gic_quirk {
u32 mask;
};

-int gic_configure_irq(unsigned int irq, unsigned int type,
+void gic_configure_irq(unsigned int irq, unsigned int type,
void __iomem *base, void (*sync_access)(void));
void gic_dist_config(void __iomem *base, int gic_irqs,
void (*sync_access)(void));
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5b7d3c2129d8..c569c466fa31 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -310,7 +310,9 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
rwp_wait = gic_dist_wait_for_rwp;
}

- return gic_configure_irq(irq, type, base, rwp_wait);
+ gic_configure_irq(irq, type, base, rwp_wait);
+
+ return 0;
}

static int gic_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 282344b95ec2..6c555a2c5315 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -279,7 +279,9 @@ static int gic_set_type(struct irq_data *d, unsigned int type)
type != IRQ_TYPE_EDGE_RISING)
return -EINVAL;

- return gic_configure_irq(gicirq, type, base, NULL);
+ gic_configure_irq(gicirq, type, base, NULL);
+
+ return 0;
}

static int gic_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)
diff --git a/drivers/irqchip/irq-hip04.c b/drivers/irqchip/irq-hip04.c
index 9688d2e2a636..12be5906e91a 100644
--- a/drivers/irqchip/irq-hip04.c
+++ b/drivers/irqchip/irq-hip04.c
@@ -120,7 +120,6 @@ static int hip04_irq_set_type(struct irq_data *d, unsigned int type)
{
void __iomem *base = hip04_dist_base(d);
unsigned int irq = hip04_irq(d);
- int ret;

/* Interrupt configuration for SGIs can't be changed */
if (irq < 16)
@@ -133,11 +132,11 @@ static int hip04_irq_set_type(struct irq_data *d, unsigned int type)

raw_spin_lock(&irq_controller_lock);

- ret = gic_configure_irq(irq, type, base, NULL);
+ gic_configure_irq(irq, type, base, NULL);

raw_spin_unlock(&irq_controller_lock);

- return ret;
+ return 0;
}

#ifdef CONFIG_SMP
--
2.1.4

2016-03-17 14:22:57

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 06/15] irqdomain: Ensure type settings match for an existing mapping

When mapping an IRQ, if a mapping already exists, then we simply return
the virtual IRQ number. However, we do not check that the type settings
for the existing mapping match those for the mapping that is about to be
created. It may be unlikely that the type settings would not match, but
check for this and don't return a valid IRQ mapping if the type settings
do not match.

WARN if the type return by irq_domain_translate() has bits outside the
sense mask set and then clear these bits. If these bits are not cleared
then this will cause the comparision of the type settings for an
existing mapping to fail with that of the new mapping even if the sense
bit themselves match. The reason being is that the existing type
settings are read by calling irq_get_trigger_type() which will clear
any bits outside the sense mask. This will allow us to detect irqchips
that are not correctly clearing these bits and fix them.

Signed-off-by: Jon Hunter <[email protected]>
---
kernel/irq/irqdomain.c | 59 ++++++++++++++++++++++++++++++++++++--------------
1 file changed, 43 insertions(+), 16 deletions(-)

diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 3a519a01118b..0ea285baa619 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -549,6 +549,13 @@ static int irq_domain_translate(struct irq_domain *d,
fwspec->param, fwspec->param_count,
hwirq, type);

+ /*
+ * WARN if the irqchip returns a type with bits
+ * outside the sense mask set and clear these bits.
+ */
+ if (WARN_ON(*type & ~IRQ_TYPE_SENSE_MASK))
+ *type &= IRQ_TYPE_SENSE_MASK;
+
/* If domain has no translation, then we assume interrupt line */
*hwirq = fwspec->param[0];
return 0;
@@ -570,6 +577,7 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
{
struct irq_domain *domain;
irq_hw_number_t hwirq;
+ unsigned int cur_type = IRQ_TYPE_NONE;
unsigned int type = IRQ_TYPE_NONE;
int virq;

@@ -592,23 +600,42 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
if (irq_domain_translate(domain, fwspec, &hwirq, &type))
return 0;

- if (irq_domain_is_hierarchy(domain)) {
- /*
- * If we've already configured this interrupt,
- * don't do it again, or hell will break loose.
- */
- virq = irq_find_mapping(domain, hwirq);
- if (virq)
- return virq;
-
- virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, fwspec);
- if (virq <= 0)
- return 0;
+ /*
+ * If we've already configured this interrupt,
+ * don't do it again, or hell will break loose.
+ */
+ virq = irq_find_mapping(domain, hwirq);
+ if (!virq) {
+ if (irq_domain_is_hierarchy(domain)) {
+ virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE,
+ fwspec);
+ if (virq <= 0)
+ return 0;
+ } else {
+ /* Create mapping */
+ virq = irq_create_mapping(domain, hwirq);
+ if (!virq)
+ return virq;
+ }
} else {
- /* Create mapping */
- virq = irq_create_mapping(domain, hwirq);
- if (!virq)
- return virq;
+ cur_type = irq_get_trigger_type(virq);
+ }
+
+ /*
+ * If the trigger type is not specified or matches the current
+ * trigger type then we are done so return the interrupt number.
+ */
+ if (type == IRQ_TYPE_NONE || type == cur_type)
+ return virq;
+
+ /*
+ * If the trigger type is already set and does
+ * not match this interrupt, then return 0.
+ */
+ if (cur_type != IRQ_TYPE_NONE) {
+ pr_warn("type mismatch, failed to map hwirq-%lu for %s!\n",
+ hwirq, of_node_full_name(to_of_node(fwspec->fwnode)));
+ return 0;
}

/* Set type if specified and different than the current one */
--
2.1.4

2016-03-17 14:23:46

by Jon Hunter

[permalink] [raw]
Subject: [PATCH 02/15] ARM: OMAP: Correct interrupt type for ARM TWD

The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
the ARM GIC documentation, whether the type for PPIs can be set is
IMPLEMENTATION DEFINED. For OMAP4 devices the PPI type cannot be set and
so when we attempt to set the type for the ARM TWD interrupt it fails.
This has done unnoticed because it fails silently and because we cannot
re-configure the type it has had no impact. Nevertheless fix the type
for the TWD interrupt so that it matches the hardware configuration.

Reported-by: Grygorii Strashko <[email protected]>
Signed-off-by: Jon Hunter <[email protected]>

---

Tony, Grygorii,
Please note that I have not tested this. Can you test this series and
see if you see any warnings on OMAP4? I am guessing that the configuration
should be LEVEL and not EDGE. This was reported here:

http://marc.info/?l=linux-tegra&m=145078316419821&w=2

arch/arm/boot/dts/omap4.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/omap4.dtsi b/arch/arm/boot/dts/omap4.dtsi
index 2bd9c83300b2..421fe9f8a9eb 100644
--- a/arch/arm/boot/dts/omap4.dtsi
+++ b/arch/arm/boot/dts/omap4.dtsi
@@ -70,7 +70,7 @@
compatible = "arm,cortex-a9-twd-timer";
clocks = <&mpu_periphclk>;
reg = <0x48240600 0x20>;
- interrupts = <GIC_PPI 13 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_LEVEL_HIGH)>;
+ interrupts = <GIC_PPI 13 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_EDGE_RISING)>;
interrupt-parent = <&gic>;
};

--
2.1.4

2016-03-17 14:53:17

by Thomas Gleixner

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On Thu, 17 Mar 2016, Jon Hunter wrote:

> Setting the interrupt type for private peripheral interrupts (PPIs) may
> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
> whether this is allowed. There is no way to know if setting the type is
> supported for a given GIC and so the value written is read back to
> verify it matches the desired configuration. If it does not match then
> an error is return.
>
> There are cases where the interrupt configuration read from firmware
> (such as a device-tree blob), has been incorrect and hence
> gic_configure_irq() has returned an error. This error has gone
> undetected because the error code returned was ignored but the interrupt
> still worked fine because the configuration for the interrupt could not
> be overwritten.
>
> Given that this has done undetected and we should only fail to set the
> type for PPIs whose configuration cannot be changed anyway, don't return
> an error and simply WARN if this fails. This will allows us to fix up any
> places in the kernel where we should be checking the return status and
> maintain back compatibility with firmware images that may have incorrect
> interrupt configurations.

Though silently returning 0 is really the wrong thing to do. You can add the
warn, but why do you want to return success?

Thanks,

tglx



2016-03-17 14:56:56

by Thomas Gleixner

[permalink] [raw]
Subject: Re: [PATCH 07/15] irqdomain: Don't set type when mapping an IRQ

On Thu, 17 Mar 2016, Jon Hunter wrote:
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -1117,6 +1117,13 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
> new->irq = irq;
>
> /*
> + * If the trigger type is not specified by the caller,
> + * then use the default for this interrupt.
> + */
> + if (!(new->flags & IRQF_TRIGGER_MASK))
> + new->flags |= irqd_get_trigger_type(&desc->irq_data);
> +

This change is independent of the irqdomain part and should be in a seperate
patch because it affects all callers.

Thanks,

tglx


2016-03-17 15:02:11

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Hi Jon,

On 17/03/16 14:19, Jon Hunter wrote:
> Some IRQ chips may be located in a power domain outside of the CPU
> subsystem and hence will require device specific runtime power
> management. In order to support such IRQ chips, add a pointer for a
> device structure to the irq_chip structure, and if this pointer is
> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
> configuration, then the pm_runtime_get/put APIs for this chip will be
> called when an IRQ is requested/freed, respectively.
>
> When entering system suspend and each interrupt is disabled if there is
> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
> power management, print a warning message for each active interrupt that
> has no wake-up set because these interrupts may be unnecessarily keeping
> the IRQ chip enabled during system suspend.
>
> Signed-off-by: Jon Hunter <[email protected]>
> ---
> include/linux/irq.h | 5 +++++
> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
> kernel/irq/internals.h | 1 +
> kernel/irq/manage.c | 14 +++++++++++---
> kernel/irq/pm.c | 3 +++
> 5 files changed, 72 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index c4de62348ff2..82f36390048d 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
> /**
> * struct irq_chip - hardware interrupt chip descriptor
> *
> + * @parent: pointer to associated device
> * @name: name for /proc/interrupts
> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
> * @flags: chip specific flags
> */
> struct irq_chip {
> + struct device *parent;

Nit: Please don't call this just "parent". We have parent fields in
irq_data and irq_domain structures, and they always are a pointer to the
same type, indicating some form of stacking. Here, we're pointing to a
different type altogether...

How about calling it "dev", or "device" instead? It would make it much
clearer that when crossing that pointer, we're in another subsystem
altogether.

I'll come back to the rest of the patch a bit later, but I wanted to put
that one out right away... ;-)

Thanks,

M.
--
Jazz is not dead. It just smells funny...

2016-03-17 15:04:13

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails


On 17/03/16 14:51, Thomas Gleixner wrote:
> On Thu, 17 Mar 2016, Jon Hunter wrote:
>
>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>> whether this is allowed. There is no way to know if setting the type is
>> supported for a given GIC and so the value written is read back to
>> verify it matches the desired configuration. If it does not match then
>> an error is return.
>>
>> There are cases where the interrupt configuration read from firmware
>> (such as a device-tree blob), has been incorrect and hence
>> gic_configure_irq() has returned an error. This error has gone
>> undetected because the error code returned was ignored but the interrupt
>> still worked fine because the configuration for the interrupt could not
>> be overwritten.
>>
>> Given that this has done undetected and we should only fail to set the
>> type for PPIs whose configuration cannot be changed anyway, don't return
>> an error and simply WARN if this fails. This will allows us to fix up any
>> places in the kernel where we should be checking the return status and
>> maintain back compatibility with firmware images that may have incorrect
>> interrupt configurations.
>
> Though silently returning 0 is really the wrong thing to do. You can add the
> warn, but why do you want to return success?

Yes that would be the correct thing to do I agree. However, the problem
is that if we do this, then after the patch "irqdomain: Don't set type
when mapping an IRQ" is applied, we may break interrupts for some
existing device-tree binaries that have bad configuration (such as omap4
and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
is a back compatibility issue.

If you are wondering why these interrupts break after "irqdomain: Don't
set type when mapping an IRQ", it is because today
irq_create_fwspec_mapping() does not check the return code from setting
the type, but if we defer setting the type until __setup_irq() which
does check the return code, then all of a sudden interrupts that were
working (even with bad configurations) start to fail.

The reason why I opted not to return an error code from
gic_configure_irq() is it really can't fail. The failure being reported
does not prevent the interrupt from working, but tells you your
configuration does not match the hardware setting which you cannot
overwrite.

So to maintain back compatibility and avoid any silent errors, I opted
to make it a WARN and not return an error.

If people are ok with potentially breaking interrupts for device-tree
binaries with bad settings, then I am ok to return an error here.

Cheers
Jon



2016-03-17 15:04:21

by Thomas Gleixner

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

On Thu, 17 Mar 2016, Jon Hunter wrote:
> /**
> * struct irq_chip - hardware interrupt chip descriptor
> *
> + * @parent: pointer to associated device

That's really a bad name. parent suggests that this is a parent interrupt chip
and your explanation sucks as well. What's an associated device? Network card?

> #include <linux/irqdesc.h>
> #include <linux/kernel_stat.h>
> +#include <linux/pm_runtime.h>
>
> #ifdef CONFIG_SPARSE_IRQ
> # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
> index b2a93a37f772..65878e7c7c82 100644
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -1114,6 +1114,10 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
> if (!try_module_get(desc->owner))
> return -ENODEV;
>
> + ret = irq_chip_pm_get(&desc->irq_data);
> + if (ret < 0)
> + goto out_mput;

So this call nests inside the chip_bus_lock() region. Is that intentional?

Thanks,

tglx

2016-03-17 15:06:40

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 07/15] irqdomain: Don't set type when mapping an IRQ


On 17/03/16 14:55, Thomas Gleixner wrote:
> On Thu, 17 Mar 2016, Jon Hunter wrote:
>> --- a/kernel/irq/manage.c
>> +++ b/kernel/irq/manage.c
>> @@ -1117,6 +1117,13 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
>> new->irq = irq;
>>
>> /*
>> + * If the trigger type is not specified by the caller,
>> + * then use the default for this interrupt.
>> + */
>> + if (!(new->flags & IRQF_TRIGGER_MASK))
>> + new->flags |= irqd_get_trigger_type(&desc->irq_data);
>> +
>
> This change is independent of the irqdomain part and should be in a seperate
> patch because it affects all callers.

Ok, I will put this into a separate patch.

Cheers
Jon

2016-03-17 15:13:28

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Hi Marc,

On 17/03/16 15:02, Marc Zyngier wrote:
> Hi Jon,
>
> On 17/03/16 14:19, Jon Hunter wrote:
>> Some IRQ chips may be located in a power domain outside of the CPU
>> subsystem and hence will require device specific runtime power
>> management. In order to support such IRQ chips, add a pointer for a
>> device structure to the irq_chip structure, and if this pointer is
>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>> configuration, then the pm_runtime_get/put APIs for this chip will be
>> called when an IRQ is requested/freed, respectively.
>>
>> When entering system suspend and each interrupt is disabled if there is
>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>> power management, print a warning message for each active interrupt that
>> has no wake-up set because these interrupts may be unnecessarily keeping
>> the IRQ chip enabled during system suspend.
>>
>> Signed-off-by: Jon Hunter <[email protected]>
>> ---
>> include/linux/irq.h | 5 +++++
>> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>> kernel/irq/internals.h | 1 +
>> kernel/irq/manage.c | 14 +++++++++++---
>> kernel/irq/pm.c | 3 +++
>> 5 files changed, 72 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index c4de62348ff2..82f36390048d 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>> /**
>> * struct irq_chip - hardware interrupt chip descriptor
>> *
>> + * @parent: pointer to associated device
>> * @name: name for /proc/interrupts
>> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
>> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>> * @flags: chip specific flags
>> */
>> struct irq_chip {
>> + struct device *parent;
>
> Nit: Please don't call this just "parent". We have parent fields in
> irq_data and irq_domain structures, and they always are a pointer to the
> same type, indicating some form of stacking. Here, we're pointing to a
> different type altogether...
>
> How about calling it "dev", or "device" instead? It would make it much
> clearer that when crossing that pointer, we're in another subsystem
> altogether.

I will defer to Linus W here, as it was his request we make this
'parent' and not 'dev'. See ...

http://marc.info/?l=linux-kernel&m=145035839623442&w=2

> I'll come back to the rest of the patch a bit later, but I wanted to put
> that one out right away... ;-)

Thanks,
Jon

2016-03-17 15:18:08

by Jason Cooper

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On Thu, Mar 17, 2016 at 03:04:01PM +0000, Jon Hunter wrote:
>
> On 17/03/16 14:51, Thomas Gleixner wrote:
> > On Thu, 17 Mar 2016, Jon Hunter wrote:
> >
> >> Setting the interrupt type for private peripheral interrupts (PPIs) may
> >> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
> >> whether this is allowed. There is no way to know if setting the type is
> >> supported for a given GIC and so the value written is read back to
> >> verify it matches the desired configuration. If it does not match then
> >> an error is return.
> >>
> >> There are cases where the interrupt configuration read from firmware
> >> (such as a device-tree blob), has been incorrect and hence
> >> gic_configure_irq() has returned an error. This error has gone
> >> undetected because the error code returned was ignored but the interrupt
> >> still worked fine because the configuration for the interrupt could not
> >> be overwritten.
> >>
> >> Given that this has done undetected and we should only fail to set the
> >> type for PPIs whose configuration cannot be changed anyway, don't return
> >> an error and simply WARN if this fails. This will allows us to fix up any
> >> places in the kernel where we should be checking the return status and
> >> maintain back compatibility with firmware images that may have incorrect
> >> interrupt configurations.
> >
> > Though silently returning 0 is really the wrong thing to do. You can add the
> > warn, but why do you want to return success?
>
> Yes that would be the correct thing to do I agree. However, the problem
> is that if we do this, then after the patch "irqdomain: Don't set type
> when mapping an IRQ" is applied, we may break interrupts for some
> existing device-tree binaries that have bad configuration (such as omap4
> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
> is a back compatibility issue.

This sounds like a textbook case for adding a boolean dt property. If
"can-set-ppi-type" is absent (old DT blobs and new blobs without the
ability), warn and return zero. If it's present, the driver can set the
type, returning errors as encountered.

thx,

Jason.

2016-03-17 15:28:44

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

On 17/03/16 15:13, Jon Hunter wrote:
> Hi Marc,
>
> On 17/03/16 15:02, Marc Zyngier wrote:
>> Hi Jon,
>>
>> On 17/03/16 14:19, Jon Hunter wrote:
>>> Some IRQ chips may be located in a power domain outside of the CPU
>>> subsystem and hence will require device specific runtime power
>>> management. In order to support such IRQ chips, add a pointer for a
>>> device structure to the irq_chip structure, and if this pointer is
>>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>>> configuration, then the pm_runtime_get/put APIs for this chip will be
>>> called when an IRQ is requested/freed, respectively.
>>>
>>> When entering system suspend and each interrupt is disabled if there is
>>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>>> power management, print a warning message for each active interrupt that
>>> has no wake-up set because these interrupts may be unnecessarily keeping
>>> the IRQ chip enabled during system suspend.
>>>
>>> Signed-off-by: Jon Hunter <[email protected]>
>>> ---
>>> include/linux/irq.h | 5 +++++
>>> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>> kernel/irq/internals.h | 1 +
>>> kernel/irq/manage.c | 14 +++++++++++---
>>> kernel/irq/pm.c | 3 +++
>>> 5 files changed, 72 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>>> index c4de62348ff2..82f36390048d 100644
>>> --- a/include/linux/irq.h
>>> +++ b/include/linux/irq.h
>>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>> /**
>>> * struct irq_chip - hardware interrupt chip descriptor
>>> *
>>> + * @parent: pointer to associated device
>>> * @name: name for /proc/interrupts
>>> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
>>> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
>>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>> * @flags: chip specific flags
>>> */
>>> struct irq_chip {
>>> + struct device *parent;
>>
>> Nit: Please don't call this just "parent". We have parent fields in
>> irq_data and irq_domain structures, and they always are a pointer to the
>> same type, indicating some form of stacking. Here, we're pointing to a
>> different type altogether...
>>
>> How about calling it "dev", or "device" instead? It would make it much
>> clearer that when crossing that pointer, we're in another subsystem
>> altogether.
>
> I will defer to Linus W here, as it was his request we make this
> 'parent' and not 'dev'. See ...
>
> http://marc.info/?l=linux-kernel&m=145035839623442&w=2

Well, that contradicts the way use use the word "parent" in the IRQ
subsystem, I guess. I'd settle for parent_device or something along
those lines (but keep in mind I'm really bad at naming things).

Anyway, enough bikeshedding... ;-)

M.
--
Jazz is not dead. It just smells funny...

2016-03-17 15:46:19

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips


On 17/03/16 15:02, Thomas Gleixner wrote:
> On Thu, 17 Mar 2016, Jon Hunter wrote:
>> /**
>> * struct irq_chip - hardware interrupt chip descriptor
>> *
>> + * @parent: pointer to associated device
>
> That's really a bad name. parent suggests that this is a parent interrupt chip
> and your explanation sucks as well. What's an associated device? Network card?

Linus, can you re-iterate your concerns here about just using 'dev' for
the name?

>> #include <linux/irqdesc.h>
>> #include <linux/kernel_stat.h>
>> +#include <linux/pm_runtime.h>
>>
>> #ifdef CONFIG_SPARSE_IRQ
>> # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
>> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
>> index b2a93a37f772..65878e7c7c82 100644
>> --- a/kernel/irq/manage.c
>> +++ b/kernel/irq/manage.c
>> @@ -1114,6 +1114,10 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
>> if (!try_module_get(desc->owner))
>> return -ENODEV;
>>
>> + ret = irq_chip_pm_get(&desc->irq_data);
>> + if (ret < 0)
>> + goto out_mput;
>
> So this call nests inside the chip_bus_lock() region. Is that intentional?

Hmm ... I was trying to simplify the call paths, but yes I think it
would be safer to move outside. Ok, will fix that.

Cheers
Jon

2016-03-17 16:20:25

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails


On 17/03/16 15:18, Jason Cooper wrote:
> On Thu, Mar 17, 2016 at 03:04:01PM +0000, Jon Hunter wrote:
>>
>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>
>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>> whether this is allowed. There is no way to know if setting the type is
>>>> supported for a given GIC and so the value written is read back to
>>>> verify it matches the desired configuration. If it does not match then
>>>> an error is return.
>>>>
>>>> There are cases where the interrupt configuration read from firmware
>>>> (such as a device-tree blob), has been incorrect and hence
>>>> gic_configure_irq() has returned an error. This error has gone
>>>> undetected because the error code returned was ignored but the interrupt
>>>> still worked fine because the configuration for the interrupt could not
>>>> be overwritten.
>>>>
>>>> Given that this has done undetected and we should only fail to set the
>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>> places in the kernel where we should be checking the return status and
>>>> maintain back compatibility with firmware images that may have incorrect
>>>> interrupt configurations.
>>>
>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>> warn, but why do you want to return success?
>>
>> Yes that would be the correct thing to do I agree. However, the problem
>> is that if we do this, then after the patch "irqdomain: Don't set type
>> when mapping an IRQ" is applied, we may break interrupts for some
>> existing device-tree binaries that have bad configuration (such as omap4
>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>> is a back compatibility issue.
>
> This sounds like a textbook case for adding a boolean dt property. If
> "can-set-ppi-type" is absent (old DT blobs and new blobs without the
> ability), warn and return zero. If it's present, the driver can set the
> type, returning errors as encountered.

True. However, if we did have this "can-set-ppi-type" property set for a
device, it really should never fail (unless someone specified it
incorrectly). So I am trying to understand the value in adding a new DT
property.

Please note that gic_configure_irq() never used to return an error and
only when adding support for setting the type of PPIs was this added.
However, given that this has gone unnoticed and does not have a real
functional impact on the device behaviour, I wonder now if this function
should return an error? Yes, ideally, it should, but does it still make
sense?

Cheers
Jon

2016-03-17 18:18:19

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 06/15] irqdomain: Ensure type settings match for an existing mapping


On 17/03/16 14:19, Jon Hunter wrote:
> When mapping an IRQ, if a mapping already exists, then we simply return
> the virtual IRQ number. However, we do not check that the type settings
> for the existing mapping match those for the mapping that is about to be
> created. It may be unlikely that the type settings would not match, but
> check for this and don't return a valid IRQ mapping if the type settings
> do not match.
>
> WARN if the type return by irq_domain_translate() has bits outside the
> sense mask set and then clear these bits. If these bits are not cleared
> then this will cause the comparision of the type settings for an
> existing mapping to fail with that of the new mapping even if the sense
> bit themselves match. The reason being is that the existing type
> settings are read by calling irq_get_trigger_type() which will clear
> any bits outside the sense mask. This will allow us to detect irqchips
> that are not correctly clearing these bits and fix them.
>
> Signed-off-by: Jon Hunter <[email protected]>
> ---
> kernel/irq/irqdomain.c | 59 ++++++++++++++++++++++++++++++++++++--------------
> 1 file changed, 43 insertions(+), 16 deletions(-)
>
> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
> index 3a519a01118b..0ea285baa619 100644
> --- a/kernel/irq/irqdomain.c
> +++ b/kernel/irq/irqdomain.c
> @@ -549,6 +549,13 @@ static int irq_domain_translate(struct irq_domain *d,
> fwspec->param, fwspec->param_count,
> hwirq, type);
>
> + /*
> + * WARN if the irqchip returns a type with bits
> + * outside the sense mask set and clear these bits.
> + */
> + if (WARN_ON(*type & ~IRQ_TYPE_SENSE_MASK))
> + *type &= IRQ_TYPE_SENSE_MASK;
> +
> /* If domain has no translation, then we assume interrupt line */
> *hwirq = fwspec->param[0];
> return 0;
> @@ -570,6 +577,7 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
> {
> struct irq_domain *domain;
> irq_hw_number_t hwirq;
> + unsigned int cur_type = IRQ_TYPE_NONE;
> unsigned int type = IRQ_TYPE_NONE;
> int virq;
>
> @@ -592,23 +600,42 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
> if (irq_domain_translate(domain, fwspec, &hwirq, &type))
> return 0;
>
> - if (irq_domain_is_hierarchy(domain)) {
> - /*
> - * If we've already configured this interrupt,
> - * don't do it again, or hell will break loose.
> - */
> - virq = irq_find_mapping(domain, hwirq);
> - if (virq)
> - return virq;
> -
> - virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, fwspec);
> - if (virq <= 0)
> - return 0;
> + /*
> + * If we've already configured this interrupt,
> + * don't do it again, or hell will break loose.
> + */
> + virq = irq_find_mapping(domain, hwirq);
> + if (!virq) {
> + if (irq_domain_is_hierarchy(domain)) {
> + virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE,
> + fwspec);
> + if (virq <= 0)
> + return 0;
> + } else {
> + /* Create mapping */
> + virq = irq_create_mapping(domain, hwirq);
> + if (!virq)
> + return virq;
> + }
> } else {
> - /* Create mapping */
> - virq = irq_create_mapping(domain, hwirq);
> - if (!virq)
> - return virq;
> + cur_type = irq_get_trigger_type(virq);
> + }
> +
> + /*
> + * If the trigger type is not specified or matches the current
> + * trigger type then we are done so return the interrupt number.
> + */
> + if (type == IRQ_TYPE_NONE || type == cur_type)
> + return virq;
> +
> + /*
> + * If the trigger type is already set and does
> + * not match this interrupt, then return 0.
> + */
> + if (cur_type != IRQ_TYPE_NONE) {
> + pr_warn("type mismatch, failed to map hwirq-%lu for %s!\n",
> + hwirq, of_node_full_name(to_of_node(fwspec->fwnode)));
> + return 0;

If we created a new mapping and return here, then this will leak the
mapping. I will fix this. Sorry should have caught this before!

Jon


2016-03-17 18:19:29

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 07/15] irqdomain: Don't set type when mapping an IRQ


On 17/03/16 14:19, Jon Hunter wrote:
> Some IRQ chips, such as GPIO controllers or secondary level interrupt
> controllers, may require require additional runtime power management
> control to ensure they are accessible. For such IRQ chips, it makes sense
> to enable the IRQ chip when interrupts are requested and disabled them
> again once all interrupts have been freed.
>
> When mapping an IRQ, the IRQ type settings are read and then programmed.
> The mapping of the IRQ happens before the IRQ is requested and so the
> programming of the type settings occurs before the IRQ is requested. This
> is a problem for IRQ chips that require additional power management
> control because they may not be accessible yet. Therefore, when mapping
> the IRQ, don't program the type settings, just save them and then program
> these saved settings when the IRQ is requested (so long as if they are not
> overridden via the call to request the IRQ).
>
> Add a stub function for irq_domain_free_irqs() to avoid any compilation
> errors when CONFIG_IRQ_DOMAIN_HIERARCHY is not selected.
>
> Signed-off-by: Jon Hunter <[email protected]>
> ---
> include/linux/irqdomain.h | 3 +++
> kernel/irq/irqdomain.c | 18 ++++++++++++++----
> kernel/irq/manage.c | 7 +++++++
> 3 files changed, 24 insertions(+), 4 deletions(-)
>
> diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
> index 2aed04396210..fc66876d1965 100644
> --- a/include/linux/irqdomain.h
> +++ b/include/linux/irqdomain.h
> @@ -440,6 +440,9 @@ static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
> return -1;
> }
>
> +static inline void irq_domain_free_irqs(unsigned int virq,
> + unsigned int nr_irqs) { }
> +
> static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
> {
> return false;
> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
> index 0ea285baa619..332bee2205e0 100644
> --- a/kernel/irq/irqdomain.c
> +++ b/kernel/irq/irqdomain.c
> @@ -576,6 +576,7 @@ static void of_phandle_args_to_fwspec(struct of_phandle_args *irq_data,
> unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
> {
> struct irq_domain *domain;
> + struct irq_data *irq_data;
> irq_hw_number_t hwirq;
> unsigned int cur_type = IRQ_TYPE_NONE;
> unsigned int type = IRQ_TYPE_NONE;
> @@ -638,10 +639,19 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec)
> return 0;
> }
>
> - /* Set type if specified and different than the current one */
> - if (type != IRQ_TYPE_NONE &&
> - type != irq_get_trigger_type(virq))
> - irq_set_irq_type(virq, type);
> + irq_data = irq_get_irq_data(virq);
> + if (!irq_data) {
> + if (irq_domain_is_hierarchy(domain))
> + irq_domain_free_irqs(virq, 1);
> + else
> + irq_dispose_mapping(virq);
> +
> + return 0;
> + }

I should only free/dispose here if we actually created it! Will fix this
as well :-(

Jon

2016-03-17 20:14:57

by Rob Herring

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

On Thu, Mar 17, 2016 at 9:19 AM, Jon Hunter <[email protected]> wrote:
> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
> Domain properties") documented optional clock and power-dmoain properties
> for the ARM GIC. Currently, there are no users of these and for the
> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
> functional clock and interface clock, that need to be enabled.
>
> To allow flexibility, drop the 'clock-names' from the GIC binding and
> just provide a list of clocks which the driver can parse. It is assumed
> that any clocks that are listed, need to be enabled in order to access
> the GIC.
>
> Signed-off-by: Jon Hunter <[email protected]>
>
> ---
>
> Please note that I am not sure if this will be popular, but I am trying
> to come up with a generic way to handle multiple clocks that may be
> required for accessing a GIC.

It's not. :)

We need to specify the number and order of clocks by compatible string
at a minimum. Sadly, ARM's GICs are well documented and include clock
names, so you can't just make up genericish names either which is
probably often the case.

Rob

2016-03-18 08:37:30

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document


On 17/03/16 20:14, Rob Herring wrote:
> On Thu, Mar 17, 2016 at 9:19 AM, Jon Hunter <[email protected]> wrote:
>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>> Domain properties") documented optional clock and power-dmoain properties
>> for the ARM GIC. Currently, there are no users of these and for the
>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>> functional clock and interface clock, that need to be enabled.
>>
>> To allow flexibility, drop the 'clock-names' from the GIC binding and
>> just provide a list of clocks which the driver can parse. It is assumed
>> that any clocks that are listed, need to be enabled in order to access
>> the GIC.
>>
>> Signed-off-by: Jon Hunter <[email protected]>
>>
>> ---
>>
>> Please note that I am not sure if this will be popular, but I am trying
>> to come up with a generic way to handle multiple clocks that may be
>> required for accessing a GIC.
>
> It's not. :)
>
> We need to specify the number and order of clocks by compatible string
> at a minimum. Sadly, ARM's GICs are well documented and include clock
> names, so you can't just make up genericish names either which is
> probably often the case.

Do you have any suggestions then?

I have had a look at the ARM TRMs and although I see that they do show
the functional clock, there is no mention of whether there are any other
clocks need in order to interface to the GIC (ie. bus clock). I know
that for other SoCs such as OMAP it is common to have both a functional
clock and interface clock. So I believe this is fairly common.

Cheers
Jon

2016-03-18 09:13:40

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Hi Jon,

On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
> Domain properties") documented optional clock and power-dmoain properties
> for the ARM GIC. Currently, there are no users of these and for the
> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
> functional clock and interface clock, that need to be enabled.

The reason that there are no users for this is twofold:
1. The GIC driver doesn't have Runtime PM support yet,
2. There was no clean way to prevent the GIC's clock from being disabled.
Due to this, adding the clocks to the DTSes would mean that they will be
disabled during boot up as unused clocks, leading to a system lock-up.

I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
but unfortunately it seems the platform driver only supports non-root
controllers, while the r8a7791 GIC is the primary one...

Alternatively, part 2 can to be fixed by "clk: introduce CLK_ENABLE_HAND_OFF
flag", combined with the clock driver setting the flag when needed.
Unfortunately that patch is not yet upstream, and not even in -next.
Note that drivers/clk/renesas/renesas-cpg-mssr.c already handles
CLK_ENABLE_HAND_OFF if present, and else just ignores the clock.
So I could already add the clock to r8a7795.dtsi, which uses that driver.

For older SoCs, the module clocks are described in the dtsi, and I would need a
crude hack to enable CLK_ENABLE_HAND_OFF in the clock driver.

> To allow flexibility, drop the 'clock-names' from the GIC binding and
> just provide a list of clocks which the driver can parse. It is assumed
> that any clocks that are listed, need to be enabled in order to access
> the GIC.

Originally I just wanted to have "clocks", and let the details be handled by
SoC-specific code. However, Mark Rutland insisted on using the clock naming
from the GIC TRMs, as the number of clocks and their names depend on the
GIC variant.

Apparently they also depend on the SoC...

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 09:20:45

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On Thu, Mar 17, 2016 at 5:20 PM, Jon Hunter <[email protected]> wrote:
> On 17/03/16 15:18, Jason Cooper wrote:
>> On Thu, Mar 17, 2016 at 03:04:01PM +0000, Jon Hunter wrote:
>>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>>> whether this is allowed. There is no way to know if setting the type is
>>>>> supported for a given GIC and so the value written is read back to
>>>>> verify it matches the desired configuration. If it does not match then
>>>>> an error is return.
>>>>>
>>>>> There are cases where the interrupt configuration read from firmware
>>>>> (such as a device-tree blob), has been incorrect and hence
>>>>> gic_configure_irq() has returned an error. This error has gone
>>>>> undetected because the error code returned was ignored but the interrupt
>>>>> still worked fine because the configuration for the interrupt could not
>>>>> be overwritten.
>>>>>
>>>>> Given that this has done undetected and we should only fail to set the
>>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>>> places in the kernel where we should be checking the return status and
>>>>> maintain back compatibility with firmware images that may have incorrect
>>>>> interrupt configurations.
>>>>
>>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>>> warn, but why do you want to return success?
>>>
>>> Yes that would be the correct thing to do I agree. However, the problem
>>> is that if we do this, then after the patch "irqdomain: Don't set type
>>> when mapping an IRQ" is applied, we may break interrupts for some
>>> existing device-tree binaries that have bad configuration (such as omap4
>>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>>> is a back compatibility issue.

Indeed (also for sh73a0 and r8a7779).

>> This sounds like a textbook case for adding a boolean dt property. If
>> "can-set-ppi-type" is absent (old DT blobs and new blobs without the
>> ability), warn and return zero. If it's present, the driver can set the
>> type, returning errors as encountered.
>
> True. However, if we did have this "can-set-ppi-type" property set for a
> device, it really should never fail (unless someone specified it
> incorrectly). So I am trying to understand the value in adding a new DT
> property.

Do we really want to add properties that basically indicate that a description
in DT is correct?

Alternatively, it can be fixed in the kernel in a DT quirk (if SoC == xxx then
fix TWD).

> Please note that gic_configure_irq() never used to return an error and
> only when adding support for setting the type of PPIs was this added.
> However, given that this has gone unnoticed and does not have a real
> functional impact on the device behaviour, I wonder now if this function
> should return an error? Yes, ideally, it should, but does it still make
> sense?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 09:54:57

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails


On 18/03/16 09:20, Geert Uytterhoeven wrote:
> On Thu, Mar 17, 2016 at 5:20 PM, Jon Hunter <[email protected]> wrote:
>> On 17/03/16 15:18, Jason Cooper wrote:
>>> On Thu, Mar 17, 2016 at 03:04:01PM +0000, Jon Hunter wrote:
>>>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>>>> whether this is allowed. There is no way to know if setting the type is
>>>>>> supported for a given GIC and so the value written is read back to
>>>>>> verify it matches the desired configuration. If it does not match then
>>>>>> an error is return.
>>>>>>
>>>>>> There are cases where the interrupt configuration read from firmware
>>>>>> (such as a device-tree blob), has been incorrect and hence
>>>>>> gic_configure_irq() has returned an error. This error has gone
>>>>>> undetected because the error code returned was ignored but the interrupt
>>>>>> still worked fine because the configuration for the interrupt could not
>>>>>> be overwritten.
>>>>>>
>>>>>> Given that this has done undetected and we should only fail to set the
>>>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>>>> places in the kernel where we should be checking the return status and
>>>>>> maintain back compatibility with firmware images that may have incorrect
>>>>>> interrupt configurations.
>>>>>
>>>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>>>> warn, but why do you want to return success?
>>>>
>>>> Yes that would be the correct thing to do I agree. However, the problem
>>>> is that if we do this, then after the patch "irqdomain: Don't set type
>>>> when mapping an IRQ" is applied, we may break interrupts for some
>>>> existing device-tree binaries that have bad configuration (such as omap4
>>>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>>>> is a back compatibility issue.
>
> Indeed (also for sh73a0 and r8a7779).

Thanks. I was wondering if there are others. Do you know what the
correct setting should be? Ie. should it be IRQ_TYPE_EDGE_RISING as
well? I can then include this with OMAP and Tegra.

>>> This sounds like a textbook case for adding a boolean dt property. If
>>> "can-set-ppi-type" is absent (old DT blobs and new blobs without the
>>> ability), warn and return zero. If it's present, the driver can set the
>>> type, returning errors as encountered.
>>
>> True. However, if we did have this "can-set-ppi-type" property set for a
>> device, it really should never fail (unless someone specified it
>> incorrectly). So I am trying to understand the value in adding a new DT
>> property.
>
> Do we really want to add properties that basically indicate that a description
> in DT is correct?
>
> Alternatively, it can be fixed in the kernel in a DT quirk (if SoC == xxx then
> fix TWD).

I am not sure I fully understand your proposal, but please note that it
may not be just limited to the TWD (although this does appear to be the
one client that is wrong for a lot of SoCs). PPIs are also used for the
armv7/8 timers as well.

The problem is that we have a lot of SoCs with twd-timers and I have no
way to test all of these to know which could be a problem. So I thought
that warning would be a good first step to fixing them.

However, I am still trying to see the real value in returning an error
in this case. May be I am the only one with that perspective?

Cheers
Jon

2016-03-18 10:04:26

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 06/15] irqdomain: Ensure type settings match for an existing mapping

On 03/17/2016 08:18 PM, Jon Hunter wrote:
>
> On 17/03/16 14:19, Jon Hunter wrote:
>> When mapping an IRQ, if a mapping already exists, then we simply return
>> the virtual IRQ number. However, we do not check that the type settings
>> for the existing mapping match those for the mapping that is about to be
>> created. It may be unlikely that the type settings would not match, but
>> check for this and don't return a valid IRQ mapping if the type settings
>> do not match.
>>
>> WARN if the type return by irq_domain_translate() has bits outside the
>> sense mask set and then clear these bits. If these bits are not cleared
>> then this will cause the comparision of the type settings for an
>> existing mapping to fail with that of the new mapping even if the sense
>> bit themselves match. The reason being is that the existing type
>> settings are read by calling irq_get_trigger_type() which will clear
>> any bits outside the sense mask. This will allow us to detect irqchips
>> that are not correctly clearing these bits and fix them.
>>
>> Signed-off-by: Jon Hunter <[email protected]>
>> ---
>> kernel/irq/irqdomain.c | 59 ++++++++++++++++++++++++++++++++++++--------------
>> 1 file changed, 43 insertions(+), 16 deletions(-)
>>
>> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
>> index 3a519a01118b..0ea285baa619 100644
>> --- a/kernel/irq/irqdomain.c
>> +++ b/kernel/irq/irqdomain.c
>> @@ -549,6 +549,13 @@ static int irq_domain_translate(struct irq_domain *d,
>> fwspec->param, fwspec->param_count,
>> hwirq, type);
>>


>> + /*
>> + * WARN if the irqchip returns a type with bits
>> + * outside the sense mask set and clear these bits.
>> + */
>> + if (WARN_ON(*type & ~IRQ_TYPE_SENSE_MASK))
>> + *type &= IRQ_TYPE_SENSE_MASK;

Not sure that this warn and in this place make sense.
type will come unchanged here from caller irq_create_fwspec_mapping()



[...]


--
regards,
-grygorii

2016-03-18 10:14:11

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document


On 18/03/16 09:13, Geert Uytterhoeven wrote:
> Hi Jon,
>
> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>> Domain properties") documented optional clock and power-dmoain properties
>> for the ARM GIC. Currently, there are no users of these and for the
>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>> functional clock and interface clock, that need to be enabled.
>
> The reason that there are no users for this is twofold:
> 1. The GIC driver doesn't have Runtime PM support yet,
> 2. There was no clean way to prevent the GIC's clock from being disabled.
> Due to this, adding the clocks to the DTSes would mean that they will be
> disabled during boot up as unused clocks, leading to a system lock-up.
>
> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
> but unfortunately it seems the platform driver only supports non-root
> controllers, while the r8a7791 GIC is the primary one...

Can you try making the following change ...

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 9e7cf7abf757..2e971e600036 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -1372,7 +1372,7 @@ static int gic_probe(struct platform_device *pdev)
void __iomem *dist_base;
void __iomem *cpu_base;
u32 percpu_offset;
- int ret, irq;
+ int ret, irq = 0;

if (dev->of_node == NULL)
return -EINVAL;
@@ -1393,11 +1393,8 @@ static int gic_probe(struct platform_device *pdev)
if (ret < 0)
goto rpm_disable;

- irq = irq_of_parse_and_map(dev->of_node, 0);
- if (!irq) {
- ret = -EINVAL;
- goto rpm_put;
- }
+ if (of_irq_count(dev->of_node) > 0)
+ irq = irq_of_parse_and_map(dev->of_node, 0);

ret = gic_of_setup(dev->of_node, &dist_base, &cpu_base, &percpu_offset);
if (ret)
@@ -1411,7 +1408,8 @@ static int gic_probe(struct platform_device *pdev)

gic->chip.parent = dev;

- irq_set_chained_handler_and_data(irq, gic_handle_cascade_irq, gic);
+ if (irq)
+ irq_set_chained_handler_and_data(irq, gic_handle_cascade_irq, gic);

pm_runtime_put(dev);

@@ -1424,7 +1422,6 @@ gic_unmap:
iounmap(cpu_base);
irq_dispose:
irq_dispose_mapping(irq);
-rpm_put:
pm_runtime_put_sync(dev);
rpm_disable:
pm_runtime_disable(dev);


> Alternatively, part 2 can to be fixed by "clk: introduce CLK_ENABLE_HAND_OFF
> flag", combined with the clock driver setting the flag when needed.
> Unfortunately that patch is not yet upstream, and not even in -next.
> Note that drivers/clk/renesas/renesas-cpg-mssr.c already handles
> CLK_ENABLE_HAND_OFF if present, and else just ignores the clock.
> So I could already add the clock to r8a7795.dtsi, which uses that driver.
>
> For older SoCs, the module clocks are described in the dtsi, and I would need a
> crude hack to enable CLK_ENABLE_HAND_OFF in the clock driver.
>
>> To allow flexibility, drop the 'clock-names' from the GIC binding and
>> just provide a list of clocks which the driver can parse. It is assumed
>> that any clocks that are listed, need to be enabled in order to access
>> the GIC.
>
> Originally I just wanted to have "clocks", and let the details be handled by
> SoC-specific code. However, Mark Rutland insisted on using the clock naming
> from the GIC TRMs, as the number of clocks and their names depend on the
> GIC variant.
>
> Apparently they also depend on the SoC...

Yes this case is a little different because the GIC is a 2nd level GIC.

Cheers
Jon

2016-03-18 10:23:04

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

Hi Jon,

On Fri, Mar 18, 2016 at 10:54 AM, Jon Hunter <[email protected]> wrote:
> On 18/03/16 09:20, Geert Uytterhoeven wrote:
>> On Thu, Mar 17, 2016 at 5:20 PM, Jon Hunter <[email protected]> wrote:
>>> On 17/03/16 15:18, Jason Cooper wrote:
>>>> On Thu, Mar 17, 2016 at 03:04:01PM +0000, Jon Hunter wrote:
>>>>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>>>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>>>>> whether this is allowed. There is no way to know if setting the type is
>>>>>>> supported for a given GIC and so the value written is read back to
>>>>>>> verify it matches the desired configuration. If it does not match then
>>>>>>> an error is return.
>>>>>>>
>>>>>>> There are cases where the interrupt configuration read from firmware
>>>>>>> (such as a device-tree blob), has been incorrect and hence
>>>>>>> gic_configure_irq() has returned an error. This error has gone
>>>>>>> undetected because the error code returned was ignored but the interrupt
>>>>>>> still worked fine because the configuration for the interrupt could not
>>>>>>> be overwritten.
>>>>>>>
>>>>>>> Given that this has done undetected and we should only fail to set the
>>>>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>>>>> places in the kernel where we should be checking the return status and
>>>>>>> maintain back compatibility with firmware images that may have incorrect
>>>>>>> interrupt configurations.
>>>>>>
>>>>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>>>>> warn, but why do you want to return success?
>>>>>
>>>>> Yes that would be the correct thing to do I agree. However, the problem
>>>>> is that if we do this, then after the patch "irqdomain: Don't set type
>>>>> when mapping an IRQ" is applied, we may break interrupts for some
>>>>> existing device-tree binaries that have bad configuration (such as omap4
>>>>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>>>>> is a back compatibility issue.
>>
>> Indeed (also for sh73a0 and r8a7779).
>
> Thanks. I was wondering if there are others. Do you know what the
> correct setting should be? Ie. should it be IRQ_TYPE_EDGE_RISING as
> well? I can then include this with OMAP and Tegra.

The warnings went away when using IRQ_TYPE_EDGE_RISING.
Patches sent.

>>>> This sounds like a textbook case for adding a boolean dt property. If
>>>> "can-set-ppi-type" is absent (old DT blobs and new blobs without the
>>>> ability), warn and return zero. If it's present, the driver can set the
>>>> type, returning errors as encountered.
>>>
>>> True. However, if we did have this "can-set-ppi-type" property set for a
>>> device, it really should never fail (unless someone specified it
>>> incorrectly). So I am trying to understand the value in adding a new DT
>>> property.
>>
>> Do we really want to add properties that basically indicate that a description
>> in DT is correct?
>>
>> Alternatively, it can be fixed in the kernel in a DT quirk (if SoC == xxx then
>> fix TWD).
>
> I am not sure I fully understand your proposal, but please note that it

The kernel can modify the DT in early startup code if it detects that the
TWD's interrupt type is wrong.

> may not be just limited to the TWD (although this does appear to be the
> one client that is wrong for a lot of SoCs). PPIs are also used for the
> armv7/8 timers as well.

True.

> The problem is that we have a lot of SoCs with twd-timers and I have no
> way to test all of these to know which could be a problem. So I thought
> that warning would be a good first step to fixing them.

Definitely.

> However, I am still trying to see the real value in returning an error
> in this case. May be I am the only one with that perspective?

Given the above, we cannot start returning an error until all problems are
fixed.

Even after that, we have to care about stable DT ABIs. In this case it's not
really an ABI change, but a real buggy description in DTS.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 10:33:44

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 06/15] irqdomain: Ensure type settings match for an existing mapping


On 18/03/16 10:03, Grygorii Strashko wrote:
> On 03/17/2016 08:18 PM, Jon Hunter wrote:
>>
>> On 17/03/16 14:19, Jon Hunter wrote:
>>> When mapping an IRQ, if a mapping already exists, then we simply return
>>> the virtual IRQ number. However, we do not check that the type settings
>>> for the existing mapping match those for the mapping that is about to be
>>> created. It may be unlikely that the type settings would not match, but
>>> check for this and don't return a valid IRQ mapping if the type settings
>>> do not match.
>>>
>>> WARN if the type return by irq_domain_translate() has bits outside the
>>> sense mask set and then clear these bits. If these bits are not cleared
>>> then this will cause the comparision of the type settings for an
>>> existing mapping to fail with that of the new mapping even if the sense
>>> bit themselves match. The reason being is that the existing type
>>> settings are read by calling irq_get_trigger_type() which will clear
>>> any bits outside the sense mask. This will allow us to detect irqchips
>>> that are not correctly clearing these bits and fix them.
>>>
>>> Signed-off-by: Jon Hunter <[email protected]>
>>> ---
>>> kernel/irq/irqdomain.c | 59
>>> ++++++++++++++++++++++++++++++++++++--------------
>>> 1 file changed, 43 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
>>> index 3a519a01118b..0ea285baa619 100644
>>> --- a/kernel/irq/irqdomain.c
>>> +++ b/kernel/irq/irqdomain.c
>>> @@ -549,6 +549,13 @@ static int irq_domain_translate(struct
>>> irq_domain *d,
>>> fwspec->param, fwspec->param_count,
>>> hwirq, type);
>>>
>
>
>>> + /*
>>> + * WARN if the irqchip returns a type with bits
>>> + * outside the sense mask set and clear these bits.
>>> + */
>>> + if (WARN_ON(*type & ~IRQ_TYPE_SENSE_MASK))
>>> + *type &= IRQ_TYPE_SENSE_MASK;
>
> Not sure that this warn and in this place make sense.
> type will come unchanged here from caller irq_create_fwspec_mapping()

Yes you are right. I missed the return statement. I can move to
irq_create_fwspec_mapping().

Cheers
Jon

2016-03-18 10:52:21

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Hi Jon,

On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>> Domain properties") documented optional clock and power-dmoain properties
>>> for the ARM GIC. Currently, there are no users of these and for the
>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>> functional clock and interface clock, that need to be enabled.
>>
>> The reason that there are no users for this is twofold:
>> 1. The GIC driver doesn't have Runtime PM support yet,
>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>> Due to this, adding the clocks to the DTSes would mean that they will be
>> disabled during boot up as unused clocks, leading to a system lock-up.
>>
>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>> but unfortunately it seems the platform driver only supports non-root
>> controllers, while the r8a7791 GIC is the primary one...
>
> Can you try making the following change ...

Thanks! I gave it a try, but no difference.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 10:57:10

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document


On 18/03/16 10:52, Geert Uytterhoeven wrote:
> Hi Jon,
>
> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>> Domain properties") documented optional clock and power-dmoain properties
>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>> functional clock and interface clock, that need to be enabled.
>>>
>>> The reason that there are no users for this is twofold:
>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>
>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>> but unfortunately it seems the platform driver only supports non-root
>>> controllers, while the r8a7791 GIC is the primary one...
>>
>> Can you try making the following change ...
>
> Thanks! I gave it a try, but no difference.

I assume you added the appropriate compatible flag? Any more details you
can share about why it is not working? Is it not registered early enough?

Jon

2016-03-18 11:12:00

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Hi Jon,

On 03/17/2016 04:19 PM, Jon Hunter wrote:
> Some IRQ chips may be located in a power domain outside of the CPU
> subsystem and hence will require device specific runtime power
> management. In order to support such IRQ chips, add a pointer for a
> device structure to the irq_chip structure, and if this pointer is
> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
> configuration, then the pm_runtime_get/put APIs for this chip will be
> called when an IRQ is requested/freed, respectively.
>
> When entering system suspend and each interrupt is disabled if there is
> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
> power management, print a warning message for each active interrupt that
> has no wake-up set because these interrupts may be unnecessarily keeping
> the IRQ chip enabled during system suspend.
>
> Signed-off-by: Jon Hunter <[email protected]>
> ---
> include/linux/irq.h | 5 +++++
> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
> kernel/irq/internals.h | 1 +
> kernel/irq/manage.c | 14 +++++++++++---
> kernel/irq/pm.c | 3 +++
> 5 files changed, 72 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index c4de62348ff2..82f36390048d 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
> /**
> * struct irq_chip - hardware interrupt chip descriptor
> *
> + * @parent: pointer to associated device
> * @name: name for /proc/interrupts
> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
> * @flags: chip specific flags
> */

[..]

>
> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
> index cea1de0161f1..ab436119084f 100644
> --- a/kernel/irq/pm.c
> +++ b/kernel/irq/pm.c
> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
> * suspend_device_irqs().
> */
> return true;
> + } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
> + pr_warn("irq %d has no wakeup set and has not been freed!\n",
> + desc->irq_data.irq);

Sry. But I did not get this part of the patch :(

static bool suspend_device_irq(struct irq_desc *desc)
{
if (!desc->action || irq_desc_is_chained(desc) ||
desc->no_suspend_depth) {
pr_err("skip irq %d\n", irq_desc_get_irq(desc));
return false;
}

if (irqd_is_wakeup_set(&desc->irq_data)) {
irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
/*
* We return true here to force the caller to issue
* synchronize_irq(). We need to make sure that the
* IRQD_WAKEUP_ARMED is visible before we return from
* suspend_device_irqs().
*/
pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
return true;
}

^^^^ Here you've added a warning

desc->istate |= IRQS_SUSPENDED;
__disable_irq(desc);

^^^^ Here non wakeup IRQs will be disabled

pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));

/*
* Hardware which has no wakeup source configuration facility
* requires that the non wakeup interrupts are masked at the
* chip level. The chip implementation indicates that with
* IRQCHIP_MASK_ON_SUSPEND.
*/
if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
mask_irq(desc);
pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
}

return true;
}

As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
is wakeup source, but all other are not.

Also I'd like to note that:
- it is not expected that any IRQs have to be freed on enter Suspend
- Primary interrupt controller is expected to be suspended from syscore_suspend()
- not Primary interrupt controllers may be Suspended from:
-- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
GPIO expanders (I2C, SPI ..)
-- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
dpm_suspend_noirq
|- suspend_device_irqs()
|- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
-- as always, some arches/maches may require hacks in platform code.

So, In my opinion, suspend has to be handled by each irqchip driver separately,
most probably at suspend_noirq level [1], because only irqchip driver
now sees a full picture and knows if it can suspend or not, and when, and how.
(may require to use pm_runtime_force_suspend/resume()).

I propose do not touch common/generic suspend code now. Any common code can be always
refactored later once there will be real drivers updated to use irqchip RPM
and which will support Suspend.


--
regards,
-grygorii

2016-03-18 12:05:35

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Hi Jon,

On Fri, Mar 18, 2016 at 11:56 AM, Jon Hunter <[email protected]> wrote:
> On 18/03/16 10:52, Geert Uytterhoeven wrote:
>> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>>> Domain properties") documented optional clock and power-dmoain properties
>>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>>> functional clock and interface clock, that need to be enabled.
>>>>
>>>> The reason that there are no users for this is twofold:
>>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>>
>>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>>> but unfortunately it seems the platform driver only supports non-root
>>>> controllers, while the r8a7791 GIC is the primary one...
>>>
>>> Can you try making the following change ...
>>
>> Thanks! I gave it a try, but no difference.
>
> I assume you added the appropriate compatible flag? Any more details you

Doh... bad assumption... Silly me.

> can share about why it is not working? Is it not registered early enough?

With

+ { .compatible = "arm,gic-400", },

the kernel no longer crashes due to accessing the GIC registers while the
GIC module clock is disabled.

However, the system doesn't boot completely, and time outs on SPI transfers
make me believe interrupts are not working.
Both with and without "the following change".

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 12:28:04

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips


On 18/03/16 11:11, Grygorii Strashko wrote:
> Hi Jon,
>
> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>> Some IRQ chips may be located in a power domain outside of the CPU
>> subsystem and hence will require device specific runtime power
>> management. In order to support such IRQ chips, add a pointer for a
>> device structure to the irq_chip structure, and if this pointer is
>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>> configuration, then the pm_runtime_get/put APIs for this chip will be
>> called when an IRQ is requested/freed, respectively.
>>
>> When entering system suspend and each interrupt is disabled if there is
>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>> power management, print a warning message for each active interrupt that
>> has no wake-up set because these interrupts may be unnecessarily keeping
>> the IRQ chip enabled during system suspend.
>>
>> Signed-off-by: Jon Hunter <[email protected]>
>> ---
>> include/linux/irq.h | 5 +++++
>> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>> kernel/irq/internals.h | 1 +
>> kernel/irq/manage.c | 14 +++++++++++---
>> kernel/irq/pm.c | 3 +++
>> 5 files changed, 72 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index c4de62348ff2..82f36390048d 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>> /**
>> * struct irq_chip - hardware interrupt chip descriptor
>> *
>> + * @parent: pointer to associated device
>> * @name: name for /proc/interrupts
>> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
>> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>> * @flags: chip specific flags
>> */
>
> [..]
>
>>
>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>> index cea1de0161f1..ab436119084f 100644
>> --- a/kernel/irq/pm.c
>> +++ b/kernel/irq/pm.c
>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>> * suspend_device_irqs().
>> */
>> return true;
>> + } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>> + pr_warn("irq %d has no wakeup set and has not been freed!\n",
>> + desc->irq_data.irq);
>
> Sry. But I did not get this part of the patch :(
>
> static bool suspend_device_irq(struct irq_desc *desc)
> {
> if (!desc->action || irq_desc_is_chained(desc) ||
> desc->no_suspend_depth) {
> pr_err("skip irq %d\n", irq_desc_get_irq(desc));
> return false;
> }
>
> if (irqd_is_wakeup_set(&desc->irq_data)) {
> irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
> /*
> * We return true here to force the caller to issue
> * synchronize_irq(). We need to make sure that the
> * IRQD_WAKEUP_ARMED is visible before we return from
> * suspend_device_irqs().
> */
> pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
> return true;
> }
>
> ^^^^ Here you've added a warning

Yes, to warn if the IRQ is enabled but not a wake-up source ...

if (irqd_is_wakeup_set(&desc->irq_data)) {
...
} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
...
}

> desc->istate |= IRQS_SUSPENDED;
> __disable_irq(desc);
>
> ^^^^ Here non wakeup IRQs will be disabled

Yes, but this will not turn off the irqchip. It is legitimate for the
chip to be enabled during suspend if an IRQ is enabled as a wakeup.

The purpose of the warning is to report any IRQs that are enabled at
this point, but NOT wake-up sources. These could be unintentionally be
keeping the chip active when it does not need to be.

> pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>
> /*
> * Hardware which has no wakeup source configuration facility
> * requires that the non wakeup interrupts are masked at the
> * chip level. The chip implementation indicates that with
> * IRQCHIP_MASK_ON_SUSPEND.
> */
> if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
> mask_irq(desc);
> pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
> }
>
> return true;
> }
>
> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
> is wakeup source, but all other are not.

No there should not be. Remember this is an else-if and ONLY if an IRQ
is not a wake-up source AND enabled will you get a warning.

> Also I'd like to note that:
> - it is not expected that any IRQs have to be freed on enter Suspend

True, but surely they should have a wake-up enabled then? If not you are
wasting power unnecessarily.

I realise that this is different to how interrupts for irqchips work
today, but when we discussed this before, the only way to ensure that we
can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
a slightly different mindset for irqchips with PM, that may be we
shouldn't hold references to IRQs forever if we are not using them.

> - Primary interrupt controller is expected to be suspended from syscore_suspend()
> - not Primary interrupt controllers may be Suspended from:
> -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
> GPIO expanders (I2C, SPI ..)
> -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
> dpm_suspend_noirq
> |- suspend_device_irqs()
> |- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
> -- as always, some arches/maches may require hacks in platform code.
>
> So, In my opinion, suspend has to be handled by each irqchip driver separately,
> most probably at suspend_noirq level [1], because only irqchip driver
> now sees a full picture and knows if it can suspend or not, and when, and how.
> (may require to use pm_runtime_force_suspend/resume()).

I understand what you are saying, but at least in my mind if would be
better if the clients of the IRQ chips using PM freed their interrupts
when entering suspend. Quite possibly I am overlooking a use-case here
or overhead of doing this, but it would avoid every irqchip having to
handle this themselves and having a custom handler.

> I propose do not touch common/generic suspend code now. Any common code can be always
> refactored later once there will be real drivers updated to use irqchip RPM
> and which will support Suspend.

If this is strongly opposed, I would concede to making this a pr_debug()
as I think it could be useful.

Jon

[0] http://marc.info/?l=linux-pm&m=145340595315514&w=2

2016-03-18 12:42:57

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document


On 18/03/16 12:05, Geert Uytterhoeven wrote:
> Hi Jon,
>
> On Fri, Mar 18, 2016 at 11:56 AM, Jon Hunter <[email protected]> wrote:
>> On 18/03/16 10:52, Geert Uytterhoeven wrote:
>>> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>>>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>>>> Domain properties") documented optional clock and power-dmoain properties
>>>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>>>> functional clock and interface clock, that need to be enabled.
>>>>>
>>>>> The reason that there are no users for this is twofold:
>>>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>>>
>>>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>>>> but unfortunately it seems the platform driver only supports non-root
>>>>> controllers, while the r8a7791 GIC is the primary one...
>>>>
>>>> Can you try making the following change ...
>>>
>>> Thanks! I gave it a try, but no difference.
>>
>> I assume you added the appropriate compatible flag? Any more details you
>
> Doh... bad assumption... Silly me.
>
>> can share about why it is not working? Is it not registered early enough?
>
> With
>
> + { .compatible = "arm,gic-400", },
>
> the kernel no longer crashes due to accessing the GIC registers while the
> GIC module clock is disabled.
>
> However, the system doesn't boot completely, and time outs on SPI transfers
> make me believe interrupts are not working.
> Both with and without "the following change".

Yes, I recall now why I did not support primary controllers and it is
because you need to call set_smp_cross_call() (for SMP) and
set_handle_irq(). Both of which are located in the __init section and
need to be called early during boot. So to make this work for primary
controllers, more work would need to be done.

Jon

2016-03-18 12:48:12

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

On 03/18/2016 02:05 PM, Geert Uytterhoeven wrote:
> Hi Jon,
>
> On Fri, Mar 18, 2016 at 11:56 AM, Jon Hunter <[email protected]> wrote:
>> On 18/03/16 10:52, Geert Uytterhoeven wrote:
>>> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>>>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>>>> Domain properties") documented optional clock and power-dmoain properties
>>>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>>>> functional clock and interface clock, that need to be enabled.
>>>>>
>>>>> The reason that there are no users for this is twofold:
>>>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>>>
>>>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>>>> but unfortunately it seems the platform driver only supports non-root
>>>>> controllers, while the r8a7791 GIC is the primary one...
>>>>
>>>> Can you try making the following change ...
>>>
>>> Thanks! I gave it a try, but no difference.
>>
>> I assume you added the appropriate compatible flag? Any more details you
>
> Doh... bad assumption... Silly me.
>
>> can share about why it is not working? Is it not registered early enough?
>
> With
>
> + { .compatible = "arm,gic-400", },
>
> the kernel no longer crashes due to accessing the GIC registers while the
> GIC module clock is disabled.
>
> However, the system doesn't boot completely, and time outs on SPI transfers
> make me believe interrupts are not working.
> Both with and without "the following change".
>

Is my assumption correct that you are trying to enable RPM for primary GIC controller?


If yes it may help to take a look on clocksource drivers which use early_platform_device/driver
sh_cmt.c sh_mtu2.c sh_tmu.c

The primary interrupt controller is initialized very early init_IRQ->irqchip_init->of_irq_init()
(IRQCHIP_DECLARE) and, at least as i can see from st_xxx code, the same case is valid for
clocksource devices and it was solved using early_platform_device/drive staff.


--
regards,
-grygorii

2016-03-18 13:02:50

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Hi Grygorii,

On Fri, Mar 18, 2016 at 1:47 PM, Grygorii Strashko
<[email protected]> wrote:
> On 03/18/2016 02:05 PM, Geert Uytterhoeven wrote:
>> On Fri, Mar 18, 2016 at 11:56 AM, Jon Hunter <[email protected]> wrote:
>>> On 18/03/16 10:52, Geert Uytterhoeven wrote:
>>>> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>>>>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>>>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>>>>> Domain properties") documented optional clock and power-dmoain properties
>>>>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>>>>> functional clock and interface clock, that need to be enabled.
>>>>>>
>>>>>> The reason that there are no users for this is twofold:
>>>>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>>>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>>>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>>>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>>>>
>>>>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>>>>> but unfortunately it seems the platform driver only supports non-root
>>>>>> controllers, while the r8a7791 GIC is the primary one...
>>>>>
>>>>> Can you try making the following change ...
>>>>
>>>> Thanks! I gave it a try, but no difference.
>>>
>>> I assume you added the appropriate compatible flag? Any more details you
>>
>> Doh... bad assumption... Silly me.
>>
>>> can share about why it is not working? Is it not registered early enough?
>>
>> With
>>
>> + { .compatible = "arm,gic-400", },
>>
>> the kernel no longer crashes due to accessing the GIC registers while the
>> GIC module clock is disabled.
>>
>> However, the system doesn't boot completely, and time outs on SPI transfers
>> make me believe interrupts are not working.
>> Both with and without "the following change".
>>
>
> Is my assumption correct that you are trying to enable RPM for primary GIC controller?

That's correct.

> If yes it may help to take a look on clocksource drivers which use early_platform_device/driver
> sh_cmt.c sh_mtu2.c sh_tmu.c
>
> The primary interrupt controller is initialized very early init_IRQ->irqchip_init->of_irq_init()
> (IRQCHIP_DECLARE) and, at least as i can see from st_xxx code, the same case is valid for
> clocksource devices and it was solved using early_platform_device/drive staff.

The GIC now depends on the clock driver, which may be a real platform driver,
not initialized from CLK_OF_DECLARE().

Or do you mean to make the clock driver an early platform driver?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-18 14:24:57

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

On 03/18/2016 02:27 PM, Jon Hunter wrote:
>
> On 18/03/16 11:11, Grygorii Strashko wrote:
>> Hi Jon,
>>
>> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>>> Some IRQ chips may be located in a power domain outside of the CPU
>>> subsystem and hence will require device specific runtime power
>>> management. In order to support such IRQ chips, add a pointer for a
>>> device structure to the irq_chip structure, and if this pointer is
>>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>>> configuration, then the pm_runtime_get/put APIs for this chip will be
>>> called when an IRQ is requested/freed, respectively.
>>>
>>> When entering system suspend and each interrupt is disabled if there is
>>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>>> power management, print a warning message for each active interrupt that
>>> has no wake-up set because these interrupts may be unnecessarily keeping
>>> the IRQ chip enabled during system suspend.
>>>
>>> Signed-off-by: Jon Hunter <[email protected]>
>>> ---
>>> include/linux/irq.h | 5 +++++
>>> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>> kernel/irq/internals.h | 1 +
>>> kernel/irq/manage.c | 14 +++++++++++---
>>> kernel/irq/pm.c | 3 +++
>>> 5 files changed, 72 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>>> index c4de62348ff2..82f36390048d 100644
>>> --- a/include/linux/irq.h
>>> +++ b/include/linux/irq.h
>>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>> /**
>>> * struct irq_chip - hardware interrupt chip descriptor
>>> *
>>> + * @parent: pointer to associated device
>>> * @name: name for /proc/interrupts
>>> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
>>> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
>>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>> * @flags: chip specific flags
>>> */
>>
>> [..]
>>
>>>
>>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>>> index cea1de0161f1..ab436119084f 100644
>>> --- a/kernel/irq/pm.c
>>> +++ b/kernel/irq/pm.c
>>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>>> * suspend_device_irqs().
>>> */
>>> return true;
>>> + } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>>> + pr_warn("irq %d has no wakeup set and has not been freed!\n",
>>> + desc->irq_data.irq);
>>
>> Sry. But I did not get this part of the patch :(
>>
>> static bool suspend_device_irq(struct irq_desc *desc)
>> {
>> if (!desc->action || irq_desc_is_chained(desc) ||
>> desc->no_suspend_depth) {
>> pr_err("skip irq %d\n", irq_desc_get_irq(desc));
>> return false;
>> }
>>
>> if (irqd_is_wakeup_set(&desc->irq_data)) {
>> irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
>> /*
>> * We return true here to force the caller to issue
>> * synchronize_irq(). We need to make sure that the
>> * IRQD_WAKEUP_ARMED is visible before we return from
>> * suspend_device_irqs().
>> */
>> pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
>> return true;
>> }
>>
>> ^^^^ Here you've added a warning
>
> Yes, to warn if the IRQ is enabled but not a wake-up source ...
>
> if (irqd_is_wakeup_set(&desc->irq_data)) {
> ...
> } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
> ...
> }
>
>> desc->istate |= IRQS_SUSPENDED;
>> __disable_irq(desc);
>>
>> ^^^^ Here non wakeup IRQs will be disabled
>
> Yes, but this will not turn off the irqchip. It is legitimate for the
> chip to be enabled during suspend if an IRQ is enabled as a wakeup.
>
> The purpose of the warning is to report any IRQs that are enabled at
> this point, but NOT wake-up sources. These could be unintentionally be
> keeping the chip active when it does not need to be.
>
>> pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>
>> /*
>> * Hardware which has no wakeup source configuration facility
>> * requires that the non wakeup interrupts are masked at the
>> * chip level. The chip implementation indicates that with
>> * IRQCHIP_MASK_ON_SUSPEND.
>> */
>> if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
>> mask_irq(desc);
>> pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>> }
>>
>> return true;
>> }
>>
>> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
>> is wakeup source, but all other are not.
>
> No there should not be. Remember this is an else-if and ONLY if an IRQ
> is not a wake-up source AND enabled will you get a warning.

Sorry, but I don't see the "AND enabled" check any where in this file and
disabled irqs can be wakeup source - they shouldn't be masked.
But I'll stop commenting until i reproduce it.

Or do you mean free?

>
>> Also I'd like to note that:
>> - it is not expected that any IRQs have to be freed on enter Suspend
>
> True, but surely they should have a wake-up enabled then? If not you are
> wasting power unnecessarily.
>
> I realise that this is different to how interrupts for irqchips work
> today, but when we discussed this before, the only way to ensure that we
> can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
> a slightly different mindset for irqchips with PM, that may be we
> shouldn't hold references to IRQs forever if we are not using them.
>
>> - Primary interrupt controller is expected to be suspended from syscore_suspend()
>> - not Primary interrupt controllers may be Suspended from:
>> -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
>> GPIO expanders (I2C, SPI ..)
>> -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
>> dpm_suspend_noirq
>> |- suspend_device_irqs()
>> |- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
>> -- as always, some arches/maches may require hacks in platform code.
>>
>> So, In my opinion, suspend has to be handled by each irqchip driver separately,
>> most probably at suspend_noirq level [1], because only irqchip driver
>> now sees a full picture and knows if it can suspend or not, and when, and how.
>> (may require to use pm_runtime_force_suspend/resume()).
>
> I understand what you are saying, but at least in my mind if would be
> better if the clients of the IRQ chips using PM freed their interrupts
> when entering suspend. Quite possibly I am overlooking a use-case here
> or overhead of doing this,

ok. seems you do mean "free".

oh :( That will require updating of all drivers (and if it will be taken into account that
wakeup can be configured from sysfs + devm_ - it will be painful).

> but it would avoid every irqchip having to
> handle this themselves and having a custom handler.

irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
to be Powered off during Suspend or deep CPUIdle states, simply because its state
in suspend is unknown - PM state managed automatically (and depends on many factors)
and wakeup can be handled by special HW in case if GPIO bank was really switched off.

>> I propose do not touch common/generic suspend code now. Any common code can be always
>> refactored later once there will be real drivers updated to use irqchip RPM
>> and which will support Suspend.
>
> If this is strongly opposed, I would concede to making this a pr_debug()
> as I think it could be useful.

Probably yes, because most of the drivers now and IRQ PM core are not ready
for this approach.

--
regards,
-grygorii

2016-03-18 14:40:47

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips


On 18/03/16 14:23, Grygorii Strashko wrote:
> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>
>> On 18/03/16 11:11, Grygorii Strashko wrote:
>>> Hi Jon,
>>>
>>> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>>>> Some IRQ chips may be located in a power domain outside of the CPU
>>>> subsystem and hence will require device specific runtime power
>>>> management. In order to support such IRQ chips, add a pointer for a
>>>> device structure to the irq_chip structure, and if this pointer is
>>>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>>>> configuration, then the pm_runtime_get/put APIs for this chip will be
>>>> called when an IRQ is requested/freed, respectively.
>>>>
>>>> When entering system suspend and each interrupt is disabled if there is
>>>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>>>> power management, print a warning message for each active interrupt that
>>>> has no wake-up set because these interrupts may be unnecessarily keeping
>>>> the IRQ chip enabled during system suspend.
>>>>
>>>> Signed-off-by: Jon Hunter <[email protected]>
>>>> ---
>>>> include/linux/irq.h | 5 +++++
>>>> kernel/irq/chip.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> kernel/irq/internals.h | 1 +
>>>> kernel/irq/manage.c | 14 +++++++++++---
>>>> kernel/irq/pm.c | 3 +++
>>>> 5 files changed, 72 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>>>> index c4de62348ff2..82f36390048d 100644
>>>> --- a/include/linux/irq.h
>>>> +++ b/include/linux/irq.h
>>>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>> /**
>>>> * struct irq_chip - hardware interrupt chip descriptor
>>>> *
>>>> + * @parent: pointer to associated device
>>>> * @name: name for /proc/interrupts
>>>> * @irq_startup: start up the interrupt (defaults to ->enable if NULL)
>>>> * @irq_shutdown: shut down the interrupt (defaults to ->disable if NULL)
>>>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>> * @flags: chip specific flags
>>>> */
>>>
>>> [..]
>>>
>>>>
>>>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>>>> index cea1de0161f1..ab436119084f 100644
>>>> --- a/kernel/irq/pm.c
>>>> +++ b/kernel/irq/pm.c
>>>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>>>> * suspend_device_irqs().
>>>> */
>>>> return true;
>>>> + } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>>>> + pr_warn("irq %d has no wakeup set and has not been freed!\n",
>>>> + desc->irq_data.irq);
>>>
>>> Sry. But I did not get this part of the patch :(
>>>
>>> static bool suspend_device_irq(struct irq_desc *desc)
>>> {
>>> if (!desc->action || irq_desc_is_chained(desc) ||
>>> desc->no_suspend_depth) {
>>> pr_err("skip irq %d\n", irq_desc_get_irq(desc));
>>> return false;
>>> }
>>>
>>> if (irqd_is_wakeup_set(&desc->irq_data)) {
>>> irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
>>> /*
>>> * We return true here to force the caller to issue
>>> * synchronize_irq(). We need to make sure that the
>>> * IRQD_WAKEUP_ARMED is visible before we return from
>>> * suspend_device_irqs().
>>> */
>>> pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
>>> return true;
>>> }
>>>
>>> ^^^^ Here you've added a warning
>>
>> Yes, to warn if the IRQ is enabled but not a wake-up source ...
>>
>> if (irqd_is_wakeup_set(&desc->irq_data)) {
>> ...
>> } else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>> ...
>> }
>>
>>> desc->istate |= IRQS_SUSPENDED;
>>> __disable_irq(desc);
>>>
>>> ^^^^ Here non wakeup IRQs will be disabled
>>
>> Yes, but this will not turn off the irqchip. It is legitimate for the
>> chip to be enabled during suspend if an IRQ is enabled as a wakeup.
>>
>> The purpose of the warning is to report any IRQs that are enabled at
>> this point, but NOT wake-up sources. These could be unintentionally be
>> keeping the chip active when it does not need to be.
>>
>>> pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>>
>>> /*
>>> * Hardware which has no wakeup source configuration facility
>>> * requires that the non wakeup interrupts are masked at the
>>> * chip level. The chip implementation indicates that with
>>> * IRQCHIP_MASK_ON_SUSPEND.
>>> */
>>> if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
>>> mask_irq(desc);
>>> pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>> }
>>>
>>> return true;
>>> }
>>>
>>> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
>>> is wakeup source, but all other are not.
>>
>> No there should not be. Remember this is an else-if and ONLY if an IRQ
>> is not a wake-up source AND enabled will you get a warning.
>
> Sorry, but I don't see the "AND enabled" check any where in this file and
> disabled irqs can be wakeup source - they shouldn't be masked.
> But I'll stop commenting until i reproduce it.
>
> Or do you mean free?

Yes, to be correct I mean not a wake-up source AND not freed (requested).

>>
>>> Also I'd like to note that:
>>> - it is not expected that any IRQs have to be freed on enter Suspend
>>
>> True, but surely they should have a wake-up enabled then? If not you are
>> wasting power unnecessarily.
>>
>> I realise that this is different to how interrupts for irqchips work
>> today, but when we discussed this before, the only way to ensure that we
>> can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
>> a slightly different mindset for irqchips with PM, that may be we
>> shouldn't hold references to IRQs forever if we are not using them.
>>
>>> - Primary interrupt controller is expected to be suspended from syscore_suspend()
>>> - not Primary interrupt controllers may be Suspended from:
>>> -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
>>> GPIO expanders (I2C, SPI ..)
>>> -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
>>> dpm_suspend_noirq
>>> |- suspend_device_irqs()
>>> |- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
>>> -- as always, some arches/maches may require hacks in platform code.
>>>
>>> So, In my opinion, suspend has to be handled by each irqchip driver separately,
>>> most probably at suspend_noirq level [1], because only irqchip driver
>>> now sees a full picture and knows if it can suspend or not, and when, and how.
>>> (may require to use pm_runtime_force_suspend/resume()).
>>
>> I understand what you are saying, but at least in my mind if would be
>> better if the clients of the IRQ chips using PM freed their interrupts
>> when entering suspend. Quite possibly I am overlooking a use-case here
>> or overhead of doing this,
>
> ok. seems you do mean "free".
>
> oh :( That will require updating of all drivers (and if it will be taken into account that
> wakeup can be configured from sysfs + devm_ - it will be painful).

Will it? I know that there are a few gpio chips that have some hacked
ways to get around the PM issue, but I wonder how many drivers this
really impacts. What sysfs entries are you referring too?

>> but it would avoid every irqchip having to
>> handle this themselves and having a custom handler.
>
> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
> to be Powered off during Suspend or deep CPUIdle states, simply because its state
> in suspend is unknown - PM state managed automatically (and depends on many factors)
> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
>
>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>> refactored later once there will be real drivers updated to use irqchip RPM
>>> and which will support Suspend.
>>
>> If this is strongly opposed, I would concede to making this a pr_debug()
>> as I think it could be useful.
>
> Probably yes, because most of the drivers now and IRQ PM core are not ready
> for this approach.

May be this calls for a new flag to not WARN if non-wakeup IRQs are not
freed when entering suspend.

Cheers
Jon

2016-03-18 14:56:36

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips


On 18/03/16 14:40, Jon Hunter wrote:
> On 18/03/16 14:23, Grygorii Strashko wrote:
>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>
>>> On 18/03/16 11:11, Grygorii Strashko wrote:

[snip]

>> oh :( That will require updating of all drivers (and if it will be taken into account that
>> wakeup can be configured from sysfs + devm_ - it will be painful).
>
> Will it? I know that there are a few gpio chips that have some hacked
> ways to get around the PM issue, but I wonder how many drivers this
> really impacts. What sysfs entries are you referring too?

Thinking about this some more, yes I guess it would impact all drivers
that use a gpio but don't use it for a wake-up. I could see that could
be a few drivers indeed.

>>> but it would avoid every irqchip having to
>>> handle this themselves and having a custom handler.
>>
>> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
>> to be Powered off during Suspend or deep CPUIdle states, simply because its state
>> in suspend is unknown - PM state managed automatically (and depends on many factors)
>> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
>>
>>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>>> refactored later once there will be real drivers updated to use irqchip RPM
>>>> and which will support Suspend.
>>>
>>> If this is strongly opposed, I would concede to making this a pr_debug()
>>> as I think it could be useful.
>>
>> Probably yes, because most of the drivers now and IRQ PM core are not ready
>> for this approach.
>
> May be this calls for a new flag to not WARN if non-wakeup IRQs are not
> freed when entering suspend.

Flag or pr_debug()?

Jon


2016-03-18 15:41:54

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 02/15] ARM: OMAP: Correct interrupt type for ARM TWD

On 03/17/2016 04:19 PM, Jon Hunter wrote:
> The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
> the ARM GIC documentation, whether the type for PPIs can be set is
> IMPLEMENTATION DEFINED. For OMAP4 devices the PPI type cannot be set and
> so when we attempt to set the type for the ARM TWD interrupt it fails.
> This has done unnoticed because it fails silently and because we cannot
> re-configure the type it has had no impact. Nevertheless fix the type
> for the TWD interrupt so that it matches the hardware configuration.
>
> Reported-by: Grygorii Strashko <[email protected]>
> Signed-off-by: Jon Hunter <[email protected]>
>
> ---
>
> Tony, Grygorii,
> Please note that I have not tested this. Can you test this series and
> see if you see any warnings on OMAP4? I am guessing that the configuration
> should be LEVEL and not EDGE. This was reported here:

Tested-by: Grygorii Strashko <[email protected]>

Tested on PandaBoard. Without this patch I can see below warning:


[ 0.000000] OMAP clockevent source: timer1 at 32768 Hz
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] WARNING: CPU: 0 PID: 0 at drivers/irqchip/irq-gic-common.c:61 gic_configure_irq+0x84/0x90()
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.5.0-05765-gd232fe5 #6
[ 0.000000] Hardware name: Generic OMAP4 (Flattened Device Tree)
[ 0.000000] Backtrace:
[ 0.000000] [<c010bc80>] (dump_backtrace) from [<c010be78>] (show_stack+0x18/0x1c)
[ 0.000000] r7:00000000 r6:c0a448a4 r5:200001d3 r4:00000000
[ 0.000000] [<c010be60>] (show_stack) from [<c03f223c>] (dump_stack+0x98/0xb4)
[ 0.000000] [<c03f21a4>] (dump_stack) from [<c0136b38>] (warn_slowpath_common+0x7c/0xb8)
[ 0.000000] r7:c08664e0 r6:0000003d r5:00000009 r4:00000000
[ 0.000000] [<c0136abc>] (warn_slowpath_common) from [<c0136c18>] (warn_slowpath_null+0x24/0x2c)
[ 0.000000] r8:00000004 r7:c0a02d98 r6:00000000 r5:ee805cc0 r4:00000000
[ 0.000000] [<c0136bf4>] (warn_slowpath_null) from [<c0420708>] (gic_configure_irq+0x84/0x90)
[ 0.000000] [<c0420684>] (gic_configure_irq) from [<c042024c>] (gic_set_type+0x50/0x60)
[ 0.000000] [<c04201fc>] (gic_set_type) from [<c017f968>] (__irq_set_trigger+0x64/0x158)
[ 0.000000] r5:ee805cc0 r4:ee828540
[ 0.000000] [<c017f904>] (__irq_set_trigger) from [<c017ff7c>] (__setup_irq+0x520/0x614)
[ 0.000000] r9:600001d3 r8:ee805d20 r7:00000011 r6:00000011 r5:ee805cc0 r4:ee828540
[ 0.000000] [<c017fa5c>] (__setup_irq) from [<c0180394>] (request_percpu_irq+0x8c/0xf0)
[ 0.000000] r9:c0110004 r8:c083237c r7:c095da80 r6:00000011 r5:ee805cc0 r4:ee828540
[ 0.000000] [<c0180308>] (request_percpu_irq) from [<c09053e4>] (twd_local_timer_common_register+0x44/0x1bc)
[ 0.000000] r9:efffc000 r8:c0a7b000 r7:c0a02900 r6:ef6c75c8 r5:ef6c75c8 r4:c0a7b2cc
[ 0.000000] [<c09053a0>] (twd_local_timer_common_register) from [<c09055ac>] (twd_local_timer_of_register+0x50/0x78)
[ 0.000000] r9:efffc000 r8:c0a7b000 r7:c0a02900 r6:ffffffff r5:ef6c75c8 r4:c0a7b2cc
[ 0.000000] [<c090555c>] (twd_local_timer_of_register) from [<c0930ba4>] (clocksource_probe+0x50/0x90)
[ 0.000000] r5:00000001 r4:ef6c75c8
[ 0.000000] [<c0930b54>] (clocksource_probe) from [<c090d5b0>] (omap4_local_timer_init+0x14/0x18)
[ 0.000000] r5:c0a7b000 r4:00000000
[ 0.000000] [<c090d59c>] (omap4_local_timer_init) from [<c090494c>] (time_init+0x24/0x38)
[ 0.000000] [<c0904928>] (time_init) from [<c0900b84>] (start_kernel+0x220/0x3bc)
[ 0.000000] [<c0900964>] (start_kernel) from [<8000807c>] (0x8000807c)
[ 0.000000] r10:00000000 r9:411fc092 r8:8000406a r7:c0a073e4 r6:c0940a2c r5:c0a029b0
[ 0.000000] r4:c0a7b214
[ 0.000000] ---[ end trace cb88537fdc8fa200 ]---
[ 0.000000] sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns


--
regards,
-grygorii

2016-03-18 17:52:56

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

On 03/18/2016 04:56 PM, Jon Hunter wrote:
>
> On 18/03/16 14:40, Jon Hunter wrote:
>> On 18/03/16 14:23, Grygorii Strashko wrote:
>>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>>
>>>> On 18/03/16 11:11, Grygorii Strashko wrote:
>
> [snip]
>
>>> oh :( That will require updating of all drivers (and if it will be taken into account that
>>> wakeup can be configured from sysfs + devm_ - it will be painful).
>>
>> Will it? I know that there are a few gpio chips that have some hacked
>> ways to get around the PM issue, but I wonder how many drivers this
>> really impacts. What sysfs entries are you referring too?

echo enabled > /sys/devices/platform/44000000.ocp/48020000.serial/tty/ttyS2/power/wakeup

>
> Thinking about this some more, yes I guess it would impact all drivers
> that use a gpio but don't use it for a wake-up. I could see that could
> be a few drivers indeed.

yep. I've just tested it
- gpio was requested through sysfs and configured as IRQ
- do suspend

the same is if GPIO is requested as IRQ only and not configured as wakeup source

[ 319.669760] PM: late suspend of devices complete after 0.213 msecs
[ 319.671195] irq 191 has no wakeup set and has not been freed!
[ 319.673453] PM: noirq suspend of devices complete after 2.258 msecs

this is very minimal configuration - the regular one is at ~30-50 devices
most of them will use IRQ and only ~10% are used as wakeup sources.


>
>>>> but it would avoid every irqchip having to
>>>> handle this themselves and having a custom handler.
>>>
>>> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
>>> to be Powered off during Suspend or deep CPUIdle states, simply because its state
>>> in suspend is unknown - PM state managed automatically (and depends on many factors)
>>> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
>>>
>>>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>>>> refactored later once there will be real drivers updated to use irqchip RPM
>>>>> and which will support Suspend.
>>>>
>>>> If this is strongly opposed, I would concede to making this a pr_debug()
>>>> as I think it could be useful.
>>>
>>> Probably yes, because most of the drivers now and IRQ PM core are not ready
>>> for this approach.
>>
>> May be this calls for a new flag to not WARN if non-wakeup IRQs are not
>> freed when entering suspend.
>
> Flag or pr_debug()?
>

Honestly, I don't know how to proceed - minimum is pr_debug.
My personal opinion is still the same - don't touch suspend core code now, within this series.


--
regards,
-grygorii

2016-03-18 18:36:54

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

On 03/18/2016 03:02 PM, Geert Uytterhoeven wrote:
> Hi Grygorii,
>
> On Fri, Mar 18, 2016 at 1:47 PM, Grygorii Strashko
> <[email protected]> wrote:
>> On 03/18/2016 02:05 PM, Geert Uytterhoeven wrote:
>>> On Fri, Mar 18, 2016 at 11:56 AM, Jon Hunter <[email protected]> wrote:
>>>> On 18/03/16 10:52, Geert Uytterhoeven wrote:
>>>>> On Fri, Mar 18, 2016 at 11:13 AM, Jon Hunter <[email protected]> wrote:
>>>>>> On 18/03/16 09:13, Geert Uytterhoeven wrote:
>>>>>>> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>>>>>>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>>>>>>> Domain properties") documented optional clock and power-dmoain properties
>>>>>>>> for the ARM GIC. Currently, there are no users of these and for the
>>>>>>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>>>>>>> functional clock and interface clock, that need to be enabled.
>>>>>>>
>>>>>>> The reason that there are no users for this is twofold:
>>>>>>> 1. The GIC driver doesn't have Runtime PM support yet,
>>>>>>> 2. There was no clean way to prevent the GIC's clock from being disabled.
>>>>>>> Due to this, adding the clocks to the DTSes would mean that they will be
>>>>>>> disabled during boot up as unused clocks, leading to a system lock-up.
>>>>>>>
>>>>>>> I had hoped your series would fix part 1. I gave it a try on r8a7791/koelsch,
>>>>>>> but unfortunately it seems the platform driver only supports non-root
>>>>>>> controllers, while the r8a7791 GIC is the primary one...
>>>>>>
>>>>>> Can you try making the following change ...
>>>>>
>>>>> Thanks! I gave it a try, but no difference.
>>>>
>>>> I assume you added the appropriate compatible flag? Any more details you
>>>
>>> Doh... bad assumption... Silly me.
>>>
>>>> can share about why it is not working? Is it not registered early enough?
>>>
>>> With
>>>
>>> + { .compatible = "arm,gic-400", },
>>>
>>> the kernel no longer crashes due to accessing the GIC registers while the
>>> GIC module clock is disabled.
>>>
>>> However, the system doesn't boot completely, and time outs on SPI transfers
>>> make me believe interrupts are not working.
>>> Both with and without "the following change".
>>>
>>
>> Is my assumption correct that you are trying to enable RPM for primary GIC controller?
>
> That's correct.
>
>> If yes it may help to take a look on clocksource drivers which use early_platform_device/driver
>> sh_cmt.c sh_mtu2.c sh_tmu.c
>>
>> The primary interrupt controller is initialized very early init_IRQ->irqchip_init->of_irq_init()
>> (IRQCHIP_DECLARE) and, at least as i can see from st_xxx code, the same case is valid for
>> clocksource devices and it was solved using early_platform_device/drive staff.
>
> The GIC now depends on the clock driver, which may be a real platform driver,
> not initialized from CLK_OF_DECLARE().

Clock need to be accessible, but, seems, there is another issue -
if you will try to use gic_driver by just adding compatible string then,
most probably, gic_init_bases() will be called twice:
1: init_IRQ->irqchip_init->of_irq_init()->__gic_init_bases()->gic_init_bases()
2: gic_probe->gic_init_bases()


And GIC data will be replaced on the fly ;P

>
> Or do you mean to make the clock driver an early platform driver?

I can't say definitely - I've just studied it some time ago, but did not try it by myself.



--
regards,
-grygorii

2016-03-21 10:09:41

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips


On 18/03/16 17:52, Grygorii Strashko wrote:
> On 03/18/2016 04:56 PM, Jon Hunter wrote:
>>
>> On 18/03/16 14:40, Jon Hunter wrote:
>>> On 18/03/16 14:23, Grygorii Strashko wrote:
>>>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>>>
>>>>> On 18/03/16 11:11, Grygorii Strashko wrote:
>>
>> [snip]
>>
>>>> oh :( That will require updating of all drivers (and if it will be taken into account that
>>>> wakeup can be configured from sysfs + devm_ - it will be painful).
>>>
>>> Will it? I know that there are a few gpio chips that have some hacked
>>> ways to get around the PM issue, but I wonder how many drivers this
>>> really impacts. What sysfs entries are you referring too?
>
> echo enabled > /sys/devices/platform/44000000.ocp/48020000.serial/tty/ttyS2/power/wakeup
>
>>
>> Thinking about this some more, yes I guess it would impact all drivers
>> that use a gpio but don't use it for a wake-up. I could see that could
>> be a few drivers indeed.
>
> yep. I've just tested it
> - gpio was requested through sysfs and configured as IRQ
> - do suspend
>
> the same is if GPIO is requested as IRQ only and not configured as wakeup source
>
> [ 319.669760] PM: late suspend of devices complete after 0.213 msecs
> [ 319.671195] irq 191 has no wakeup set and has not been freed!
> [ 319.673453] PM: noirq suspend of devices complete after 2.258 msecs
>
> this is very minimal configuration - the regular one is at ~30-50 devices
> most of them will use IRQ and only ~10% are used as wakeup sources.

Then it is working as intended :-)

However, if this is too verbose for some irqchips, then as I mentioned
we can have a flag to avoid these messages.

Cheers
Jon



2016-03-22 11:47:01

by Linus Walleij

[permalink] [raw]
Subject: Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

On Thu, Mar 17, 2016 at 4:28 PM, Marc Zyngier <[email protected]> wrote:
> On 17/03/16 15:13, Jon Hunter wrote:

>>>> struct irq_chip {
>>>> + struct device *parent;
>>>
>>> Nit: Please don't call this just "parent". We have parent fields in
>>> irq_data and irq_domain structures, and they always are a pointer to the
>>> same type, indicating some form of stacking. Here, we're pointing to a
>>> different type altogether...
>>>
>>> How about calling it "dev", or "device" instead? It would make it much
>>> clearer that when crossing that pointer, we're in another subsystem
>>> altogether.
>>
>> I will defer to Linus W here, as it was his request we make this
>> 'parent' and not 'dev'. See ...
>>
>> http://marc.info/?l=linux-kernel&m=145035839623442&w=2
>
> Well, that contradicts the way use use the word "parent" in the IRQ
> subsystem, I guess. I'd settle for parent_device or something along
> those lines (but keep in mind I'm really bad at naming things).
>
> Anyway, enough bikeshedding... ;-)

I'm happy with .parent_device.

Yours,
Linus Walleij

2016-03-22 11:49:26

by Linus Walleij

[permalink] [raw]
Subject: Re: [PATCH 10/15] irqchip/gic: Remove static irq_chip definition for eoimode1

On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:

> There are only 3 differences (not including the name) in the definitions
> of the gic_chip and gic_eoimode1_chip structures. Instead of statically
> defining the gic_eoimode1_chip structure, remove it and populate the
> eoimode1 functions dynamically for the appropriate GIC irqchips.
>
> Signed-off-by: Jon Hunter <[email protected]>

> if (static_key_true(&supports_deactivate) && gic_nr == 0) {
> - gic->chip = gic_eoimode1_chip;
> + gic->chip.irq_mask = gic_eoimode1_mask_irq;
> + gic->chip.irq_eoi = gic_eoimode1_eoi_irq;
> + gic->chip.irq_set_vcpu_affinity = gic_irq_set_vcpu_affinity;
> + gic->chip.name = kasprintf(GFP_KERNEL, "GICv2");

I think you can just = "GICv2" there. No need to kasprintf()

Yours,
Linus Walleij

2016-03-29 13:04:06

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 13/15] irqchip/gic: Prepare for adding platform driver

Hi Jon,

On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c

> -#ifdef CONFIG_SMP
> - set_smp_cross_call(gic_raise_softirq);
> - register_cpu_notifier(&gic_cpu_notifier);
> -#endif


> + if (gic_nr == 0) {
> + if (IS_ENABLED(CONFIG_SMP)) {
> + set_smp_cross_call(gic_raise_softirq);

If CONFIG_SMP=n:

drivers/irqchip/irq-gic.c: In function '__gic_init_bases':
drivers/irqchip/irq-gic.c:1165:4: error: implicit declaration of
function 'set_smp_cross_call' [-Werror=implicit-function-declaration]
set_smp_cross_call(gic_raise_softirq);
^

There's no dummy for the CONFIG_SMP=n case, so you either have to
keep the #ifdef, or add a dummy.

> + register_cpu_notifier(&gic_cpu_notifier);
> + }




--
Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

2016-03-29 13:59:34

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 13/15] irqchip/gic: Prepare for adding platform driver

Hi Geert,

On 29/03/16 14:04, Geert Uytterhoeven wrote:
> Hi Jon,
>
> On Thu, Mar 17, 2016 at 3:19 PM, Jon Hunter <[email protected]> wrote:
>> --- a/drivers/irqchip/irq-gic.c
>> +++ b/drivers/irqchip/irq-gic.c
>
>> -#ifdef CONFIG_SMP
>> - set_smp_cross_call(gic_raise_softirq);
>> - register_cpu_notifier(&gic_cpu_notifier);
>> -#endif
>
>
>> + if (gic_nr == 0) {
>> + if (IS_ENABLED(CONFIG_SMP)) {
>> + set_smp_cross_call(gic_raise_softirq);
>
> If CONFIG_SMP=n:
>
> drivers/irqchip/irq-gic.c: In function '__gic_init_bases':
> drivers/irqchip/irq-gic.c:1165:4: error: implicit declaration of
> function 'set_smp_cross_call' [-Werror=implicit-function-declaration]
> set_smp_cross_call(gic_raise_softirq);
> ^
>
> There's no dummy for the CONFIG_SMP=n case, so you either have to
> keep the #ifdef, or add a dummy.
>
>> + register_cpu_notifier(&gic_cpu_notifier);
>> + }

Thanks. I have already fixed this in my tree locally.

Cheers
Jon

2016-03-29 14:02:29

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 02/15] ARM: OMAP: Correct interrupt type for ARM TWD

Hi Tony,

On 18/03/16 15:41, Grygorii Strashko wrote:
> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>> The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
>> the ARM GIC documentation, whether the type for PPIs can be set is
>> IMPLEMENTATION DEFINED. For OMAP4 devices the PPI type cannot be set and
>> so when we attempt to set the type for the ARM TWD interrupt it fails.
>> This has done unnoticed because it fails silently and because we cannot
>> re-configure the type it has had no impact. Nevertheless fix the type
>> for the TWD interrupt so that it matches the hardware configuration.
>>
>> Reported-by: Grygorii Strashko <[email protected]>
>> Signed-off-by: Jon Hunter <[email protected]>
>>
>> ---
>>
>> Tony, Grygorii,
>> Please note that I have not tested this. Can you test this series and
>> see if you see any warnings on OMAP4? I am guessing that the configuration
>> should be LEVEL and not EDGE. This was reported here:
>
> Tested-by: Grygorii Strashko <[email protected]>
>
> Tested on PandaBoard. Without this patch I can see below warning:

I think that you can pick up this fix independently of this series as it
is something that has been broken for sometime and should be fixed. I
included it here for completeness to highlight the issue but if you want
to take it now, please go ahead.

Cheers
Jon

2016-03-30 21:24:00

by Tony Lindgren

[permalink] [raw]
Subject: Re: [PATCH 02/15] ARM: OMAP: Correct interrupt type for ARM TWD

* Jon Hunter <[email protected]> [160329 07:03]:
> Hi Tony,
>
> On 18/03/16 15:41, Grygorii Strashko wrote:
> > On 03/17/2016 04:19 PM, Jon Hunter wrote:
> >> The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
> >> the ARM GIC documentation, whether the type for PPIs can be set is
> >> IMPLEMENTATION DEFINED. For OMAP4 devices the PPI type cannot be set and
> >> so when we attempt to set the type for the ARM TWD interrupt it fails.
> >> This has done unnoticed because it fails silently and because we cannot
> >> re-configure the type it has had no impact. Nevertheless fix the type
> >> for the TWD interrupt so that it matches the hardware configuration.
> >>
> >> Reported-by: Grygorii Strashko <[email protected]>
> >> Signed-off-by: Jon Hunter <[email protected]>
> >>
> >> ---
> >>
> >> Tony, Grygorii,
> >> Please note that I have not tested this. Can you test this series and
> >> see if you see any warnings on OMAP4? I am guessing that the configuration
> >> should be LEVEL and not EDGE. This was reported here:
> >
> > Tested-by: Grygorii Strashko <[email protected]>
> >
> > Tested on PandaBoard. Without this patch I can see below warning:
>
> I think that you can pick up this fix independently of this series as it
> is something that has been broken for sometime and should be fixed. I
> included it here for completeness to highlight the issue but if you want
> to take it now, please go ahead.

OK applying into omap-for-v4.6/fixes thanks.

Tony

2016-04-01 23:26:01

by Kevin Hilman

[permalink] [raw]
Subject: Re: [PATCH 01/15] ARM: tegra: Correct interrupt type for ARM TWD

Jon Hunter <[email protected]> writes:

> The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
> the ARM GIC documentation, whether the type for PPIs can be set is
> IMPLEMENTATION DEFINED. For Tegra20/30 devices the PPI type cannot be
> set and so when we attempt to set the type for the ARM TWD interrupt it
> fails. This has done unnoticed because it fails silently and because we

nit: s/done/gone/

> cannot re-configure the type it has had no impact. Nevertheless fix the
> type for the TWD interrupt so that it matches the hardware configuration.
>
> Signed-off-by: Jon Hunter <[email protected]>

Kevin

2016-04-05 14:04:24

by Thierry Reding

[permalink] [raw]
Subject: Re: [PATCH 01/15] ARM: tegra: Correct interrupt type for ARM TWD

On Thu, Mar 17, 2016 at 02:19:05PM +0000, Jon Hunter wrote:
> The ARM TWD interrupt is a private peripheral interrupt (PPI) and per
> the ARM GIC documentation, whether the type for PPIs can be set is
> IMPLEMENTATION DEFINED. For Tegra20/30 devices the PPI type cannot be
> set and so when we attempt to set the type for the ARM TWD interrupt it
> fails. This has done unnoticed because it fails silently and because we
> cannot re-configure the type it has had no impact. Nevertheless fix the
> type for the TWD interrupt so that it matches the hardware configuration.
>
> Signed-off-by: Jon Hunter <[email protected]>
>
> ---
> Ideally, we would not be attempting to set the type for an interrupt
> where it cannot be programmed but this would require changes to the
> device-tree bindings for the GIC. This series adds a WARNING to catch
> any of these silent failures.
> ---
> arch/arm/boot/dts/tegra20.dtsi | 2 +-
> arch/arm/boot/dts/tegra30.dtsi | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)

Applied, thanks.

Thierry


Attachments:
(No filename) (1.02 kB)
signature.asc (819.00 B)
Download all attachments

2016-04-09 10:50:44

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 03/15] irqchip/gic: Don't unnecessarily write the IRQ configuration

On Thu, 17 Mar 2016 14:19:07 +0000
Jon Hunter <[email protected]> wrote:

> If the interrupt configuration matches the current configuration, then
> don't bother writing the configuration again.
>
> Signed-off-by: Jon Hunter <[email protected]>

Acked-by: Marc Zyngier <[email protected]>

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 10:59:08

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On Thu, 17 Mar 2016 15:04:01 +0000
Jon Hunter <[email protected]> wrote:

>
> On 17/03/16 14:51, Thomas Gleixner wrote:
> > On Thu, 17 Mar 2016, Jon Hunter wrote:
> >
> >> Setting the interrupt type for private peripheral interrupts (PPIs) may
> >> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
> >> whether this is allowed. There is no way to know if setting the type is
> >> supported for a given GIC and so the value written is read back to
> >> verify it matches the desired configuration. If it does not match then
> >> an error is return.
> >>
> >> There are cases where the interrupt configuration read from firmware
> >> (such as a device-tree blob), has been incorrect and hence
> >> gic_configure_irq() has returned an error. This error has gone
> >> undetected because the error code returned was ignored but the interrupt
> >> still worked fine because the configuration for the interrupt could not
> >> be overwritten.
> >>
> >> Given that this has done undetected and we should only fail to set the
> >> type for PPIs whose configuration cannot be changed anyway, don't return
> >> an error and simply WARN if this fails. This will allows us to fix up any
> >> places in the kernel where we should be checking the return status and
> >> maintain back compatibility with firmware images that may have incorrect
> >> interrupt configurations.
> >
> > Though silently returning 0 is really the wrong thing to do. You can add the
> > warn, but why do you want to return success?
>
> Yes that would be the correct thing to do I agree. However, the problem
> is that if we do this, then after the patch "irqdomain: Don't set type
> when mapping an IRQ" is applied, we may break interrupts for some
> existing device-tree binaries that have bad configuration (such as omap4
> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
> is a back compatibility issue.
>
> If you are wondering why these interrupts break after "irqdomain: Don't
> set type when mapping an IRQ", it is because today
> irq_create_fwspec_mapping() does not check the return code from setting
> the type, but if we defer setting the type until __setup_irq() which
> does check the return code, then all of a sudden interrupts that were
> working (even with bad configurations) start to fail.
>
> The reason why I opted not to return an error code from
> gic_configure_irq() is it really can't fail. The failure being reported
> does not prevent the interrupt from working, but tells you your
> configuration does not match the hardware setting which you cannot
> overwrite.
>
> So to maintain back compatibility and avoid any silent errors, I opted
> to make it a WARN and not return an error.
>
> If people are ok with potentially breaking interrupts for device-tree
> binaries with bad settings, then I am ok to return an error here.

I think we need to phase things. Let's start with warning people for a
few kernel releases. Actively maintained platforms will quickly address
the issue (fixing their DT). As I see it, this issue seems rather
widespread (even kvmtool outputs a DT with the wrong triggering
information).

Once we've fixed the bulk of the platforms and virtual environments, we
can start thinking about making it fail harder.

Thanks,

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 11:03:46

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 05/15] irqchip: Mask the non-type/sense bits when translating an IRQ

On Thu, 17 Mar 2016 14:19:09 +0000
Jon Hunter <[email protected]> wrote:

> The firmware parameter that contains the IRQ sense bits may also contain
> other data. When return the IRQ type, bits outside of these sense bits
> should be masked. If these bits are not masked and
> irq_create_fwspec_mapping() is called to map an IRQ, then the comparison
> of the type returned from irq_domain_translate() will never match
> that returned by irq_get_trigger_type() (because this function masks the
> none sense bits) and so we will always call irq_set_irq_type() to program
> the type even if it was not really necessary.
>
> Currently, the downside to this is unnecessarily re-programmming the type
> but nevertheless this should be avoided.
>
> The Tegra LIC, TI Crossbar and GIC-V3 irqchips all have client instances
> (from reviewing the device-tree sources) where bits outside the IRQ sense
> bits are set, but do not mask these bits. Therefore, ensure these bits
> are masked for these irqchips.

For GICv3, this shouldn't be the case. The DT clearly says that the 3rd
field should only contain the trigger description. It looks like people
have been copying their GICv2 DT and simply slapped the v3 description
in the middle, without changing their interrupt specifiers. Duh.

I guess this patch doesn't hurt though.

>
> Signed-off-by: Jon Hunter <[email protected]>

Acked-by: Marc Zyngier <[email protected]>

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 11:09:25

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 09/15] irqchip/gic: Don't initialise chip if mapping IO space fails

On Thu, 17 Mar 2016 14:19:13 +0000
Jon Hunter <[email protected]> wrote:

> If we fail to map the address space for the GIC distributor or CPU
> interface, then don't attempt to initialise the chip, just WARN and
> return.
>
> Signed-off-by: Jon Hunter <[email protected]>

Acked-by: Marc Zyngier <[email protected]>

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 11:38:40

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 10/15] irqchip/gic: Remove static irq_chip definition for eoimode1

On Thu, 17 Mar 2016 14:19:14 +0000
Jon Hunter <[email protected]> wrote:

> There are only 3 differences (not including the name) in the definitions
> of the gic_chip and gic_eoimode1_chip structures. Instead of statically
> defining the gic_eoimode1_chip structure, remove it and populate the
> eoimode1 functions dynamically for the appropriate GIC irqchips.
>
> Signed-off-by: Jon Hunter <[email protected]>

Linus' remark notwithstanding,

Acked-by: Marc Zyngier <[email protected]>

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 11:43:47

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 11/15] irqchip/gic: Return an error if GIC initialisation fails

On Thu, 17 Mar 2016 14:19:15 +0000
Jon Hunter <[email protected]> wrote:

> If the GIC initialisation fails, then currently we do not return an error
> or clean-up afterwards. Although for root controllers, this failure may be
> fatal anyway, for secondary controllers, it may not be fatal and so return
> an error on failure and clean-up.
>
> For non-banked GIC controllers, make sure that we free any memory
> allocated if we fail to initialise the IRQ domain. Please note that
> free_percpu() only frees memory if the pointer passed to it is not NULL
> and so it is unnecessary to check if both pointers are valid or not.
>
> Signed-off-by: Jon Hunter <[email protected]>
> ---
> drivers/irqchip/irq-gic.c | 57 ++++++++++++++++++++++++++++++++++-------------
> 1 file changed, 41 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index b0a781f8c450..42a1412b5186 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -999,13 +999,13 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
> .unmap = gic_irq_domain_unmap,
> };
>
> -static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> +static int __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> void __iomem *dist_base, void __iomem *cpu_base,
> u32 percpu_offset, struct fwnode_handle *handle)
> {
> irq_hw_number_t hwirq_base;
> struct gic_chip_data *gic;
> - int gic_irqs, irq_base, i;
> + int gic_irqs, irq_base, i, ret;
>
> BUG_ON(gic_nr >= CONFIG_ARM_GIC_MAX_NR);
>
> @@ -1030,17 +1030,16 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> gic->chip.irq_set_affinity = gic_set_affinity;
> #endif
>
> -#ifdef CONFIG_GIC_NON_BANKED
> - if (percpu_offset) { /* Frankein-GIC without banked registers... */
> + if (IS_ENABLED(CONFIG_GIC_NON_BANKED) && percpu_offset) {
> + /* Frankein-GIC without banked registers... */
> unsigned int cpu;
>
> gic->dist_base.percpu_base = alloc_percpu(void __iomem *);
> gic->cpu_base.percpu_base = alloc_percpu(void __iomem *);
> if (WARN_ON(!gic->dist_base.percpu_base ||
> !gic->cpu_base.percpu_base)) {
> - free_percpu(gic->dist_base.percpu_base);
> - free_percpu(gic->cpu_base.percpu_base);
> - return;
> + ret = -ENOMEM;
> + goto error;
> }
>
> for_each_possible_cpu(cpu) {
> @@ -1052,9 +1051,8 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> }
>
> gic_set_base_accessor(gic, gic_get_percpu_base);
> - } else
> -#endif
> - { /* Normal, sane GIC... */
> + } else {
> + /* Normal, sane GIC... */
> WARN(percpu_offset,
> "GIC_NON_BANKED not enabled, ignoring %08x offset!",
> percpu_offset);
> @@ -1104,8 +1102,10 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> hwirq_base, &gic_irq_domain_ops, gic);
> }
>
> - if (WARN_ON(!gic->domain))
> - return;
> + if (WARN_ON(!gic->domain)) {
> + ret = -ENODEV;
> + goto error;
> + }
>
> if (gic_nr == 0) {
> /*
> @@ -1127,6 +1127,18 @@ static void __init __gic_init_bases(unsigned int gic_nr, int irq_start,
> gic_dist_init(gic);
> gic_cpu_init(gic);
> gic_pm_init(gic);
> +
> + return 0;
> +
> +error:
> + if (IS_ENABLED(CONFIG_GIC_NON_BANKED) && percpu_offset) {
> + free_percpu(gic->dist_base.percpu_base);
> + free_percpu(gic->cpu_base.percpu_base);
> + }
> +
> + kfree(gic->chip.name);

Ah, this is where Linus' remark about "GICv2" clashes. Either you keep
the allocation in the previous patch, or you guard this with a test on
the EOImode. I'll leave it up to you.

> +
> + return ret;
> }
>
> void __init gic_init(unsigned int gic_nr, int irq_start,
> @@ -1187,7 +1199,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> void __iomem *cpu_base;
> void __iomem *dist_base;
> u32 percpu_offset;
> - int irq;
> + int irq, ret;
>
> if (WARN_ON(!node))
> return -ENODEV;
> @@ -1212,8 +1224,14 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
> percpu_offset = 0;
>
> - __gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset,
> + ret = __gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset,
> &node->fwnode);
> + if (ret) {
> + iounmap(dist_base);
> + iounmap(cpu_base);
> + return ret;
> + }
> +
> if (!gic_cnt)
> gic_init_physaddr(node);
>
> @@ -1302,7 +1320,7 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,
> struct acpi_madt_generic_distributor *dist;
> void __iomem *cpu_base, *dist_base;
> struct fwnode_handle *domain_handle;
> - int count;
> + int count, ret;
>
> /* Collect CPU base addresses */
> count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
> @@ -1345,7 +1363,14 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,
> return -ENOMEM;
> }
>
> - __gic_init_bases(0, -1, dist_base, cpu_base, 0, domain_handle);
> + ret = __gic_init_bases(0, -1, dist_base, cpu_base, 0, domain_handle);
> + if (ret) {
> + pr_err("Failed to initialise GIC\n");
> + irq_domain_free_fwnode(domain_handle);
> + iounmap(cpu_base);
> + iounmap(dist_base);
> + return ret;
> + }
>
> acpi_set_irq_model(ACPI_IRQ_MODEL_GIC, domain_handle);
>

Otherwise looks good.

M.
--
Jazz is not dead. It just smells funny.

2016-04-09 11:52:22

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 12/15] irqchip/gic: Pass GIC pointer to save/restore functions

On Thu, 17 Mar 2016 14:19:16 +0000
Jon Hunter <[email protected]> wrote:

> Instead of passing the GIC index to the save/restore functions pass a
> pointer to the GIC chip data. This will allow these save/restore
> functions to be re-used by a platform driver where the GIC chip data
> structure is allocated dynamically and so there is no applicable index
> for identifying the GIC.
>
> Signed-off-by: Jon Hunter <[email protected]>

Eventually, I'd like to move to a situation where we don't statically
allocate the chip data anymore, and kill CONFIG_ARM_GIC_MAX_NR
altogether. This is obviously a step in the right direction.

Acked-by: Marc Zyngier <[email protected]>

M.
--
Jazz is not dead. It just smells funny.

2016-04-11 15:31:36

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

Hi Mark,

On 09/04/16 11:58, Marc Zyngier wrote:
> On Thu, 17 Mar 2016 15:04:01 +0000
> Jon Hunter <[email protected]> wrote:
>
>>
>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>
>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>> whether this is allowed. There is no way to know if setting the type is
>>>> supported for a given GIC and so the value written is read back to
>>>> verify it matches the desired configuration. If it does not match then
>>>> an error is return.
>>>>
>>>> There are cases where the interrupt configuration read from firmware
>>>> (such as a device-tree blob), has been incorrect and hence
>>>> gic_configure_irq() has returned an error. This error has gone
>>>> undetected because the error code returned was ignored but the interrupt
>>>> still worked fine because the configuration for the interrupt could not
>>>> be overwritten.
>>>>
>>>> Given that this has done undetected and we should only fail to set the
>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>> places in the kernel where we should be checking the return status and
>>>> maintain back compatibility with firmware images that may have incorrect
>>>> interrupt configurations.
>>>
>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>> warn, but why do you want to return success?
>>
>> Yes that would be the correct thing to do I agree. However, the problem
>> is that if we do this, then after the patch "irqdomain: Don't set type
>> when mapping an IRQ" is applied, we may break interrupts for some
>> existing device-tree binaries that have bad configuration (such as omap4
>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>> is a back compatibility issue.
>>
>> If you are wondering why these interrupts break after "irqdomain: Don't
>> set type when mapping an IRQ", it is because today
>> irq_create_fwspec_mapping() does not check the return code from setting
>> the type, but if we defer setting the type until __setup_irq() which
>> does check the return code, then all of a sudden interrupts that were
>> working (even with bad configurations) start to fail.
>>
>> The reason why I opted not to return an error code from
>> gic_configure_irq() is it really can't fail. The failure being reported
>> does not prevent the interrupt from working, but tells you your
>> configuration does not match the hardware setting which you cannot
>> overwrite.
>>
>> So to maintain back compatibility and avoid any silent errors, I opted
>> to make it a WARN and not return an error.
>>
>> If people are ok with potentially breaking interrupts for device-tree
>> binaries with bad settings, then I am ok to return an error here.
>
> I think we need to phase things. Let's start with warning people for a
> few kernel releases. Actively maintained platforms will quickly address
> the issue (fixing their DT). As I see it, this issue seems rather
> widespread (even kvmtool outputs a DT with the wrong triggering
> information).
>
> Once we've fixed the bulk of the platforms and virtual environments, we
> can start thinking about making it fail harder.

Ok, so are you OK with this patch as-is? If so, can I add your ACK?

Cheers
Jon

2016-04-11 15:39:28

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On 11/04/16 16:31, Jon Hunter wrote:
> Hi Mark,
>
> On 09/04/16 11:58, Marc Zyngier wrote:
>> On Thu, 17 Mar 2016 15:04:01 +0000
>> Jon Hunter <[email protected]> wrote:
>>
>>>
>>> On 17/03/16 14:51, Thomas Gleixner wrote:
>>>> On Thu, 17 Mar 2016, Jon Hunter wrote:
>>>>
>>>>> Setting the interrupt type for private peripheral interrupts (PPIs) may
>>>>> not be supported by a given GIC because it is IMPLEMENTATION DEFINED
>>>>> whether this is allowed. There is no way to know if setting the type is
>>>>> supported for a given GIC and so the value written is read back to
>>>>> verify it matches the desired configuration. If it does not match then
>>>>> an error is return.
>>>>>
>>>>> There are cases where the interrupt configuration read from firmware
>>>>> (such as a device-tree blob), has been incorrect and hence
>>>>> gic_configure_irq() has returned an error. This error has gone
>>>>> undetected because the error code returned was ignored but the interrupt
>>>>> still worked fine because the configuration for the interrupt could not
>>>>> be overwritten.
>>>>>
>>>>> Given that this has done undetected and we should only fail to set the
>>>>> type for PPIs whose configuration cannot be changed anyway, don't return
>>>>> an error and simply WARN if this fails. This will allows us to fix up any
>>>>> places in the kernel where we should be checking the return status and
>>>>> maintain back compatibility with firmware images that may have incorrect
>>>>> interrupt configurations.
>>>>
>>>> Though silently returning 0 is really the wrong thing to do. You can add the
>>>> warn, but why do you want to return success?
>>>
>>> Yes that would be the correct thing to do I agree. However, the problem
>>> is that if we do this, then after the patch "irqdomain: Don't set type
>>> when mapping an IRQ" is applied, we may break interrupts for some
>>> existing device-tree binaries that have bad configuration (such as omap4
>>> and tegra20/30 ... see patches 1 and 2) that have gone unnoticed. So it
>>> is a back compatibility issue.
>>>
>>> If you are wondering why these interrupts break after "irqdomain: Don't
>>> set type when mapping an IRQ", it is because today
>>> irq_create_fwspec_mapping() does not check the return code from setting
>>> the type, but if we defer setting the type until __setup_irq() which
>>> does check the return code, then all of a sudden interrupts that were
>>> working (even with bad configurations) start to fail.
>>>
>>> The reason why I opted not to return an error code from
>>> gic_configure_irq() is it really can't fail. The failure being reported
>>> does not prevent the interrupt from working, but tells you your
>>> configuration does not match the hardware setting which you cannot
>>> overwrite.
>>>
>>> So to maintain back compatibility and avoid any silent errors, I opted
>>> to make it a WARN and not return an error.
>>>
>>> If people are ok with potentially breaking interrupts for device-tree
>>> binaries with bad settings, then I am ok to return an error here.
>>
>> I think we need to phase things. Let's start with warning people for a
>> few kernel releases. Actively maintained platforms will quickly address
>> the issue (fixing their DT). As I see it, this issue seems rather
>> widespread (even kvmtool outputs a DT with the wrong triggering
>> information).
>>
>> Once we've fixed the bulk of the platforms and virtual environments, we
>> can start thinking about making it fail harder.
>
> Ok, so are you OK with this patch as-is? If so, can I add your ACK?

It depends where you plan to handle the error. Ideally, I'd keep on
returning the error (because that's the right thing to do), and move the
WARN_ON() into the core code. We'd keep on ignoring the error as we're
doing today, but we'd scream about it.

After a couple of releases, we'd turn the WARN_ON into a hard fail.

Thoughts?

M.
--
Jazz is not dead. It just smells funny...

2016-04-12 08:50:49

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails


On 11/04/16 16:39, Marc Zyngier wrote:
> On 11/04/16 16:31, Jon Hunter wrote:
>> On 09/04/16 11:58, Marc Zyngier wrote:

[snip]

>>> I think we need to phase things. Let's start with warning people for a
>>> few kernel releases. Actively maintained platforms will quickly address
>>> the issue (fixing their DT). As I see it, this issue seems rather
>>> widespread (even kvmtool outputs a DT with the wrong triggering
>>> information).
>>>
>>> Once we've fixed the bulk of the platforms and virtual environments, we
>>> can start thinking about making it fail harder.
>>
>> Ok, so are you OK with this patch as-is? If so, can I add your ACK?
>
> It depends where you plan to handle the error. Ideally, I'd keep on
> returning the error (because that's the right thing to do), and move the
> WARN_ON() into the core code. We'd keep on ignoring the error as we're
> doing today, but we'd scream about it.
>
> After a couple of releases, we'd turn the WARN_ON into a hard fail.
>
> Thoughts?

I agree that would be best/ideal, but looking at it, I don't believe it
is possible and this is why I have not done that so far.

If we were to add the WARN to the core code, then we would need to add a
warning everywhere __irq_set_trigger() is called. One of the places it
is called today is from __setup_irq() and today this does the right
thing and handle any error returned. The problem is that in
irq_create_fwspec_mapping() we have never checked the return code from
irq_set_irq_type() (which calls __irq_set_trigger()) or attempted to
handle any errors. So the problem is that depending on the path through
which the type is programmed, errors may or may not be detected. This is
the actual headache :-(

Given that this problem so far only pertains to GIC PPI interrupts and
that it is a not a catastrophic error (interrupts still work fine), I
was thinking we add the warning to the GIC driver.

May be a less severe change would be to only return an error if
configuring an SPI fails and if it is a PPI then simply WARN and
carry-on as we assume we cannot change it.

I hope this summarises the issue a bit further.

Cheers
Jon




2016-04-12 08:54:49

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 14/15] dt-bindings: arm-gic: Drop 'clock-names' from binding document

Rob, Mark, Marc,

On 18/03/16 08:37, Jon Hunter wrote:
>
> On 17/03/16 20:14, Rob Herring wrote:
>> On Thu, Mar 17, 2016 at 9:19 AM, Jon Hunter <[email protected]> wrote:
>>> Commit afbbd2338176 ("irqchip/gic: Document optional Clock and Power
>>> Domain properties") documented optional clock and power-dmoain properties
>>> for the ARM GIC. Currently, there are no users of these and for the
>>> Tegra210 Audio GIC (based upon the GIC-400) there are two clocks, a
>>> functional clock and interface clock, that need to be enabled.
>>>
>>> To allow flexibility, drop the 'clock-names' from the GIC binding and
>>> just provide a list of clocks which the driver can parse. It is assumed
>>> that any clocks that are listed, need to be enabled in order to access
>>> the GIC.
>>>
>>> Signed-off-by: Jon Hunter <[email protected]>
>>>
>>> ---
>>>
>>> Please note that I am not sure if this will be popular, but I am trying
>>> to come up with a generic way to handle multiple clocks that may be
>>> required for accessing a GIC.
>>
>> It's not. :)
>>
>> We need to specify the number and order of clocks by compatible string
>> at a minimum. Sadly, ARM's GICs are well documented and include clock
>> names, so you can't just make up genericish names either which is
>> probably often the case.
>
> Do you have any suggestions then?
>
> I have had a look at the ARM TRMs and although I see that they do show
> the functional clock, there is no mention of whether there are any other
> clocks need in order to interface to the GIC (ie. bus clock). I know
> that for other SoCs such as OMAP it is common to have both a functional
> clock and interface clock. So I believe this is fairly common.

Any thoughts on how I should handle this for the Tegra AGIC?

Cheers
Jon

2016-04-12 10:15:01

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 04/15] irqchip/gic: WARN if setting the interrupt type fails

On 12/04/16 09:50, Jon Hunter wrote:
>
> On 11/04/16 16:39, Marc Zyngier wrote:
>> On 11/04/16 16:31, Jon Hunter wrote:
>>> On 09/04/16 11:58, Marc Zyngier wrote:
>
> [snip]
>
>>>> I think we need to phase things. Let's start with warning people for a
>>>> few kernel releases. Actively maintained platforms will quickly address
>>>> the issue (fixing their DT). As I see it, this issue seems rather
>>>> widespread (even kvmtool outputs a DT with the wrong triggering
>>>> information).
>>>>
>>>> Once we've fixed the bulk of the platforms and virtual environments, we
>>>> can start thinking about making it fail harder.
>>>
>>> Ok, so are you OK with this patch as-is? If so, can I add your ACK?
>>
>> It depends where you plan to handle the error. Ideally, I'd keep on
>> returning the error (because that's the right thing to do), and move the
>> WARN_ON() into the core code. We'd keep on ignoring the error as we're
>> doing today, but we'd scream about it.
>>
>> After a couple of releases, we'd turn the WARN_ON into a hard fail.
>>
>> Thoughts?
>
> I agree that would be best/ideal, but looking at it, I don't believe it
> is possible and this is why I have not done that so far.
>
> If we were to add the WARN to the core code, then we would need to add a
> warning everywhere __irq_set_trigger() is called. One of the places it
> is called today is from __setup_irq() and today this does the right
> thing and handle any error returned. The problem is that in
> irq_create_fwspec_mapping() we have never checked the return code from
> irq_set_irq_type() (which calls __irq_set_trigger()) or attempted to
> handle any errors. So the problem is that depending on the path through
> which the type is programmed, errors may or may not be detected. This is
> the actual headache :-(
>
> Given that this problem so far only pertains to GIC PPI interrupts and
> that it is a not a catastrophic error (interrupts still work fine), I
> was thinking we add the warning to the GIC driver.
>
> May be a less severe change would be to only return an error if
> configuring an SPI fails and if it is a PPI then simply WARN and
> carry-on as we assume we cannot change it.

I'd take that. Limiting it to PPIs would be a minimal change, and the
warning would hopefully make people realize their DT is wrong. Failing
to program an SPI is really not expected, and should definitely explode.

Thanks,

M.
--
Jazz is not dead. It just smells funny...

2016-04-19 14:14:17

by Jon Hunter

[permalink] [raw]
Subject: Re: [PATCH 05/15] irqchip: Mask the non-type/sense bits when translating an IRQ


On 09/04/16 12:03, Marc Zyngier wrote:
> On Thu, 17 Mar 2016 14:19:09 +0000
> Jon Hunter <[email protected]> wrote:
>
>> The firmware parameter that contains the IRQ sense bits may also contain
>> other data. When return the IRQ type, bits outside of these sense bits
>> should be masked. If these bits are not masked and
>> irq_create_fwspec_mapping() is called to map an IRQ, then the comparison
>> of the type returned from irq_domain_translate() will never match
>> that returned by irq_get_trigger_type() (because this function masks the
>> none sense bits) and so we will always call irq_set_irq_type() to program
>> the type even if it was not really necessary.
>>
>> Currently, the downside to this is unnecessarily re-programmming the type
>> but nevertheless this should be avoided.
>>
>> The Tegra LIC, TI Crossbar and GIC-V3 irqchips all have client instances
>> (from reviewing the device-tree sources) where bits outside the IRQ sense
>> bits are set, but do not mask these bits. Therefore, ensure these bits
>> are masked for these irqchips.
>
> For GICv3, this shouldn't be the case. The DT clearly says that the 3rd
> field should only contain the trigger description. It looks like people
> have been copying their GICv2 DT and simply slapped the v3 description
> in the middle, without changing their interrupt specifiers. Duh.

Hmmm ... I was just double checking on this for the gic-v3 by wading
through the DTS files, and may be there is no issue here. However,
looking at the current code it is a bit inconsistent between firmware
types ...

static int gic_irq_domain_translate(struct irq_domain *d,
struct irq_fwspec *fwspec,
unsigned long *hwirq,
unsigned int *type)
{
if (is_of_node(fwspec->fwnode)) {
if (fwspec->param_count < 3)
return -EINVAL;

switch (fwspec->param[0]) {
case 0: /* SPI */
*hwirq = fwspec->param[1] + 32;
break;
case 1: /* PPI */
*hwirq = fwspec->param[1] + 16;
break;
case GIC_IRQ_TYPE_LPI: /* LPI */
*hwirq = fwspec->param[1];
break;
default:
return -EINVAL;
}

*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
return 0;
}

if (is_fwnode_irqchip(fwspec->fwnode)) {
if(fwspec->param_count != 2)
return -EINVAL;

*hwirq = fwspec->param[0];
*type = fwspec->param[1];
return 0;
}

return -EINVAL;

> I guess this patch doesn't hurt though.

Yes, it doesn't but I think I will leave this alone for now, given that
I can't find a case where this would be a problem.

Cheers
Jon

2016-04-19 14:23:42

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 05/15] irqchip: Mask the non-type/sense bits when translating an IRQ

Hi Jon,

On 19/04/16 15:14, Jon Hunter wrote:
>
> On 09/04/16 12:03, Marc Zyngier wrote:
>> On Thu, 17 Mar 2016 14:19:09 +0000
>> Jon Hunter <[email protected]> wrote:
>>
>>> The firmware parameter that contains the IRQ sense bits may also contain
>>> other data. When return the IRQ type, bits outside of these sense bits
>>> should be masked. If these bits are not masked and
>>> irq_create_fwspec_mapping() is called to map an IRQ, then the comparison
>>> of the type returned from irq_domain_translate() will never match
>>> that returned by irq_get_trigger_type() (because this function masks the
>>> none sense bits) and so we will always call irq_set_irq_type() to program
>>> the type even if it was not really necessary.
>>>
>>> Currently, the downside to this is unnecessarily re-programmming the type
>>> but nevertheless this should be avoided.
>>>
>>> The Tegra LIC, TI Crossbar and GIC-V3 irqchips all have client instances
>>> (from reviewing the device-tree sources) where bits outside the IRQ sense
>>> bits are set, but do not mask these bits. Therefore, ensure these bits
>>> are masked for these irqchips.
>>
>> For GICv3, this shouldn't be the case. The DT clearly says that the 3rd
>> field should only contain the trigger description. It looks like people
>> have been copying their GICv2 DT and simply slapped the v3 description
>> in the middle, without changing their interrupt specifiers. Duh.
>
> Hmmm ... I was just double checking on this for the gic-v3 by wading
> through the DTS files, and may be there is no issue here. However,
> looking at the current code it is a bit inconsistent between firmware
> types ...
>
> static int gic_irq_domain_translate(struct irq_domain *d,
> struct irq_fwspec *fwspec,
> unsigned long *hwirq,
> unsigned int *type)
> {
> if (is_of_node(fwspec->fwnode)) {
> if (fwspec->param_count < 3)
> return -EINVAL;
>
> switch (fwspec->param[0]) {
> case 0: /* SPI */
> *hwirq = fwspec->param[1] + 32;
> break;
> case 1: /* PPI */
> *hwirq = fwspec->param[1] + 16;
> break;
> case GIC_IRQ_TYPE_LPI: /* LPI */
> *hwirq = fwspec->param[1];
> break;
> default:
> return -EINVAL;
> }
>
> *type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
> return 0;
> }
>
> if (is_fwnode_irqchip(fwspec->fwnode)) {
> if(fwspec->param_count != 2)
> return -EINVAL;
>
> *hwirq = fwspec->param[0];
> *type = fwspec->param[1];

That's because param[1] doesn't really come from the firmware. It is
actually provided directly by acpi_dev_get_irq_type, so more or less
guaranteed to be a valid value.

Thanks,

M.
--
Jazz is not dead. It just smells funny...