This commit adds suspend/resume support to the irqchip driver used on
Armada XP platforms (amongst others). It does so by adding a set of
suspend/resume syscore_ops, that will respectively save and restore
the necessary registers to ensure interrupts continue to work after
resume.
It is worth mentioning that the affinity is lost during a
suspend/resume cycle, because when a secondary CPU is brought
off-line, all interrupts that are assigned to this CPU in terms of
affinity gets re-assigned to a still running CPU. Therefore, right
before entering suspend, all interrupts are assigned to the boot CPU.
Signed-off-by: Thomas Petazzoni <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Jason Cooper <[email protected]>
Cc: [email protected]
---
drivers/irqchip/irq-armada-370-xp.c | 52 +++++++++++++++++++++++++++++++++++++
1 file changed, 52 insertions(+)
diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index 9e630f2..3c2b89d 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -26,6 +26,7 @@
#include <linux/of_pci.h>
#include <linux/irqdomain.h>
#include <linux/slab.h>
+#include <linux/syscore_ops.h>
#include <linux/msi.h>
#include <asm/mach/arch.h>
#include <asm/exception.h>
@@ -66,6 +67,7 @@
static void __iomem *per_cpu_int_base;
static void __iomem *main_int_base;
static struct irq_domain *armada_370_xp_mpic_domain;
+static u32 doorbell_mask_reg;
#ifdef CONFIG_PCI_MSI
static struct irq_domain *armada_370_xp_msi_domain;
static DECLARE_BITMAP(msi_used, PCI_MSI_DOORBELL_NR);
@@ -474,6 +476,54 @@ armada_370_xp_handle_irq(struct pt_regs *regs)
} while (1);
}
+static int armada_370_xp_mpic_suspend(void)
+{
+ doorbell_mask_reg = readl(per_cpu_int_base +
+ ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ return 0;
+}
+
+static void armada_370_xp_mpic_resume(void)
+{
+ int nirqs;
+ irq_hw_number_t irq;
+
+ /* Re-enable interrupts */
+ nirqs = (readl(main_int_base + ARMADA_370_XP_INT_CONTROL) >> 2) & 0x3ff;
+ for (irq = 0; irq < nirqs; irq++) {
+ struct irq_data *data;
+ int virq;
+
+ virq = irq_linear_revmap(armada_370_xp_mpic_domain, irq);
+ if (virq == 0)
+ continue;
+
+ if (irq != ARMADA_370_XP_TIMER0_PER_CPU_IRQ)
+ writel(irq, per_cpu_int_base +
+ ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ else
+ writel(irq, main_int_base +
+ ARMADA_370_XP_INT_SET_ENABLE_OFFS);
+
+ data = irq_get_irq_data(virq);
+ if (!irqd_irq_disabled(data))
+ armada_370_xp_irq_unmask(data);
+ }
+
+ /* Reconfigure doorbells for IPIs and MSIs */
+ writel(doorbell_mask_reg,
+ per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ if (doorbell_mask_reg & IPI_DOORBELL_MASK)
+ writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ if (doorbell_mask_reg & PCI_MSI_DOORBELL_MASK)
+ writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+}
+
+struct syscore_ops armada_370_xp_mpic_syscore_ops = {
+ .suspend = armada_370_xp_mpic_suspend,
+ .resume = armada_370_xp_mpic_resume,
+};
+
static int __init armada_370_xp_mpic_of_init(struct device_node *node,
struct device_node *parent)
{
@@ -530,6 +580,8 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
armada_370_xp_mpic_handle_cascade_irq);
}
+ register_syscore_ops(&armada_370_xp_mpic_syscore_ops);
+
return 0;
}
--
2.0.0
Hi Thomas,
On 24/10/2014 13:59, Thomas Petazzoni wrote:
> This commit adds suspend/resume support to the irqchip driver used on
> Armada XP platforms (amongst others). It does so by adding a set of
> suspend/resume syscore_ops, that will respectively save and restore
> the necessary registers to ensure interrupts continue to work after
> resume.
>
> It is worth mentioning that the affinity is lost during a
> suspend/resume cycle, because when a secondary CPU is brought
> off-line, all interrupts that are assigned to this CPU in terms of
> affinity gets re-assigned to a still running CPU. Therefore, right
> before entering suspend, all interrupts are assigned to the boot CPU.
So what about /proc/irq/*/smp_affinity ?
Do this files still represent accurate information?
Thanks,
Gregory
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Dear Gregory CLEMENT,
On Mon, 03 Nov 2014 18:38:10 +0100, Gregory CLEMENT wrote:
> > It is worth mentioning that the affinity is lost during a
> > suspend/resume cycle, because when a secondary CPU is brought
> > off-line, all interrupts that are assigned to this CPU in terms of
> > affinity gets re-assigned to a still running CPU. Therefore, right
> > before entering suspend, all interrupts are assigned to the boot CPU.
>
> So what about /proc/irq/*/smp_affinity ?
>
> Do this files still represent accurate information?
See migrate_irqs() in arch/arm/kernel/irq.c. Basically, when you do a
suspend, the kernel hot unplugs the non-boot CPUs: in our case CPU1,
CPU2 and CPU3. The migrate_irqs() function above makes sure that the
affinity used for each interrupt continue to contain at least one
online CPU. Therefore, it modifies the affinity information across
suspend/resume.
For example, let's say you initially have an affinity of 0xf (i.e all
CPUs can receive the interrupt) and you modify it to 0x2 (i.e only CPU1
can take the interrupt).
Then, when migrate_irqs() is called when CPU1 is brought offline, it
will realize that the interrupt will no longer be affine to any CPU,
and it will re-assign the affinity to the current cpu_online_mask
(which will be 0xd, i.e all CPUs except CPU1). Therefore, when you get
out of suspend, the affinity for this interrupt will be 0xd.
Note that in this case, a message is displayed during the suspend
procedure:
IRQ19 no longer affine to CPU1
On the other hand, if you leave the affinity to its default of 0xf, it
remains to 0xf, because there's always at least one online CPU in the
affinity (the boot CPU). Same thing if you change the affinity to 0x3
for example, since 0x3 is an affinity that contains the boot CPU (of
course, assuming the boot CPU is CPU0).
That's not a logic that is controlled in the irqchip driver, it's
entirely handled by the kernel core. I hope this explanation clarifies
the situation.
Thanks,
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com