Changes since last time:
* correct return value of xen_irq_from_pirq
* WARN if a pirq cannot be allocated for a legacy IRQ
* Updated checking comment of "xen: events: do not workaround
too-small nr_irqs"
The following series makes a few cleanups to the Xen IRQ infrastructure.
The most important thing is that it removes the need to know about
nr_irqs and in particular the reliance on nr_irqs being static.
Apart from being generally a good thing this is needed because in 2.6.39
nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
"genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
at 2.6.39.
Dynamically growing nr_irqs also allows us to remove the workaround
which eats into GSI space if a dynamic IRQ cannot be allocated.
There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
have gone in before, but the penultimate patch really needed to be
simultaneous) so I haven't bothered to try and pull anything from tip
into this branch -- it should all be resolved during the merge window
and bisection won't be too broken since the "eat into GSI space"
workaround only appears to be needed on a small number of older
platforms (qemu being the main exception).
I have tested:
* Domain 0 on real h/w and under qemu
* PV guest, including migration and passthrough of both VF and PF.
* PVHVM guest, including migration and passthrough of both VF and
PF.
The git pull is a branch on top of konrad/stable/irq.cleanup. However
there is an interaction with konrad/devel/xen-pciback-0.4.driver (the
addition of the domid parameter) so for convenience I have also produced
an irq-pciback branch at the same location which has
konrad/devel/xen-pciback-0.4.driver merged into this branch.
Note that this series obsoletes an older patcho f mine "xen: events:
mark cpu_evtchn_mask_p as __refdata" by virtue of removing the code in
question...
The following changes since commit c5ae07bb307b658c8458f29ca77d237aec0f9327:
Ian Campbell (1):
xen: events: remove dom0 specific xen_create_msi_irq
are available in the git repository at:
git://xenbits.xen.org/people/ianc/linux-2.6.git irq
Ian Campbell (14):
xen: events: separate two unrelated halves of if condition
xen: events: fix xen_map_pirq_gsi error return
xen: events: simplify comment
xen: events: remove unused public functions
xen: events: rename restore_cpu_pirqs -> restore_pirqs
xen: events: refactor GSI pirq bindings functions
xen: events: use per-cpu variable for cpu_evtchn_mask
xen: events: turn irq_info constructors into initialiser functions
xen: events: push setup of irq<->{evtchn,ipi,virq,pirq} maps into irq_info init functions
xen: events: maintain a list of Xen interrupts
xen: events: dynamically allocate irq info structures
xen: events: remove use of nr_irqs as upper bound on number of pirqs
xen: events: do not workaround too-small nr_irqs
xen: events: propagate irq allocation failure instead of panicking
arch/x86/pci/xen.c | 41 ++++--
drivers/xen/events.c | 357 ++++++++++++++++++++++++++------------------------
include/xen/events.h | 24 ++--
3 files changed, 226 insertions(+), 196 deletions(-)
Clarifies which bit the comment applies to.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 12 ++++++++----
1 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 6befe62..684b095 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -1143,10 +1143,14 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu)
struct evtchn_bind_vcpu bind_vcpu;
int evtchn = evtchn_from_irq(irq);
- /* events delivered via platform PCI interrupts are always
- * routed to vcpu 0 */
- if (!VALID_EVTCHN(evtchn) ||
- (xen_hvm_domain() && !xen_have_vector_callback))
+ if (!VALID_EVTCHN(evtchn))
+ return -1;
+
+ /*
+ * Events delivered via platform PCI interrupts are always
+ * routed to vcpu 0 and hence cannot be rebound.
+ */
+ if (xen_hvm_domain() && !xen_have_vector_callback)
return -1;
/* Send future instances of this interrupt to other vcpu. */
--
1.5.6.5
It is never valid assume any particular relationship between a Xen
PIRQ number and and Linux IRQ number so there is no need to hedge when
saying so.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 10 +++-------
include/xen/events.h | 4 +---
2 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 2ce95a6..a9c154d 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -583,13 +583,9 @@ int xen_allocate_pirq(unsigned gsi, int shareable, char *name)
return xen_map_pirq_gsi(gsi, gsi, shareable, name);
}
-/* xen_map_pirq_gsi might allocate irqs from the top down, as a
- * consequence don't assume that the irq number returned has a low value
- * or can be used as a pirq number unless you know otherwise.
- *
- * One notable exception is when xen_map_pirq_gsi is called passing an
- * hardware gsi as argument, in that case the irq number returned
- * matches the gsi number passed as second argument.
+/*
+ * Do not make any assumptions regarding the relationship between the
+ * IRQ number returned here and the Xen pirq argument.
*
* Note: We don't assign an event channel until the irq actually started
* up. Return an existing irq if we've already got one for the gsi.
diff --git a/include/xen/events.h b/include/xen/events.h
index 962da2c..f6fed94 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -68,9 +68,7 @@ int xen_set_callback_via(uint64_t via);
void xen_evtchn_do_upcall(struct pt_regs *regs);
void xen_hvm_evtchn_do_upcall(void);
-/* Allocate an irq for a physical interrupt, given a gsi. "Legacy"
- * GSIs are identity mapped; others are dynamically allocated as
- * usual. */
+/* Allocate an irq for a physical interrupt, given a gsi. */
int xen_allocate_pirq(unsigned gsi, int shareable, char *name);
int xen_map_pirq_gsi(unsigned pirq, unsigned gsi, int shareable, char *name);
--
1.5.6.5
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 104 +++++++++++++++++++++++++++++++------------------
1 files changed, 66 insertions(+), 38 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 2dffa43..72725fa 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -118,46 +118,76 @@ static struct irq_chip xen_dynamic_chip;
static struct irq_chip xen_percpu_chip;
static struct irq_chip xen_pirq_chip;
-/* Constructor for packed IRQ information. */
-static struct irq_info mk_unbound_info(void)
+/* Get info for IRQ */
+static struct irq_info *info_for_irq(unsigned irq)
{
- return (struct irq_info) { .type = IRQT_UNBOUND };
+ return &irq_info[irq];
}
-static struct irq_info mk_evtchn_info(unsigned short evtchn)
+/* Constructors for packed IRQ information. */
+static void xen_irq_info_common_init(struct irq_info *info,
+ enum xen_irq_type type,
+ unsigned short evtchn,
+ unsigned short cpu)
{
- return (struct irq_info) { .type = IRQT_EVTCHN, .evtchn = evtchn,
- .cpu = 0 };
+
+ BUG_ON(info->type != IRQT_UNBOUND && info->type != type);
+
+ info->type = type;
+ info->evtchn = evtchn;
+ info->cpu = cpu;
}
-static struct irq_info mk_ipi_info(unsigned short evtchn, enum ipi_vector ipi)
+static void xen_irq_info_evtchn_init(unsigned irq,
+ unsigned short evtchn)
{
- return (struct irq_info) { .type = IRQT_IPI, .evtchn = evtchn,
- .cpu = 0, .u.ipi = ipi };
+ struct irq_info *info = info_for_irq(irq);
+
+ xen_irq_info_common_init(info, IRQT_EVTCHN, evtchn, 0);
}
-static struct irq_info mk_virq_info(unsigned short evtchn, unsigned short virq)
+static void xen_irq_info_ipi_init(unsigned irq,
+ unsigned short evtchn,
+ enum ipi_vector ipi)
{
- return (struct irq_info) { .type = IRQT_VIRQ, .evtchn = evtchn,
- .cpu = 0, .u.virq = virq };
+ struct irq_info *info = info_for_irq(irq);
+
+ xen_irq_info_common_init(info, IRQT_IPI, evtchn, 0);
+
+ info->u.ipi = ipi;
}
-static struct irq_info mk_pirq_info(unsigned short evtchn, unsigned short pirq,
- unsigned short gsi, unsigned short vector)
+static void xen_irq_info_virq_init(unsigned irq,
+ unsigned short evtchn,
+ unsigned short virq)
{
- return (struct irq_info) { .type = IRQT_PIRQ, .evtchn = evtchn,
- .cpu = 0,
- .u.pirq = { .pirq = pirq, .gsi = gsi, .vector = vector } };
+ struct irq_info *info = info_for_irq(irq);
+
+ xen_irq_info_common_init(info, IRQT_VIRQ, evtchn, 0);
+
+ info->u.virq = virq;
}
-/*
- * Accessors for packed IRQ information.
- */
-static struct irq_info *info_for_irq(unsigned irq)
+static void xen_irq_info_pirq_init(unsigned irq,
+ unsigned short evtchn,
+ unsigned short pirq,
+ unsigned short gsi,
+ unsigned short vector,
+ unsigned char flags)
{
- return &irq_info[irq];
+ struct irq_info *info = info_for_irq(irq);
+
+ xen_irq_info_common_init(info, IRQT_PIRQ, evtchn, 0);
+
+ info->u.pirq.pirq = pirq;
+ info->u.pirq.gsi = gsi;
+ info->u.pirq.vector = vector;
+ info->u.pirq.flags = flags;
}
+/*
+ * Accessors for packed IRQ information.
+ */
static unsigned int evtchn_from_irq(unsigned irq)
{
if (unlikely(WARN(irq < 0 || irq >= nr_irqs, "Invalid irq %d!\n", irq)))
@@ -414,6 +444,8 @@ static int xen_allocate_irq_gsi(unsigned gsi)
static void xen_free_irq(unsigned irq)
{
+ irq_info[irq].type = IRQT_UNBOUND;
+
/* Legacy IRQ descriptors are managed by the arch. */
if (irq < NR_IRQS_LEGACY)
return;
@@ -610,8 +642,8 @@ int xen_bind_pirq_gsi_to_irq(unsigned gsi,
goto out;
}
- irq_info[irq] = mk_pirq_info(0, pirq, gsi, irq_op.vector);
- irq_info[irq].u.pirq.flags |= shareable ? PIRQ_SHAREABLE : 0;
+ xen_irq_info_pirq_init(irq, 0, pirq, gsi, irq_op.vector,
+ shareable ? PIRQ_SHAREABLE : 0);
pirq_to_irq[pirq] = irq;
out:
@@ -649,7 +681,7 @@ int xen_bind_pirq_msi_to_irq(struct pci_dev *dev, struct msi_desc *msidesc,
set_irq_chip_and_handler_name(irq, &xen_pirq_chip,
handle_level_irq, name);
- irq_info[irq] = mk_pirq_info(0, pirq, 0, vector);
+ xen_irq_info_pirq_init(irq, 0, pirq, 0, vector, 0);
pirq_to_irq[pirq] = irq;
ret = set_irq_msi(irq, msidesc);
if (ret < 0)
@@ -688,8 +720,6 @@ int xen_destroy_irq(int irq)
}
pirq_to_irq[info->u.pirq.pirq] = -1;
- irq_info[irq] = mk_unbound_info();
-
xen_free_irq(irq);
out:
@@ -717,7 +747,7 @@ int bind_evtchn_to_irq(unsigned int evtchn)
handle_fasteoi_irq, "event");
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_evtchn_info(evtchn);
+ xen_irq_info_evtchn_init(irq, evtchn);
}
spin_unlock(&irq_mapping_update_lock);
@@ -750,7 +780,7 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
evtchn = bind_ipi.port;
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_ipi_info(evtchn, ipi);
+ xen_irq_info_ipi_init(irq, evtchn, ipi);
per_cpu(ipi_to_irq, cpu)[ipi] = irq;
bind_evtchn_to_cpu(evtchn, cpu);
@@ -785,7 +815,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
evtchn = bind_virq.port;
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_virq_info(evtchn, virq);
+ xen_irq_info_virq_init(irq, evtchn, virq);
per_cpu(virq_to_irq, cpu)[virq] = irq;
@@ -828,11 +858,9 @@ static void unbind_from_irq(unsigned int irq)
evtchn_to_irq[evtchn] = -1;
}
- if (irq_info[irq].type != IRQT_UNBOUND) {
- irq_info[irq] = mk_unbound_info();
+ BUG_ON(irq_info[irq].type == IRQT_UNBOUND);
- xen_free_irq(irq);
- }
+ xen_free_irq(irq);
spin_unlock(&irq_mapping_update_lock);
}
@@ -1093,7 +1121,7 @@ void rebind_evtchn_irq(int evtchn, int irq)
BUG_ON(info->type == IRQT_UNBOUND);
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_evtchn_info(evtchn);
+ xen_irq_info_evtchn_init(irq, evtchn);
spin_unlock(&irq_mapping_update_lock);
@@ -1229,7 +1257,7 @@ static void restore_pirqs(void)
if (rc) {
printk(KERN_WARNING "xen map irq failed gsi=%d irq=%d pirq=%d rc=%d\n",
gsi, irq, pirq, rc);
- irq_info[irq] = mk_unbound_info();
+ xen_free_irq(irq);
pirq_to_irq[pirq] = -1;
continue;
}
@@ -1261,7 +1289,7 @@ static void restore_cpu_virqs(unsigned int cpu)
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_virq_info(evtchn, virq);
+ xen_irq_info_virq_init(irq, evtchn, virq);
bind_evtchn_to_cpu(evtchn, cpu);
}
}
@@ -1286,7 +1314,7 @@ static void restore_cpu_ipis(unsigned int cpu)
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
- irq_info[irq] = mk_ipi_info(evtchn, ipi);
+ xen_irq_info_ipi_init(irq, evtchn, ipi);
bind_evtchn_to_cpu(evtchn, cpu);
}
}
--
1.5.6.5
Encapsulate setup of XXX_to_irq array in the relevant
xen_irq_info_*_init function.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 42 +++++++++++++++++++++---------------------
1 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 72725fa..cf372d4 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -126,6 +126,7 @@ static struct irq_info *info_for_irq(unsigned irq)
/* Constructors for packed IRQ information. */
static void xen_irq_info_common_init(struct irq_info *info,
+ unsigned irq,
enum xen_irq_type type,
unsigned short evtchn,
unsigned short cpu)
@@ -136,6 +137,8 @@ static void xen_irq_info_common_init(struct irq_info *info,
info->type = type;
info->evtchn = evtchn;
info->cpu = cpu;
+
+ evtchn_to_irq[evtchn] = irq;
}
static void xen_irq_info_evtchn_init(unsigned irq,
@@ -143,29 +146,35 @@ static void xen_irq_info_evtchn_init(unsigned irq,
{
struct irq_info *info = info_for_irq(irq);
- xen_irq_info_common_init(info, IRQT_EVTCHN, evtchn, 0);
+ xen_irq_info_common_init(info, irq, IRQT_EVTCHN, evtchn, 0);
}
-static void xen_irq_info_ipi_init(unsigned irq,
+static void xen_irq_info_ipi_init(unsigned cpu,
+ unsigned irq,
unsigned short evtchn,
enum ipi_vector ipi)
{
struct irq_info *info = info_for_irq(irq);
- xen_irq_info_common_init(info, IRQT_IPI, evtchn, 0);
+ xen_irq_info_common_init(info, irq, IRQT_IPI, evtchn, 0);
info->u.ipi = ipi;
+
+ per_cpu(ipi_to_irq, cpu)[ipi] = irq;
}
-static void xen_irq_info_virq_init(unsigned irq,
+static void xen_irq_info_virq_init(unsigned cpu,
+ unsigned irq,
unsigned short evtchn,
unsigned short virq)
{
struct irq_info *info = info_for_irq(irq);
- xen_irq_info_common_init(info, IRQT_VIRQ, evtchn, 0);
+ xen_irq_info_common_init(info, irq, IRQT_VIRQ, evtchn, 0);
info->u.virq = virq;
+
+ per_cpu(virq_to_irq, cpu)[virq] = irq;
}
static void xen_irq_info_pirq_init(unsigned irq,
@@ -177,12 +186,14 @@ static void xen_irq_info_pirq_init(unsigned irq,
{
struct irq_info *info = info_for_irq(irq);
- xen_irq_info_common_init(info, IRQT_PIRQ, evtchn, 0);
+ xen_irq_info_common_init(info, irq, IRQT_PIRQ, evtchn, 0);
info->u.pirq.pirq = pirq;
info->u.pirq.gsi = gsi;
info->u.pirq.vector = vector;
info->u.pirq.flags = flags;
+
+ pirq_to_irq[pirq] = irq;
}
/*
@@ -644,7 +655,6 @@ int xen_bind_pirq_gsi_to_irq(unsigned gsi,
xen_irq_info_pirq_init(irq, 0, pirq, gsi, irq_op.vector,
shareable ? PIRQ_SHAREABLE : 0);
- pirq_to_irq[pirq] = irq;
out:
spin_unlock(&irq_mapping_update_lock);
@@ -682,7 +692,6 @@ int xen_bind_pirq_msi_to_irq(struct pci_dev *dev, struct msi_desc *msidesc,
handle_level_irq, name);
xen_irq_info_pirq_init(irq, 0, pirq, 0, vector, 0);
- pirq_to_irq[pirq] = irq;
ret = set_irq_msi(irq, msidesc);
if (ret < 0)
goto error_irq;
@@ -746,7 +755,6 @@ int bind_evtchn_to_irq(unsigned int evtchn)
set_irq_chip_and_handler_name(irq, &xen_dynamic_chip,
handle_fasteoi_irq, "event");
- evtchn_to_irq[evtchn] = irq;
xen_irq_info_evtchn_init(irq, evtchn);
}
@@ -779,9 +787,7 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
BUG();
evtchn = bind_ipi.port;
- evtchn_to_irq[evtchn] = irq;
- xen_irq_info_ipi_init(irq, evtchn, ipi);
- per_cpu(ipi_to_irq, cpu)[ipi] = irq;
+ xen_irq_info_ipi_init(cpu, irq, evtchn, ipi);
bind_evtchn_to_cpu(evtchn, cpu);
}
@@ -814,10 +820,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
BUG();
evtchn = bind_virq.port;
- evtchn_to_irq[evtchn] = irq;
- xen_irq_info_virq_init(irq, evtchn, virq);
-
- per_cpu(virq_to_irq, cpu)[virq] = irq;
+ xen_irq_info_virq_init(cpu, irq, evtchn, virq);
bind_evtchn_to_cpu(evtchn, cpu);
}
@@ -1120,7 +1123,6 @@ void rebind_evtchn_irq(int evtchn, int irq)
so there should be a proper type */
BUG_ON(info->type == IRQT_UNBOUND);
- evtchn_to_irq[evtchn] = irq;
xen_irq_info_evtchn_init(irq, evtchn);
spin_unlock(&irq_mapping_update_lock);
@@ -1288,8 +1290,7 @@ static void restore_cpu_virqs(unsigned int cpu)
evtchn = bind_virq.port;
/* Record the new mapping. */
- evtchn_to_irq[evtchn] = irq;
- xen_irq_info_virq_init(irq, evtchn, virq);
+ xen_irq_info_virq_init(cpu, irq, evtchn, virq);
bind_evtchn_to_cpu(evtchn, cpu);
}
}
@@ -1313,8 +1314,7 @@ static void restore_cpu_ipis(unsigned int cpu)
evtchn = bind_ipi.port;
/* Record the new mapping. */
- evtchn_to_irq[evtchn] = irq;
- xen_irq_info_ipi_init(irq, evtchn, ipi);
+ xen_irq_info_ipi_init(cpu, irq, evtchn, ipi);
bind_evtchn_to_cpu(evtchn, cpu);
}
}
--
1.5.6.5
With the introduction of e7bcecb7b1d2 "genirq: Make nr_irqs runtime expandable"
nr_irqs can grow as necessary to accommodate our allocation requests.
Signed-off-by: Ian Campbell <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
---
drivers/xen/events.c | 7 -------
1 files changed, 0 insertions(+), 7 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 6782251..7c36689 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -423,15 +423,8 @@ static int xen_allocate_irq_dynamic(void)
first = get_nr_irqs_gsi();
#endif
-retry:
irq = irq_alloc_desc_from(first, -1);
- if (irq == -ENOMEM && first > NR_IRQS_LEGACY) {
- printk(KERN_ERR "Out of dynamic IRQ space and eating into GSI space. You should increase nr_irqs\n");
- first = max(NR_IRQS_LEGACY, first - NR_IRQS_LEGACY);
- goto retry;
- }
-
if (irq < 0)
panic("No available IRQ to bind to: increase nr_irqs!\n");
--
1.5.6.5
There isn't really much relationship between the two, other than
nr_irqs often being the larger of the two.
Allows us to remove a nr_irqs sized array, the only users of this
array are MSI setup and restore, neither of which are particularly
performance critical.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 59 +++++++++++++++++++++----------------------------
1 files changed, 25 insertions(+), 34 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 002283e..6782251 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -107,8 +107,6 @@ struct irq_info
#define PIRQ_NEEDS_EOI (1 << 0)
#define PIRQ_SHAREABLE (1 << 1)
-static int *pirq_to_irq;
-
static int *evtchn_to_irq;
static DEFINE_PER_CPU(unsigned long [NR_EVENT_CHANNELS/BITS_PER_LONG],
@@ -196,8 +194,6 @@ static void xen_irq_info_pirq_init(unsigned irq,
info->u.pirq.gsi = gsi;
info->u.pirq.vector = vector;
info->u.pirq.flags = flags;
-
- pirq_to_irq[pirq] = irq;
}
/*
@@ -247,16 +243,6 @@ static unsigned pirq_from_irq(unsigned irq)
return info->u.pirq.pirq;
}
-static unsigned gsi_from_irq(unsigned irq)
-{
- struct irq_info *info = info_for_irq(irq);
-
- BUG_ON(info == NULL);
- BUG_ON(info->type != IRQT_PIRQ);
-
- return info->u.pirq.gsi;
-}
-
static enum xen_irq_type type_from_irq(unsigned irq)
{
return info_for_irq(irq)->type;
@@ -653,12 +639,6 @@ int xen_bind_pirq_gsi_to_irq(unsigned gsi,
spin_lock(&irq_mapping_update_lock);
- if (pirq > nr_irqs) {
- printk(KERN_WARNING "xen_map_pirq_gsi: pirq %d > nr_irqs %d!\n",
- pirq, nr_irqs);
- goto out;
- }
-
irq = find_irq_by_gsi(gsi);
if (irq != -1) {
printk(KERN_INFO "xen_map_pirq_gsi: returning irq %d for gsi %u\n",
@@ -758,7 +738,6 @@ int xen_destroy_irq(int irq)
goto out;
}
}
- pirq_to_irq[info->u.pirq.pirq] = -1;
xen_free_irq(irq);
@@ -769,7 +748,24 @@ out:
int xen_irq_from_pirq(unsigned pirq)
{
- return pirq_to_irq[pirq];
+ int irq;
+
+ struct irq_info *info;
+
+ spin_lock(&irq_mapping_update_lock);
+
+ list_for_each_entry(info, &xen_irq_list_head, list) {
+ if (info == NULL || info->type != IRQT_PIRQ)
+ continue;
+ irq = info->irq;
+ if (info->u.pirq.pirq == pirq)
+ goto out;
+ }
+ irq = -1;
+out:
+ spin_lock(&irq_mapping_update_lock);
+
+ return irq;
}
int bind_evtchn_to_irq(unsigned int evtchn)
@@ -1269,15 +1265,18 @@ static void restore_pirqs(void)
{
int pirq, rc, irq, gsi;
struct physdev_map_pirq map_irq;
+ struct irq_info *info;
- for (pirq = 0; pirq < nr_irqs; pirq++) {
- irq = pirq_to_irq[pirq];
- if (irq == -1)
+ list_for_each_entry(info, &xen_irq_list_head, list) {
+ if (info->type != IRQT_PIRQ)
continue;
+ pirq = info->u.pirq.pirq;
+ gsi = info->u.pirq.gsi;
+ irq = info->irq;
+
/* save/restore of PT devices doesn't work, so at this point the
* only devices present are GSI based emulated devices */
- gsi = gsi_from_irq(irq);
if (!gsi)
continue;
@@ -1291,7 +1290,6 @@ static void restore_pirqs(void)
printk(KERN_WARNING "xen map irq failed gsi=%d irq=%d pirq=%d rc=%d\n",
gsi, irq, pirq, rc);
xen_free_irq(irq);
- pirq_to_irq[pirq] = -1;
continue;
}
@@ -1512,13 +1510,6 @@ void __init xen_init_IRQ(void)
{
int i;
- /* We are using nr_irqs as the maximum number of pirq available but
- * that number is actually chosen by Xen and we don't know exactly
- * what it is. Be careful choosing high pirq numbers. */
- pirq_to_irq = kcalloc(nr_irqs, sizeof(*pirq_to_irq), GFP_KERNEL);
- for (i = 0; i < nr_irqs; i++)
- pirq_to_irq[i] = -1;
-
evtchn_to_irq = kcalloc(NR_EVENT_CHANNELS, sizeof(*evtchn_to_irq),
GFP_KERNEL);
for (i = 0; i < NR_EVENT_CHANNELS; i++)
--
1.5.6.5
Running out of IRQs need not be fatal to the machine as a whole.
Signed-off-by: Ian Campbell <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
---
drivers/xen/events.c | 22 ++++++++++++++--------
1 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 7c36689..3566c00 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -406,7 +406,7 @@ static void xen_irq_init(unsigned irq)
list_add_tail(&info->list, &xen_irq_list_head);
}
-static int xen_allocate_irq_dynamic(void)
+static int __must_check xen_allocate_irq_dynamic(void)
{
int first = 0;
int irq;
@@ -425,15 +425,12 @@ static int xen_allocate_irq_dynamic(void)
irq = irq_alloc_desc_from(first, -1);
- if (irq < 0)
- panic("No available IRQ to bind to: increase nr_irqs!\n");
-
xen_irq_init(irq);
return irq;
}
-static int xen_allocate_irq_gsi(unsigned gsi)
+static int __must_check xen_allocate_irq_gsi(unsigned gsi)
{
int irq;
@@ -452,9 +449,6 @@ static int xen_allocate_irq_gsi(unsigned gsi)
else
irq = irq_alloc_desc_at(gsi, -1);
- if (irq < 0)
- panic("Unable to allocate to IRQ%d (%d)\n", gsi, irq);
-
xen_irq_init(irq);
return irq;
@@ -640,6 +634,8 @@ int xen_bind_pirq_gsi_to_irq(unsigned gsi,
}
irq = xen_allocate_irq_gsi(gsi);
+ if (irq < 0)
+ goto out;
set_irq_chip_and_handler_name(irq, &xen_pirq_chip,
handle_level_irq, name);
@@ -771,6 +767,8 @@ int bind_evtchn_to_irq(unsigned int evtchn)
if (irq == -1) {
irq = xen_allocate_irq_dynamic();
+ if (irq == -1)
+ goto out;
set_irq_chip_and_handler_name(irq, &xen_dynamic_chip,
handle_fasteoi_irq, "event");
@@ -778,6 +776,7 @@ int bind_evtchn_to_irq(unsigned int evtchn)
xen_irq_info_evtchn_init(irq, evtchn);
}
+out:
spin_unlock(&irq_mapping_update_lock);
return irq;
@@ -829,6 +828,8 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
if (irq == -1) {
irq = xen_allocate_irq_dynamic();
+ if (irq == -1)
+ goto out;
set_irq_chip_and_handler_name(irq, &xen_percpu_chip,
handle_percpu_irq, "virq");
@@ -845,6 +846,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
bind_evtchn_to_cpu(evtchn, cpu);
}
+out:
spin_unlock(&irq_mapping_update_lock);
return irq;
@@ -897,6 +899,8 @@ int bind_evtchn_to_irqhandler(unsigned int evtchn,
int retval;
irq = bind_evtchn_to_irq(evtchn);
+ if (irq < 0)
+ return irq;
retval = request_irq(irq, handler, irqflags, devname, dev_id);
if (retval != 0) {
unbind_from_irq(irq);
@@ -915,6 +919,8 @@ int bind_virq_to_irqhandler(unsigned int virq, unsigned int cpu,
int retval;
irq = bind_virq_to_irq(virq, cpu);
+ if (irq < 0)
+ return irq;
retval = request_irq(irq, handler, irqflags, devname, dev_id);
if (retval != 0) {
unbind_from_irq(irq);
--
1.5.6.5
In a PVHVM kernel not all interrupts are Xen interrupts (APIC interrupts can also be present).
Currently we get away with walking over all interrupts because the
lookup in the irq_info array simply returns IRQT_UNBOUND and we ignore
it. However this array will be going away in a future patch so we need
to manually track which interrupts have been allocated by the Xen
events infrastructure.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 59 +++++++++++++++++++++++++++++++++++++------------
1 files changed, 44 insertions(+), 15 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index cf372d4..e119989 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -56,6 +56,8 @@
*/
static DEFINE_SPINLOCK(irq_mapping_update_lock);
+static LIST_HEAD(xen_irq_list_head);
+
/* IRQ <-> VIRQ mapping. */
static DEFINE_PER_CPU(int [NR_VIRQS], virq_to_irq) = {[0 ... NR_VIRQS-1] = -1};
@@ -85,7 +87,9 @@ enum xen_irq_type {
*/
struct irq_info
{
+ struct list_head list;
enum xen_irq_type type; /* type */
+ unsigned irq;
unsigned short evtchn; /* event channel */
unsigned short cpu; /* cpu bound */
@@ -135,6 +139,7 @@ static void xen_irq_info_common_init(struct irq_info *info,
BUG_ON(info->type != IRQT_UNBOUND && info->type != type);
info->type = type;
+ info->irq = irq;
info->evtchn = evtchn;
info->cpu = cpu;
@@ -311,10 +316,11 @@ static void init_evtchn_cpu_bindings(void)
{
int i;
#ifdef CONFIG_SMP
- struct irq_desc *desc;
+ struct irq_info *info;
/* By default all event channels notify CPU#0. */
- for_each_irq_desc(i, desc) {
+ list_for_each_entry(info, &xen_irq_list_head, list) {
+ struct irq_desc *desc = irq_to_desc(info->irq);
cpumask_copy(desc->irq_data.affinity, cpumask_of(0));
}
#endif
@@ -397,6 +403,21 @@ static void unmask_evtchn(int port)
put_cpu();
}
+static void xen_irq_init(unsigned irq)
+{
+ struct irq_info *info;
+ struct irq_desc *desc = irq_to_desc(irq);
+
+ /* By default all event channels notify CPU#0. */
+ cpumask_copy(desc->irq_data.affinity, cpumask_of(0));
+
+ info = &irq_info[irq];
+
+ info->type = IRQT_UNBOUND;
+
+ list_add_tail(&info->list, &xen_irq_list_head);
+}
+
static int xen_allocate_irq_dynamic(void)
{
int first = 0;
@@ -426,6 +447,8 @@ retry:
if (irq < 0)
panic("No available IRQ to bind to: increase nr_irqs!\n");
+ xen_irq_init(irq);
+
return irq;
}
@@ -444,18 +467,25 @@ static int xen_allocate_irq_gsi(unsigned gsi)
/* Legacy IRQ descriptors are already allocated by the arch. */
if (gsi < NR_IRQS_LEGACY)
- return gsi;
+ irq = gsi;
+ else
+ irq = irq_alloc_desc_at(gsi, -1);
- irq = irq_alloc_desc_at(gsi, -1);
if (irq < 0)
panic("Unable to allocate to IRQ%d (%d)\n", gsi, irq);
+ xen_irq_init(irq);
+
return irq;
}
static void xen_free_irq(unsigned irq)
{
- irq_info[irq].type = IRQT_UNBOUND;
+ struct irq_info *info = &irq_info[irq];
+
+ info->type = IRQT_UNBOUND;
+
+ list_del(&info->list);
/* Legacy IRQ descriptors are managed by the arch. */
if (irq < NR_IRQS_LEGACY)
@@ -586,16 +616,14 @@ static void ack_pirq(struct irq_data *data)
static int find_irq_by_gsi(unsigned gsi)
{
- int irq;
-
- for (irq = 0; irq < nr_irqs; irq++) {
- struct irq_info *info = info_for_irq(irq);
+ struct irq_info *info;
- if (info == NULL || info->type != IRQT_PIRQ)
+ list_for_each_entry(info, &xen_irq_list_head, list) {
+ if (info->type != IRQT_PIRQ)
continue;
- if (gsi_from_irq(irq) == gsi)
- return irq;
+ if (info->u.pirq.gsi == gsi)
+ return info->irq;
}
return -1;
@@ -1374,7 +1402,8 @@ void xen_poll_irq(int irq)
void xen_irq_resume(void)
{
- unsigned int cpu, irq, evtchn;
+ unsigned int cpu, evtchn;
+ struct irq_info *info;
init_evtchn_cpu_bindings();
@@ -1383,8 +1412,8 @@ void xen_irq_resume(void)
mask_evtchn(evtchn);
/* No IRQ <-> event-channel mappings. */
- for (irq = 0; irq < nr_irqs; irq++)
- irq_info[irq].evtchn = 0; /* zap event-channel binding */
+ list_for_each_entry(info, &xen_irq_list_head, list)
+ info->evtchn = 0; /* zap event-channel binding */
for (evtchn = 0; evtchn < NR_EVENT_CHANNELS; evtchn++)
evtchn_to_irq[evtchn] = -1;
--
1.5.6.5
Following the example set by xen_allocate_pirq_msi and
xen_bind_pirq_msi_to_irq:
xen_allocate_pirq becomes xen_allocate_pirq_gsi and now only allocates
a pirq number and does not bind it.
xen_map_pirq_gsi becomes xen_bind_pirq_gsi_to_irq and binds an
existing pirq.
Signed-off-by: Ian Campbell <[email protected]>
---
arch/x86/pci/xen.c | 41 +++++++++++++++++++++++++++++------------
drivers/xen/events.c | 7 ++++---
include/xen/events.h | 10 +++++++---
3 files changed, 40 insertions(+), 18 deletions(-)
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 8c4085a..e37b407 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -50,7 +50,7 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi,
name = "ioapic-level";
}
- irq = xen_map_pirq_gsi(map_irq.pirq, gsi, shareable, name);
+ irq = xen_bind_pirq_gsi_to_irq(gsi, map_irq.pirq, shareable, name);
printk(KERN_DEBUG "xen: --> irq=%d, pirq=%d\n", irq, map_irq.pirq);
@@ -237,6 +237,7 @@ static int xen_pcifront_enable_irq(struct pci_dev *dev)
{
int rc;
int share = 1;
+ int pirq;
u8 gsi;
rc = pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &gsi);
@@ -246,13 +247,21 @@ static int xen_pcifront_enable_irq(struct pci_dev *dev)
return rc;
}
+ rc = xen_allocate_pirq_gsi(gsi);
+ if (rc < 0) {
+ dev_warn(&dev->dev, "Xen PCI: failed to allocate a PIRQ for GSI%d: %d\n",
+ gsi, rc);
+ return rc;
+ }
+ pirq = rc;
+
if (gsi < NR_IRQS_LEGACY)
share = 0;
- rc = xen_allocate_pirq(gsi, share, "pcifront");
+ rc = xen_bind_pirq_gsi_to_irq(gsi, pirq, share, "pcifront");
if (rc < 0) {
- dev_warn(&dev->dev, "Xen PCI: failed to register GSI%d: %d\n",
- gsi, rc);
+ dev_warn(&dev->dev, "Xen PCI: failed to bind GSI%d (PIRQ%d) to IRQ: %d\n",
+ gsi, pirq, rc);
return rc;
}
@@ -309,7 +318,7 @@ int __init pci_xen_hvm_init(void)
#ifdef CONFIG_XEN_DOM0
static int xen_register_pirq(u32 gsi, int triggering)
{
- int rc, irq;
+ int rc, pirq, irq = -1;
struct physdev_map_pirq map_irq;
int shareable = 0;
char *name;
@@ -325,17 +334,20 @@ static int xen_register_pirq(u32 gsi, int triggering)
name = "ioapic-level";
}
- irq = xen_allocate_pirq(gsi, shareable, name);
-
- printk(KERN_DEBUG "xen: --> irq=%d\n", irq);
+ pirq = xen_allocate_pirq_gsi(gsi);
+ if (pirq < 0)
+ goto out;
+ irq = xen_bind_pirq_gsi_to_irq(gsi, pirq, shareable, name);
if (irq < 0)
goto out;
+ printk(KERN_DEBUG "xen: --> pirq=%d -> irq=%d\n", pirq, irq);
+
map_irq.domid = DOMID_SELF;
map_irq.type = MAP_PIRQ_TYPE_GSI;
map_irq.index = gsi;
- map_irq.pirq = irq;
+ map_irq.pirq = pirq;
rc = HYPERVISOR_physdev_op(PHYSDEVOP_map_pirq, &map_irq);
if (rc) {
@@ -422,13 +434,18 @@ static int __init pci_xen_initial_domain(void)
void __init xen_setup_pirqs(void)
{
- int irq;
+ int pirq, irq;
pci_xen_initial_domain();
if (0 == nr_ioapics) {
- for (irq = 0; irq < NR_IRQS_LEGACY; irq++)
- xen_allocate_pirq(irq, 0, "xt-pic");
+ for (irq = 0; irq < NR_IRQS_LEGACY; irq++) {
+ pirq = xen_allocate_pirq_gsi(irq);
+ if (WARN(pirq < 0,
+ "Could not allocate PIRQ for legacy interrupt\n"))
+ break;
+ irq = xen_bind_pirq_gsi_to_irq(irq, pirq, 0, "xt-pic");
+ }
return;
}
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index e828456..a40b2a1 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -568,9 +568,9 @@ static int find_irq_by_gsi(unsigned gsi)
return -1;
}
-int xen_allocate_pirq(unsigned gsi, int shareable, char *name)
+int xen_allocate_pirq_gsi(unsigned gsi)
{
- return xen_map_pirq_gsi(gsi, gsi, shareable, name);
+ return gsi;
}
/*
@@ -580,7 +580,8 @@ int xen_allocate_pirq(unsigned gsi, int shareable, char *name)
* Note: We don't assign an event channel until the irq actually started
* up. Return an existing irq if we've already got one for the gsi.
*/
-int xen_map_pirq_gsi(unsigned pirq, unsigned gsi, int shareable, char *name)
+int xen_bind_pirq_gsi_to_irq(unsigned gsi,
+ unsigned pirq, int shareable, char *name)
{
int irq = -1;
struct physdev_irq irq_op;
diff --git a/include/xen/events.h b/include/xen/events.h
index 99a64f0..32ebeba 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -68,12 +68,16 @@ int xen_set_callback_via(uint64_t via);
void xen_evtchn_do_upcall(struct pt_regs *regs);
void xen_hvm_evtchn_do_upcall(void);
-/* Allocate an irq for a physical interrupt, given a gsi. */
-int xen_allocate_pirq(unsigned gsi, int shareable, char *name);
-int xen_map_pirq_gsi(unsigned pirq, unsigned gsi, int shareable, char *name);
+/* Allocate a pirq for a physical interrupt, given a gsi. */
+int xen_allocate_pirq_gsi(unsigned gsi);
+/* Bind a pirq for a physical interrupt to an irq. */
+int xen_bind_pirq_gsi_to_irq(unsigned gsi,
+ unsigned pirq, int shareable, char *name);
#ifdef CONFIG_PCI_MSI
+/* Allocate a pirq for a MSI style physical interrupt. */
int xen_allocate_pirq_msi(struct pci_dev *dev, struct msi_desc *msidesc);
+/* Bind an PSI pirq to an irq. */
int xen_bind_pirq_msi_to_irq(struct pci_dev *dev, struct msi_desc *msidesc,
int pirq, int vector, const char *name);
#endif
--
1.5.6.5
Removes nr_irq sized array allocation at start of day.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 31 ++++++++++++++++---------------
1 files changed, 16 insertions(+), 15 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index e119989..002283e 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -107,7 +107,6 @@ struct irq_info
#define PIRQ_NEEDS_EOI (1 << 0)
#define PIRQ_SHAREABLE (1 << 1)
-static struct irq_info *irq_info;
static int *pirq_to_irq;
static int *evtchn_to_irq;
@@ -125,7 +124,7 @@ static struct irq_chip xen_pirq_chip;
/* Get info for IRQ */
static struct irq_info *info_for_irq(unsigned irq)
{
- return &irq_info[irq];
+ return get_irq_data(irq);
}
/* Constructors for packed IRQ information. */
@@ -309,7 +308,7 @@ static void bind_evtchn_to_cpu(unsigned int chn, unsigned int cpu)
clear_bit(chn, per_cpu(cpu_evtchn_mask, cpu_from_irq(irq)));
set_bit(chn, per_cpu(cpu_evtchn_mask, cpu));
- irq_info[irq].cpu = cpu;
+ info_for_irq(irq)->cpu = cpu;
}
static void init_evtchn_cpu_bindings(void)
@@ -328,7 +327,6 @@ static void init_evtchn_cpu_bindings(void)
for_each_possible_cpu(i)
memset(per_cpu(cpu_evtchn_mask, i),
(i == 0) ? ~0 : 0, sizeof(*per_cpu(cpu_evtchn_mask, i)));
-
}
static inline void clear_evtchn(int port)
@@ -411,10 +409,14 @@ static void xen_irq_init(unsigned irq)
/* By default all event channels notify CPU#0. */
cpumask_copy(desc->irq_data.affinity, cpumask_of(0));
- info = &irq_info[irq];
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL)
+ panic("Unable to allocate metadata for IRQ%d\n", irq);
info->type = IRQT_UNBOUND;
+ set_irq_data(irq, info);
+
list_add_tail(&info->list, &xen_irq_list_head);
}
@@ -481,12 +483,14 @@ static int xen_allocate_irq_gsi(unsigned gsi)
static void xen_free_irq(unsigned irq)
{
- struct irq_info *info = &irq_info[irq];
-
- info->type = IRQT_UNBOUND;
+ struct irq_info *info = get_irq_data(irq);
list_del(&info->list);
+ set_irq_data(irq, NULL);
+
+ kfree(info);
+
/* Legacy IRQ descriptors are managed by the arch. */
if (irq < NR_IRQS_LEGACY)
return;
@@ -649,10 +653,9 @@ int xen_bind_pirq_gsi_to_irq(unsigned gsi,
spin_lock(&irq_mapping_update_lock);
- if ((pirq > nr_irqs) || (gsi > nr_irqs)) {
- printk(KERN_WARNING "xen_map_pirq_gsi: %s %s is incorrect!\n",
- pirq > nr_irqs ? "pirq" :"",
- gsi > nr_irqs ? "gsi" : "");
+ if (pirq > nr_irqs) {
+ printk(KERN_WARNING "xen_map_pirq_gsi: pirq %d > nr_irqs %d!\n",
+ pirq, nr_irqs);
goto out;
}
@@ -889,7 +892,7 @@ static void unbind_from_irq(unsigned int irq)
evtchn_to_irq[evtchn] = -1;
}
- BUG_ON(irq_info[irq].type == IRQT_UNBOUND);
+ BUG_ON(info_for_irq(irq)->type == IRQT_UNBOUND);
xen_free_irq(irq);
@@ -1509,8 +1512,6 @@ void __init xen_init_IRQ(void)
{
int i;
- irq_info = kcalloc(nr_irqs, sizeof(*irq_info), GFP_KERNEL);
-
/* We are using nr_irqs as the maximum number of pirq available but
* that number is actually chosen by Xen and we don't know exactly
* what it is. Be careful choosing high pirq numbers. */
--
1.5.6.5
I can't see any reason why it isn't already.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 28 ++++++++--------------------
1 files changed, 8 insertions(+), 20 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index a40b2a1..2dffa43 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -107,19 +107,9 @@ static struct irq_info *irq_info;
static int *pirq_to_irq;
static int *evtchn_to_irq;
-struct cpu_evtchn_s {
- unsigned long bits[NR_EVENT_CHANNELS/BITS_PER_LONG];
-};
-
-static __initdata struct cpu_evtchn_s init_evtchn_mask = {
- .bits[0 ... (NR_EVENT_CHANNELS/BITS_PER_LONG)-1] = ~0ul,
-};
-static struct cpu_evtchn_s *cpu_evtchn_mask_p = &init_evtchn_mask;
-static inline unsigned long *cpu_evtchn_mask(int cpu)
-{
- return cpu_evtchn_mask_p[cpu].bits;
-}
+static DEFINE_PER_CPU(unsigned long [NR_EVENT_CHANNELS/BITS_PER_LONG],
+ cpu_evtchn_mask);
/* Xen will never allocate port zero for any purpose. */
#define VALID_EVTCHN(chn) ((chn) != 0)
@@ -257,7 +247,7 @@ static inline unsigned long active_evtchns(unsigned int cpu,
unsigned int idx)
{
return (sh->evtchn_pending[idx] &
- cpu_evtchn_mask(cpu)[idx] &
+ per_cpu(cpu_evtchn_mask, cpu)[idx] &
~sh->evtchn_mask[idx]);
}
@@ -270,8 +260,8 @@ static void bind_evtchn_to_cpu(unsigned int chn, unsigned int cpu)
cpumask_copy(irq_to_desc(irq)->irq_data.affinity, cpumask_of(cpu));
#endif
- clear_bit(chn, cpu_evtchn_mask(cpu_from_irq(irq)));
- set_bit(chn, cpu_evtchn_mask(cpu));
+ clear_bit(chn, per_cpu(cpu_evtchn_mask, cpu_from_irq(irq)));
+ set_bit(chn, per_cpu(cpu_evtchn_mask, cpu));
irq_info[irq].cpu = cpu;
}
@@ -289,8 +279,8 @@ static void init_evtchn_cpu_bindings(void)
#endif
for_each_possible_cpu(i)
- memset(cpu_evtchn_mask(i),
- (i == 0) ? ~0 : 0, sizeof(struct cpu_evtchn_s));
+ memset(per_cpu(cpu_evtchn_mask, i),
+ (i == 0) ? ~0 : 0, sizeof(*per_cpu(cpu_evtchn_mask, i)));
}
@@ -925,7 +915,7 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id)
{
struct shared_info *sh = HYPERVISOR_shared_info;
int cpu = smp_processor_id();
- unsigned long *cpu_evtchn = cpu_evtchn_mask(cpu);
+ unsigned long *cpu_evtchn = per_cpu(cpu_evtchn_mask, cpu);
int i;
unsigned long flags;
static DEFINE_SPINLOCK(debug_lock);
@@ -1462,8 +1452,6 @@ void __init xen_init_IRQ(void)
{
int i;
- cpu_evtchn_mask_p = kcalloc(nr_cpu_ids, sizeof(struct cpu_evtchn_s),
- GFP_KERNEL);
irq_info = kcalloc(nr_irqs, sizeof(*irq_info), GFP_KERNEL);
/* We are using nr_irqs as the maximum number of pirq available but
--
1.5.6.5
Fix initial value of irq so that first goto out (if pirq or gsi
arguments are too large) actually returns an error.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 684b095..2ce95a6 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -596,7 +596,7 @@ int xen_allocate_pirq(unsigned gsi, int shareable, char *name)
*/
int xen_map_pirq_gsi(unsigned pirq, unsigned gsi, int shareable, char *name)
{
- int irq = 0;
+ int irq = -1;
struct physdev_irq irq_op;
spin_lock(&irq_mapping_update_lock);
--
1.5.6.5
I was unable to find any user of these functions in either the
functionality pending for 2.6.39 or the xen/next-2.6.32 branch of
xen.git
An exception to this was xen_gsi_from_irq which did appear to be used
in xen/next-2.6.32's pciback. However in the 2.6.39 version of pciback
xen_pirq_from_irq is, correctly AFAICT, used instead.
Only a minority of functions in events.h use "extern" so drop it from
those places for consistency.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 20 --------------------
include/xen/events.h | 12 +++---------
2 files changed, 3 insertions(+), 29 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index a9c154d..d52defd 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -222,16 +222,6 @@ static unsigned gsi_from_irq(unsigned irq)
return info->u.pirq.gsi;
}
-static unsigned vector_from_irq(unsigned irq)
-{
- struct irq_info *info = info_for_irq(irq);
-
- BUG_ON(info == NULL);
- BUG_ON(info->type != IRQT_PIRQ);
-
- return info->u.pirq.vector;
-}
-
static enum xen_irq_type type_from_irq(unsigned irq)
{
return info_for_irq(irq)->type;
@@ -716,16 +706,6 @@ out:
return rc;
}
-int xen_vector_from_irq(unsigned irq)
-{
- return vector_from_irq(irq);
-}
-
-int xen_gsi_from_irq(unsigned irq)
-{
- return gsi_from_irq(irq);
-}
-
int xen_irq_from_pirq(unsigned pirq)
{
return pirq_to_irq[pirq];
diff --git a/include/xen/events.h b/include/xen/events.h
index f6fed94..99a64f0 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -41,9 +41,9 @@ static inline void notify_remote_via_evtchn(int port)
(void)HYPERVISOR_event_channel_op(EVTCHNOP_send, &send);
}
-extern void notify_remote_via_irq(int irq);
+void notify_remote_via_irq(int irq);
-extern void xen_irq_resume(void);
+void xen_irq_resume(void);
/* Clear an irq's pending state, in preparation for polling on it */
void xen_clear_irq_pending(int irq);
@@ -62,7 +62,7 @@ void xen_poll_irq_timeout(int irq, u64 timeout);
unsigned irq_from_evtchn(unsigned int evtchn);
/* Xen HVM evtchn vector callback */
-extern void xen_hvm_callback_vector(void);
+void xen_hvm_callback_vector(void);
extern int xen_have_vector_callback;
int xen_set_callback_via(uint64_t via);
void xen_evtchn_do_upcall(struct pt_regs *regs);
@@ -81,12 +81,6 @@ int xen_bind_pirq_msi_to_irq(struct pci_dev *dev, struct msi_desc *msidesc,
/* De-allocates the above mentioned physical interrupt. */
int xen_destroy_irq(int irq);
-/* Return vector allocated to pirq */
-int xen_vector_from_irq(unsigned pirq);
-
-/* Return gsi allocated to pirq */
-int xen_gsi_from_irq(unsigned pirq);
-
/* Return irq from pirq */
int xen_irq_from_pirq(unsigned pirq);
--
1.5.6.5
There is nothing per-cpu about this function.
Signed-off-by: Ian Campbell <[email protected]>
---
drivers/xen/events.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index d52defd..e828456 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -1213,7 +1213,7 @@ static int retrigger_dynirq(struct irq_data *data)
return ret;
}
-static void restore_cpu_pirqs(void)
+static void restore_pirqs(void)
{
int pirq, rc, irq, gsi;
struct physdev_map_pirq map_irq;
@@ -1375,7 +1375,7 @@ void xen_irq_resume(void)
restore_cpu_ipis(cpu);
}
- restore_cpu_pirqs();
+ restore_pirqs();
}
static struct irq_chip xen_dynamic_chip __read_mostly = {
--
1.5.6.5
On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> Changes since last time:
> * correct return value of xen_irq_from_pirq
> * WARN if a pirq cannot be allocated for a legacy IRQ
> * Updated checking comment of "xen: events: do not workaround
> too-small nr_irqs"
>
> The following series makes a few cleanups to the Xen IRQ infrastructure.
> The most important thing is that it removes the need to know about
> nr_irqs and in particular the reliance on nr_irqs being static.
>
> Apart from being generally a good thing this is needed because in 2.6.39
> nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> at 2.6.39.
>
> Dynamically growing nr_irqs also allows us to remove the workaround
> which eats into GSI space if a dynamic IRQ cannot be allocated.
>
> There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> have gone in before, but the penultimate patch really needed to be
> simultaneous) so I haven't bothered to try and pull anything from tip
> into this branch -- it should all be resolved during the merge window
> and bisection won't be too broken since the "eat into GSI space"
> workaround only appears to be needed on a small number of older
> platforms (qemu being the main exception).
<nods>
>
> I have tested:
> * Domain 0 on real h/w and under qemu
> * PV guest, including migration and passthrough of both VF and PF.
> * PVHVM guest, including migration and passthrough of both VF and
> PF.
I am having difficulties with passthrough of an USB device. Somehow the
irq count is not going up at all (both in dom0 and domU) and it looks to
be doing just simple polling. I've rebased the xen-pciback to be on top
of your changes and apply cleanly. The whole lot is now in #master
MSI and MSI-X devices work just fine in both Dom0 and DomU case so
it is something special with the legacy IRQs. Probably forgot something
simple...
> > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > have gone in before, but the penultimate patch really needed to be
> > simultaneous) so I haven't bothered to try and pull anything from tip
> > into this branch -- it should all be resolved during the merge window
> > and bisection won't be too broken since the "eat into GSI space"
> > workaround only appears to be needed on a small number of older
> > platforms (qemu being the main exception).
>
> <nods>
> >
> > I have tested:
> > * Domain 0 on real h/w and under qemu
> > * PV guest, including migration and passthrough of both VF and PF.
> > * PVHVM guest, including migration and passthrough of both VF and
> > PF.
>
> I am having difficulties with passthrough of an USB device. Somehow the
> irq count is not going up at all (both in dom0 and domU) and it looks to
> be doing just simple polling. I've rebased the xen-pciback to be on top
> of your changes and apply cleanly. The whole lot is now in #master
>
> MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> it is something special with the legacy IRQs. Probably forgot something
> simple...
This looks like it would fix the culprit:
[edit: not entirely]
diff --git a/drivers/xen/pciback/conf_space_header.c b/drivers/xen/pciback/conf_space_header.c
index 22ad0f5..3eeb197 100644
--- a/drivers/xen/pciback/conf_space_header.c
+++ b/drivers/xen/pciback/conf_space_header.c
@@ -6,6 +6,7 @@
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <xen/events.h>
#include "pciback.h"
#include "conf_space.h"
@@ -246,7 +247,7 @@ static int pciback_read_device(struct pci_dev *dev, int offset,
static int interrupt_read(struct pci_dev *dev, int offset, u8 * value,
void *data)
{
- *value = (u8) dev->irq;
+ *value = (u8) dev->irq ? xen_pirq_from_irq(dev->irq) : 0;
return 0;
}
Thought I am not sure if this is absolutly correct. Are the PIRQs == GSI or should
we provide another piece of code to extract info->u.pirq.gsi?
I also see this when using xen-pciback.hide=..
pciback 0000:00:1d.7: found PCI INT A -> IRQ 14
pciback 0000:00:1d.7: sharing IRQ 14 with 0000:00:1d.0
pciback 0000:00:1d.0: found PCI INT A -> IRQ 14
pciback 0000:00:1d.0: sharing IRQ 14 with 0000:00:1d.7
pciback 0000:00:1d.1: found PCI INT B -> IRQ 5
pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:16.3
pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:16.7
pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:1a.2
..
.. which are clearly the Linux IRQ's, not the PIRQ/GSI values,
so some other patch for the xen-pciback will be needed as well.
> I am having difficulties with passthrough of an USB device. Somehow the
> irq count is not going up at all (both in dom0 and domU) and it looks to
> be doing just simple polling. I've rebased the xen-pciback to be on top
> of your changes and apply cleanly. The whole lot is now in #master
I think your patchset breaks the 'xc_physdev_map_pirq' call. Looks like
libxl_pci.c reads from /sys/bus/pci/devices/<BDF>/irq to figure out what value
to pass. With these patches we wouldn't neccessarily pass in the right
PIRQ value. Any ideas?
On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > Changes since last time:
> > * correct return value of xen_irq_from_pirq
> > * WARN if a pirq cannot be allocated for a legacy IRQ
> > * Updated checking comment of "xen: events: do not workaround
> > too-small nr_irqs"
> >
> > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > The most important thing is that it removes the need to know about
> > nr_irqs and in particular the reliance on nr_irqs being static.
> >
> > Apart from being generally a good thing this is needed because in 2.6.39
> > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > at 2.6.39.
> >
> > Dynamically growing nr_irqs also allows us to remove the workaround
> > which eats into GSI space if a dynamic IRQ cannot be allocated.
> >
> > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > have gone in before, but the penultimate patch really needed to be
> > simultaneous) so I haven't bothered to try and pull anything from tip
> > into this branch -- it should all be resolved during the merge window
> > and bisection won't be too broken since the "eat into GSI space"
> > workaround only appears to be needed on a small number of older
> > platforms (qemu being the main exception).
>
> <nods>
> >
> > I have tested:
> > * Domain 0 on real h/w and under qemu
> > * PV guest, including migration and passthrough of both VF and PF.
> > * PVHVM guest, including migration and passthrough of both VF and
> > PF.
>
> I am having difficulties with passthrough of an USB device. Somehow the
> irq count is not going up at all (both in dom0 and domU) and it looks to
> be doing just simple polling. I've rebased the xen-pciback to be on top
> of your changes and apply cleanly. The whole lot is now in #master
>
> MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> it is something special with the legacy IRQs. Probably forgot something
> simple...
Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
not-MSI(-X)?
Ian.
On Thu, 2011-03-10 at 23:23 +0000, Konrad Rzeszutek Wilk wrote:
> > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > have gone in before, but the penultimate patch really needed to be
> > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > into this branch -- it should all be resolved during the merge window
> > > and bisection won't be too broken since the "eat into GSI space"
> > > workaround only appears to be needed on a small number of older
> > > platforms (qemu being the main exception).
> >
> > <nods>
> > >
> > > I have tested:
> > > * Domain 0 on real h/w and under qemu
> > > * PV guest, including migration and passthrough of both VF and PF.
> > > * PVHVM guest, including migration and passthrough of both VF and
> > > PF.
> >
> > I am having difficulties with passthrough of an USB device. Somehow the
> > irq count is not going up at all (both in dom0 and domU) and it looks to
> > be doing just simple polling. I've rebased the xen-pciback to be on top
> > of your changes and apply cleanly. The whole lot is now in #master
> >
> > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > it is something special with the legacy IRQs. Probably forgot something
> > simple...
>
> This looks like it would fix the culprit:
> [edit: not entirely]
>
> diff --git a/drivers/xen/pciback/conf_space_header.c b/drivers/xen/pciback/conf_space_header.c
> index 22ad0f5..3eeb197 100644
> --- a/drivers/xen/pciback/conf_space_header.c
> +++ b/drivers/xen/pciback/conf_space_header.c
> @@ -6,6 +6,7 @@
>
> #include <linux/kernel.h>
> #include <linux/pci.h>
> +#include <xen/events.h>
> #include "pciback.h"
> #include "conf_space.h"
>
> @@ -246,7 +247,7 @@ static int pciback_read_device(struct pci_dev *dev, int offset,
> static int interrupt_read(struct pci_dev *dev, int offset, u8 * value,
> void *data)
> {
> - *value = (u8) dev->irq;
> + *value = (u8) dev->irq ? xen_pirq_from_irq(dev->irq) : 0;
>
> return 0;
> }
>
> Thought I am not sure if this is absolutly correct. Are the PIRQs == GSI
not necessarily. PIRQs (at least as used here) are also an internal Xen
concept which shouldn't be exposed outside of the lowlevel event
handling.
> or should
> we provide another piece of code to extract info->u.pirq.gsi?
I think we've got one already. But I'll need to have a think about what
the correct return is here.
> I also see this when using xen-pciback.hide=..
>
> pciback 0000:00:1d.7: found PCI INT A -> IRQ 14
> pciback 0000:00:1d.7: sharing IRQ 14 with 0000:00:1d.0
> pciback 0000:00:1d.0: found PCI INT A -> IRQ 14
> pciback 0000:00:1d.0: sharing IRQ 14 with 0000:00:1d.7
> pciback 0000:00:1d.1: found PCI INT B -> IRQ 5
> pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:16.3
> pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:16.7
> pciback 0000:00:1d.1: sharing IRQ 5 with 0000:00:1a.2
> ..
> .. which are clearly the Linux IRQ's, not the PIRQ/GSI values,
> so some other patch for the xen-pciback will be needed as well.
Hmm, yes.
Ian.
On Fri, 2011-03-11 at 00:38 +0000, Konrad Rzeszutek Wilk wrote:
> > I am having difficulties with passthrough of an USB device. Somehow the
> > irq count is not going up at all (both in dom0 and domU) and it looks to
> > be doing just simple polling. I've rebased the xen-pciback to be on top
> > of your changes and apply cleanly. The whole lot is now in #master
>
> I think your patchset breaks the 'xc_physdev_map_pirq' call. Looks like
> libxl_pci.c reads from /sys/bus/pci/devices/<BDF>/irq to figure out what value
> to pass. With these patches we wouldn't neccessarily pass in the right
> PIRQ value. Any ideas?
I'll have to have a think. At best there's some confusing usage of the
term pirq in there. (which is not unexpected, various subsystems use it
for different things.)
Certainly I would never expect to read a Xen pirq out of /sys/.../irq
and that appears to be how this value is being used...
Ian.
On Fri, 2011-03-11 at 09:12 +0000, Ian Campbell wrote:
> On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> > On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > > Changes since last time:
> > > * correct return value of xen_irq_from_pirq
> > > * WARN if a pirq cannot be allocated for a legacy IRQ
> > > * Updated checking comment of "xen: events: do not workaround
> > > too-small nr_irqs"
> > >
> > > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > > The most important thing is that it removes the need to know about
> > > nr_irqs and in particular the reliance on nr_irqs being static.
> > >
> > > Apart from being generally a good thing this is needed because in 2.6.39
> > > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > > at 2.6.39.
> > >
> > > Dynamically growing nr_irqs also allows us to remove the workaround
> > > which eats into GSI space if a dynamic IRQ cannot be allocated.
> > >
> > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > have gone in before, but the penultimate patch really needed to be
> > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > into this branch -- it should all be resolved during the merge window
> > > and bisection won't be too broken since the "eat into GSI space"
> > > workaround only appears to be needed on a small number of older
> > > platforms (qemu being the main exception).
> >
> > <nods>
> > >
> > > I have tested:
> > > * Domain 0 on real h/w and under qemu
> > > * PV guest, including migration and passthrough of both VF and PF.
> > > * PVHVM guest, including migration and passthrough of both VF and
> > > PF.
> >
> > I am having difficulties with passthrough of an USB device. Somehow the
> > irq count is not going up at all (both in dom0 and domU) and it looks to
> > be doing just simple polling. I've rebased the xen-pciback to be on top
> > of your changes and apply cleanly. The whole lot is now in #master
> >
> > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > it is something special with the legacy IRQs. Probably forgot something
> > simple...
>
> Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> not-MSI(-X)?
Works for me in a PV guest, at least under the second interpretation (I
don't have any useful devices IRQ < 16). This is with your pciback 0.5.
I see something like what you describe under HVM, although pciback is
not used in that case. xc_physdev_map_pirq is used but not with
the /sys/.../bdf node AFAICT.
More digging required I guess...
Ian.
On Fri, 2011-03-11 at 11:24 +0000, Ian Campbell wrote:
> On Fri, 2011-03-11 at 09:12 +0000, Ian Campbell wrote:
> > On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> > > On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > > > Changes since last time:
> > > > * correct return value of xen_irq_from_pirq
> > > > * WARN if a pirq cannot be allocated for a legacy IRQ
> > > > * Updated checking comment of "xen: events: do not workaround
> > > > too-small nr_irqs"
> > > >
> > > > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > > > The most important thing is that it removes the need to know about
> > > > nr_irqs and in particular the reliance on nr_irqs being static.
> > > >
> > > > Apart from being generally a good thing this is needed because in 2.6.39
> > > > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > > > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > > > at 2.6.39.
> > > >
> > > > Dynamically growing nr_irqs also allows us to remove the workaround
> > > > which eats into GSI space if a dynamic IRQ cannot be allocated.
> > > >
> > > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > > have gone in before, but the penultimate patch really needed to be
> > > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > > into this branch -- it should all be resolved during the merge window
> > > > and bisection won't be too broken since the "eat into GSI space"
> > > > workaround only appears to be needed on a small number of older
> > > > platforms (qemu being the main exception).
> > >
> > > <nods>
> > > >
> > > > I have tested:
> > > > * Domain 0 on real h/w and under qemu
> > > > * PV guest, including migration and passthrough of both VF and PF.
> > > > * PVHVM guest, including migration and passthrough of both VF and
> > > > PF.
> > >
> > > I am having difficulties with passthrough of an USB device. Somehow the
> > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > of your changes and apply cleanly. The whole lot is now in #master
> > >
> > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > it is something special with the legacy IRQs. Probably forgot something
> > > simple...
> >
> > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > not-MSI(-X)?
>
> Works for me in a PV guest, at least under the second interpretation (I
> don't have any useful devices IRQ < 16). This is with your pciback 0.5.
>
> I see something like what you describe under HVM, although pciback is
> not used in that case. xc_physdev_map_pirq is used but not with
> the /sys/.../bdf node AFAICT.
>
> More digging required I guess...
If I just pass through the EHCI device it doesn't work. However if I
also pass-through the associated UHCI controller then I do see the
usb-storage device, interrupts and all. The EHCI controller still
doesn't work though. Note that with a PV guest passing-through the EHCI
controller by itself does work.
The issue I'm seeing seems to have more to do with accessing the memory
mapped BAR than IRQs though:
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
PHYSDEVOP_map_pirq mapped gsi 46 to pirq 18
xen_allocate_pirq_gsi for gsi 46, pirq 18 "ioapic-level"
xen: --> irq=46, pirq=18
ehci_hcd 0000:00:07.0: PCI INT C -> GSI 46 (level, low) -> IRQ 46
ehci: remapped rsrc 0xf3021000 to c8064000
ehci_hcd 0000:00:07.0: setting latency timer to 64
ehci_hcd 0000:00:07.0: EHCI Host Controller
ehci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1
ehci_reset: handshake failed with -19. regs c80640ff
ehci_hcd 0000:00:07.0: Enabling legacy PCI PM
__startup_pirq: irq46 bound to evtchn 19
ehci_hcd 0000:00:07.0: irq 46, io mem 0xf3021000
ehci_reset: handshake failed with -19. regs c80640ff
ehci_run: ehci_reset failed with -19
ehci_hcd 0000:00:07.0: startup error -19
ehci_hcd 0000:00:07.0: USB bus 1 deregistered
ehci_hcd 0000:00:07.0: PCI INT C disabled
ehci_hcd 0000:00:07.0: init 0000:00:07.0 fail, -19
The -19 is -ENODEV from driver/usb/host/ehci-hcd.c:handshake() which is
a readl with nothing to do with IRQs.
Ian.
On Fri, 2011-03-11 at 13:28 +0000, Ian Campbell wrote:
> On Fri, 2011-03-11 at 11:24 +0000, Ian Campbell wrote:
> > On Fri, 2011-03-11 at 09:12 +0000, Ian Campbell wrote:
> > > On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> > > > On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > > > > Changes since last time:
> > > > > * correct return value of xen_irq_from_pirq
> > > > > * WARN if a pirq cannot be allocated for a legacy IRQ
> > > > > * Updated checking comment of "xen: events: do not workaround
> > > > > too-small nr_irqs"
> > > > >
> > > > > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > > > > The most important thing is that it removes the need to know about
> > > > > nr_irqs and in particular the reliance on nr_irqs being static.
> > > > >
> > > > > Apart from being generally a good thing this is needed because in 2.6.39
> > > > > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > > > > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > > > > at 2.6.39.
> > > > >
> > > > > Dynamically growing nr_irqs also allows us to remove the workaround
> > > > > which eats into GSI space if a dynamic IRQ cannot be allocated.
> > > > >
> > > > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > > > have gone in before, but the penultimate patch really needed to be
> > > > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > > > into this branch -- it should all be resolved during the merge window
> > > > > and bisection won't be too broken since the "eat into GSI space"
> > > > > workaround only appears to be needed on a small number of older
> > > > > platforms (qemu being the main exception).
> > > >
> > > > <nods>
> > > > >
> > > > > I have tested:
> > > > > * Domain 0 on real h/w and under qemu
> > > > > * PV guest, including migration and passthrough of both VF and PF.
> > > > > * PVHVM guest, including migration and passthrough of both VF and
> > > > > PF.
> > > >
> > > > I am having difficulties with passthrough of an USB device. Somehow the
> > > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > > of your changes and apply cleanly. The whole lot is now in #master
> > > >
> > > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > > it is something special with the legacy IRQs. Probably forgot something
> > > > simple...
> > >
> > > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > > not-MSI(-X)?
> >
> > Works for me in a PV guest, at least under the second interpretation (I
> > don't have any useful devices IRQ < 16). This is with your pciback 0.5.
> >
> > I see something like what you describe under HVM, although pciback is
> > not used in that case. xc_physdev_map_pirq is used but not with
> > the /sys/.../bdf node AFAICT.
> >
> > More digging required I guess...
>
> If I just pass through the EHCI device it doesn't work. However if I
> also pass-through the associated UHCI controller then I do see the
> usb-storage device, interrupts and all. The EHCI controller still
> doesn't work though. Note that with a PV guest passing-through the EHCI
> controller by itself does work.
>
> The issue I'm seeing seems to have more to do with accessing the memory
> mapped BAR than IRQs though:
I seem to see this with 2.6.32 dom0 as well. This IRQ series of patches
doesn't appear to make a difference when running 2.6.38-rcX+.
Unfortunately I think the error must be elsewhere :-(
Ian.
> > > > I am having difficulties with passthrough of an USB device. Somehow the
> > > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > > of your changes and apply cleanly. The whole lot is now in #master
> > > >
> > > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > > it is something special with the legacy IRQs. Probably forgot something
> > > > simple...
> > >
> > > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > > not-MSI(-X)?
not-MSI(-X) ones. I've this fancy LSI 1030 controller that sits at GSI 46 on this
SuperMicro board and it is quite confused.
> >
> > Works for me in a PV guest, at least under the second interpretation (I
> > don't have any useful devices IRQ < 16). This is with your pciback 0.5.
OK. Let me boot up that machine and see what /proc/interrupts look like. Maybe
the xen-pciback.hide=() throws a lot of things out of whack.
> >
> > I see something like what you describe under HVM, although pciback is
> > not used in that case. xc_physdev_map_pirq is used but not with
> > the /sys/.../bdf node AFAICT.
> >
> > More digging required I guess...
>
> If I just pass through the EHCI device it doesn't work. However if I
> also pass-through the associated UHCI controller then I do see the
> usb-storage device, interrupts and all. The EHCI controller still
> doesn't work though. Note that with a PV guest passing-through the EHCI
> controller by itself does work.
>
> The issue I'm seeing seems to have more to do with accessing the memory
> mapped BAR than IRQs though:
>
> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> PHYSDEVOP_map_pirq mapped gsi 46 to pirq 18
> xen_allocate_pirq_gsi for gsi 46, pirq 18 "ioapic-level"
> xen: --> irq=46, pirq=18
> ehci_hcd 0000:00:07.0: PCI INT C -> GSI 46 (level, low) -> IRQ 46
> ehci: remapped rsrc 0xf3021000 to c8064000
<scratch> If you hit the 'i' in the debug console is this
BAR visible there? That is f3021?
> ehci_hcd 0000:00:07.0: setting latency timer to 64
> ehci_hcd 0000:00:07.0: EHCI Host Controller
> ehci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1
> ehci_reset: handshake failed with -19. regs c80640ff
> ehci_hcd 0000:00:07.0: Enabling legacy PCI PM
> __startup_pirq: irq46 bound to evtchn 19
> ehci_hcd 0000:00:07.0: irq 46, io mem 0xf3021000
> ehci_reset: handshake failed with -19. regs c80640ff
> ehci_run: ehci_reset failed with -19
> ehci_hcd 0000:00:07.0: startup error -19
> ehci_hcd 0000:00:07.0: USB bus 1 deregistered
> ehci_hcd 0000:00:07.0: PCI INT C disabled
> ehci_hcd 0000:00:07.0: init 0000:00:07.0 fail, -19
>
> The -19 is -ENODEV from driver/usb/host/ehci-hcd.c:handshake() which is
> a readl with nothing to do with IRQs.
<nods>
>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> [email protected]
> http://lists.xensource.com/xen-devel
On Fri, 2011-03-11 at 14:24 +0000, Ian Campbell wrote:
> On Fri, 2011-03-11 at 13:28 +0000, Ian Campbell wrote:
> > On Fri, 2011-03-11 at 11:24 +0000, Ian Campbell wrote:
> > > On Fri, 2011-03-11 at 09:12 +0000, Ian Campbell wrote:
> > > > On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> > > > > On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > > > > > Changes since last time:
> > > > > > * correct return value of xen_irq_from_pirq
> > > > > > * WARN if a pirq cannot be allocated for a legacy IRQ
> > > > > > * Updated checking comment of "xen: events: do not workaround
> > > > > > too-small nr_irqs"
> > > > > >
> > > > > > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > > > > > The most important thing is that it removes the need to know about
> > > > > > nr_irqs and in particular the reliance on nr_irqs being static.
> > > > > >
> > > > > > Apart from being generally a good thing this is needed because in 2.6.39
> > > > > > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > > > > > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > > > > > at 2.6.39.
> > > > > >
> > > > > > Dynamically growing nr_irqs also allows us to remove the workaround
> > > > > > which eats into GSI space if a dynamic IRQ cannot be allocated.
> > > > > >
> > > > > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > > > > have gone in before, but the penultimate patch really needed to be
> > > > > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > > > > into this branch -- it should all be resolved during the merge window
> > > > > > and bisection won't be too broken since the "eat into GSI space"
> > > > > > workaround only appears to be needed on a small number of older
> > > > > > platforms (qemu being the main exception).
> > > > >
> > > > > <nods>
> > > > > >
> > > > > > I have tested:
> > > > > > * Domain 0 on real h/w and under qemu
> > > > > > * PV guest, including migration and passthrough of both VF and PF.
> > > > > > * PVHVM guest, including migration and passthrough of both VF and
> > > > > > PF.
> > > > >
> > > > > I am having difficulties with passthrough of an USB device. Somehow the
> > > > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > > > of your changes and apply cleanly. The whole lot is now in #master
> > > > >
> > > > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > > > it is something special with the legacy IRQs. Probably forgot something
> > > > > simple...
> > > >
> > > > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > > > not-MSI(-X)?
> > >
> > > Works for me in a PV guest, at least under the second interpretation (I
> > > don't have any useful devices IRQ < 16). This is with your pciback 0.5.
> > >
> > > I see something like what you describe under HVM, although pciback is
> > > not used in that case. xc_physdev_map_pirq is used but not with
> > > the /sys/.../bdf node AFAICT.
> > >
> > > More digging required I guess...
> >
> > If I just pass through the EHCI device it doesn't work. However if I
> > also pass-through the associated UHCI controller then I do see the
> > usb-storage device, interrupts and all. The EHCI controller still
> > doesn't work though. Note that with a PV guest passing-through the EHCI
> > controller by itself does work.
> >
> > The issue I'm seeing seems to have more to do with accessing the memory
> > mapped BAR than IRQs though:
>
> I seem to see this with 2.6.32 dom0 as well. This IRQ series of patches
> doesn't appear to make a difference when running 2.6.38-rcX+.
>
> Unfortunately I think the error must be elsewhere :-(
BTW, this is with:
$ git log --pretty=oneline v2.6.38-rc8..HEAD --committer=Ian.Campbell |
grep Merge
00fbbdc4df0cf3419bde8bb0f88d8c8dd26945cb Merge branch 'devel/xen-blkback-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen
41e9c499ac8a4582037d250d5c524265426b5077 Merge branch 'devel/xen-pciback-0.5' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen
aca829e02d62e56498c12410acdd04fd7604eaab Merge branch 'for-2.6.39/irq.phase.two' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen
2365df51ebb8092b4ce07dba55fe1f2bbb3fbdc5 Merge branch 'upstream/dom0/backend/netback' of git://xenbits.xen.org/people/ianc/linux-2.6
3d3e02a231154e7e564a0620dcfd88ceedf8c64a Merge branch '2.6.38-rc6-pvhvm' of git://xenbits.xen.org/people/sstabellini/linux-pvhvm
c6d63750ec3d00da603ea42c9dd1e3454ec12d17 Merge branches 'for-2.6.39/e820', 'for-2.6.39/irq', 'for-2.6.39/cleanup' and 'for-2.6.39/drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen
I also tested a variant where I replaced the devel/xen-pciback-0.5 and
for-2.6.39/irq.phase.two merges with xen-pciback-0.4.driver (rebased
onto 71eef7d1e3d9d instead of c5ae07bb307b658) and dropped blkback. No
difference.
Ian.
On Fri, 2011-03-11 at 14:32 +0000, Ian Campbell wrote:
> On Fri, 2011-03-11 at 14:24 +0000, Ian Campbell wrote:
> > On Fri, 2011-03-11 at 13:28 +0000, Ian Campbell wrote:
> > > On Fri, 2011-03-11 at 11:24 +0000, Ian Campbell wrote:
> > > > On Fri, 2011-03-11 at 09:12 +0000, Ian Campbell wrote:
> > > > > On Thu, 2011-03-10 at 22:57 +0000, Konrad Rzeszutek Wilk wrote:
> > > > > > On Thu, Mar 10, 2011 at 04:07:59PM +0000, Ian Campbell wrote:
> > > > > > > Changes since last time:
> > > > > > > * correct return value of xen_irq_from_pirq
> > > > > > > * WARN if a pirq cannot be allocated for a legacy IRQ
> > > > > > > * Updated checking comment of "xen: events: do not workaround
> > > > > > > too-small nr_irqs"
> > > > > > >
> > > > > > > The following series makes a few cleanups to the Xen IRQ infrastructure.
> > > > > > > The most important thing is that it removes the need to know about
> > > > > > > nr_irqs and in particular the reliance on nr_irqs being static.
> > > > > > >
> > > > > > > Apart from being generally a good thing this is needed because in 2.6.39
> > > > > > > nr_irqs will be able to grow dynamically, specifically e7bcecb7b1d2
> > > > > > > "genirq: Make nr_irqs runtime expandable" from tip/core/irq is targeted
> > > > > > > at 2.6.39.
> > > > > > >
> > > > > > > Dynamically growing nr_irqs also allows us to remove the workaround
> > > > > > > which eats into GSI space if a dynamic IRQ cannot be allocated.
> > > > > > >
> > > > > > > There is no ideal sequencing of this series vs e7bcecb7b1d2 (most should
> > > > > > > have gone in before, but the penultimate patch really needed to be
> > > > > > > simultaneous) so I haven't bothered to try and pull anything from tip
> > > > > > > into this branch -- it should all be resolved during the merge window
> > > > > > > and bisection won't be too broken since the "eat into GSI space"
> > > > > > > workaround only appears to be needed on a small number of older
> > > > > > > platforms (qemu being the main exception).
> > > > > >
> > > > > > <nods>
> > > > > > >
> > > > > > > I have tested:
> > > > > > > * Domain 0 on real h/w and under qemu
> > > > > > > * PV guest, including migration and passthrough of both VF and PF.
> > > > > > > * PVHVM guest, including migration and passthrough of both VF and
> > > > > > > PF.
> > > > > >
> > > > > > I am having difficulties with passthrough of an USB device. Somehow the
> > > > > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > > > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > > > > of your changes and apply cleanly. The whole lot is now in #master
> > > > > >
> > > > > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > > > > it is something special with the legacy IRQs. Probably forgot something
> > > > > > simple...
> > > > >
> > > > > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > > > > not-MSI(-X)?
> > > >
> > > > Works for me in a PV guest, at least under the second interpretation (I
> > > > don't have any useful devices IRQ < 16). This is with your pciback 0.5.
> > > >
> > > > I see something like what you describe under HVM, although pciback is
> > > > not used in that case. xc_physdev_map_pirq is used but not with
> > > > the /sys/.../bdf node AFAICT.
> > > >
> > > > More digging required I guess...
> > >
> > > If I just pass through the EHCI device it doesn't work. However if I
> > > also pass-through the associated UHCI controller then I do see the
> > > usb-storage device, interrupts and all. The EHCI controller still
> > > doesn't work though. Note that with a PV guest passing-through the EHCI
> > > controller by itself does work.
> > >
> > > The issue I'm seeing seems to have more to do with accessing the memory
> > > mapped BAR than IRQs though:
> >
> > I seem to see this with 2.6.32 dom0 as well. This IRQ series of patches
> > doesn't appear to make a difference when running 2.6.38-rcX+.
> >
> > Unfortunately I think the error must be elsewhere :-(
>
> BTW, this is with:
BTW2, compared with your merged up branches I get:
$ git log --no-merges --pretty=oneline HEAD..konrad/linux-next
$
(i.e. nothing)
the inverse (konrad/linux-next..HEAD) looks like blkback+pciback
+stefano's branch and nothing else.
$ git log --no-merges --pretty=oneline HEAD...konrad/master
ac55b353aba741aded22e1cbd4a9b59e999b9c89 xen: update mask_rw_pte after kernel page tables init changes
ff5959a1215b9e6d39b86520d62e7d6fc124bfb3 xen: set max_pfn_mapped to the last pfn mapped
09a741fee2cdd4ffa2cf926b70083a6f63073700 x86: Cleanup highmap after brk is concluded
733301920082553b52ce4453493fe6abf6aa7d1a ttm: Pass in 'struct device' to TTM so it can do DMA API on behalf of device.
02bbfbab7dd6a107ea2f5d6e882631cd31c72eda ttm: Include the 'struct dev' when using the DMA API.
7c20cf7644dc632b61a6f7ee52fa4ae2f8abc1b2 nouveau/ttm/PCIe: Use dma_addr if TTM has set it.
eadf243146fbcd88ead082be5341996e52c13b73 radeon/ttm/PCIe: Use dma_addr if TTM has set it.
6b1f175951de66ac07e765a07d4f22004c0d7ce2 ttm: Expand (*populate) to support an array of DMA addresses.
0438748313f46e887aca12ef0cfc4178f2dfe920 ttm: Utilize the DMA API for pages that have TTM_PAGE_FLAG_DMA32 set.
36ff9d246ac231fe6bce55121ddd6f799c9b0c3b ttm: Introduce a placeholder for DMA (bus) addresses.
2bd3f2608788e776ff0524ce77cca995094191a1 drm/radeon/kms: clean up gart dummy page handling
and
$ git log --no-merges --pretty=oneline konrad/master..HEAD
$
(again nothing).
No smoking guns there AFAICT...
On Fri, 2011-03-11 at 14:29 +0000, Konrad Rzeszutek Wilk wrote:
> > > > > I am having difficulties with passthrough of an USB device. Somehow the
> > > > > irq count is not going up at all (both in dom0 and domU) and it looks to
> > > > > be doing just simple polling. I've rebased the xen-pciback to be on top
> > > > > of your changes and apply cleanly. The whole lot is now in #master
> > > > >
> > > > > MSI and MSI-X devices work just fine in both Dom0 and DomU case so
> > > > > it is something special with the legacy IRQs. Probably forgot something
> > > > > simple...
> > > >
> > > > Thanks, I'll take a look. Did you mean "legacy IRQ" as in <16 or as in
> > > > not-MSI(-X)?
>
> not-MSI(-X) ones. I've this fancy LSI 1030 controller that sits at GSI 46 on this
> SuperMicro board and it is quite confused.
> > >
> > > Works for me in a PV guest, at least under the second interpretation (I
> > > don't have any useful devices IRQ < 16). This is with your pciback 0.5.
>
> OK. Let me boot up that machine and see what /proc/interrupts look like. Maybe
> the xen-pciback.hide=() throws a lot of things out of whack.
FWIW I'm using xen-pciback.hide too.
> > >
> > > I see something like what you describe under HVM, although pciback is
> > > not used in that case. xc_physdev_map_pirq is used but not with
> > > the /sys/.../bdf node AFAICT.
> > >
> > > More digging required I guess...
> >
> > If I just pass through the EHCI device it doesn't work. However if I
> > also pass-through the associated UHCI controller then I do see the
> > usb-storage device, interrupts and all. The EHCI controller still
> > doesn't work though. Note that with a PV guest passing-through the EHCI
> > controller by itself does work.
> >
> > The issue I'm seeing seems to have more to do with accessing the memory
> > mapped BAR than IRQs though:
>
> >
> > ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> > PHYSDEVOP_map_pirq mapped gsi 46 to pirq 18
> > xen_allocate_pirq_gsi for gsi 46, pirq 18 "ioapic-level"
> > xen: --> irq=46, pirq=18
> > ehci_hcd 0000:00:07.0: PCI INT C -> GSI 46 (level, low) -> IRQ 46
> > ehci: remapped rsrc 0xf3021000 to c8064000
>
> <scratch> If you hit the 'i' in the debug console is this
> BAR visible there? That is f3021?
'i' is irq information not BARs, but FWIW the IRQ associated with this
H/W appears to be mapped correctly.
I couldn't find another debug key which looked like it contained the
sort of thing you meant.
Ian.