otherwise Xen is _completely_ unusable with 5 or more VCPUs.
( when !CONFIG_HAVE_SPARSE_IRQ)
based on Alex's patch
also add +1 offset after redir_entries
Signed-off-by: Yinghai Lu <[email protected]>
Cc: Alex Nixon <[email protected]>
Index: linux-2.6/arch/x86/kernel/io_apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/io_apic.c
+++ linux-2.6/arch/x86/kernel/io_apic.c
@@ -3625,16 +3625,21 @@ int __init probe_nr_irqs(void)
{
int idx;
int nr = 0;
+#ifndef CONFIG_XEN
+ int nr_min = 32;
+#else
+ int nr_min = NR_IRQS;
+#endif
for (idx = 0; idx < nr_ioapics; idx++)
- nr += io_apic_get_redir_entries(idx);
+ nr += io_apic_get_redir_entries(idx) + 1;
/* double it for hotplug and msi and nmi */
nr <<= 1;
/* something wrong ? */
- if (nr < 32)
- nr = 32;
+ if (nr < nr_min)
+ nr = nr_min;
return nr;
}
Yinghai Lu wrote:
> otherwise Xen is _completely_ unusable with 5 or more VCPUs.
> ( when !CONFIG_HAVE_SPARSE_IRQ)
>
> based on Alex's patch
>
> also add +1 offset after redir_entries
>
> Signed-off-by: Yinghai Lu <[email protected]>
> Cc: Alex Nixon <[email protected]>
>
Looks good to me. We can fix this in a better way later.
Acked-by: Jeremy Fitzhardinge <[email protected]>
> Index: linux-2.6/arch/x86/kernel/io_apic.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/kernel/io_apic.c
> +++ linux-2.6/arch/x86/kernel/io_apic.c
> @@ -3625,16 +3625,21 @@ int __init probe_nr_irqs(void)
> {
> int idx;
> int nr = 0;
> +#ifndef CONFIG_XEN
> + int nr_min = 32;
> +#else
> + int nr_min = NR_IRQS;
> +#endif
>
> for (idx = 0; idx < nr_ioapics; idx++)
> - nr += io_apic_get_redir_entries(idx);
> + nr += io_apic_get_redir_entries(idx) + 1;
>
> /* double it for hotplug and msi and nmi */
> nr <<= 1;
>
> /* something wrong ? */
> - if (nr < 32)
> - nr = 32;
> + if (nr < nr_min)
> + nr = nr_min;
>
> return nr;
> }
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Thu, Aug 21, 2008 at 4:33 PM, Jeremy Fitzhardinge <[email protected]> wrote:
> Yinghai Lu wrote:
>> otherwise Xen is _completely_ unusable with 5 or more VCPUs.
>> ( when !CONFIG_HAVE_SPARSE_IRQ)
>>
>> based on Alex's patch
>>
>> also add +1 offset after redir_entries
>>
>> Signed-off-by: Yinghai Lu <[email protected]>
>> Cc: Alex Nixon <[email protected]>
>>
>
> Looks good to me. We can fix this in a better way later.
later could set CONFIG_HAVE_SPARSE_IRQ to true by default.
YH
Yinghai Lu wrote:
> On Thu, Aug 21, 2008 at 4:33 PM, Jeremy Fitzhardinge <[email protected]> wrote:
>
>> Yinghai Lu wrote:
>>
>>> otherwise Xen is _completely_ unusable with 5 or more VCPUs.
>>> ( when !CONFIG_HAVE_SPARSE_IRQ)
>>>
>>> based on Alex's patch
>>>
>>> also add +1 offset after redir_entries
>>>
>>> Signed-off-by: Yinghai Lu <[email protected]>
>>> Cc: Alex Nixon <[email protected]>
>>>
>>>
>> Looks good to me. We can fix this in a better way later.
>>
>
> later could set CONFIG_HAVE_SPARSE_IRQ to true by default.
>
Well, perhaps. But we can use vectors rather than IRQs for Xen event
channel mappings, which will save us from having to allocate lots of
per-cpu irqs.
Also, implementing probe_nr_irqs() in a more generic (rather than
io_apic-specific) way. Using some interface that queries each irq chip
for how many irqs it supports, or something.
J
On Thu, Aug 21, 2008 at 4:52 PM, Jeremy Fitzhardinge <[email protected]> wrote:
> Yinghai Lu wrote:
>> On Thu, Aug 21, 2008 at 4:33 PM, Jeremy Fitzhardinge <[email protected]> wrote:
>>
>>> Yinghai Lu wrote:
>>>
>>>> otherwise Xen is _completely_ unusable with 5 or more VCPUs.
>>>> ( when !CONFIG_HAVE_SPARSE_IRQ)
>>>>
>>>> based on Alex's patch
>>>>
>>>> also add +1 offset after redir_entries
>>>>
>>>> Signed-off-by: Yinghai Lu <[email protected]>
>>>> Cc: Alex Nixon <[email protected]>
>>>>
>>>>
>>> Looks good to me. We can fix this in a better way later.
>>>
>>
>> later could set CONFIG_HAVE_SPARSE_IRQ to true by default.
>>
>
> Well, perhaps. But we can use vectors rather than IRQs for Xen event
> channel mappings, which will save us from having to allocate lots of
> per-cpu irqs.
like system vectors with smp-ipi etc?
>
> Also, implementing probe_nr_irqs() in a more generic (rather than
> io_apic-specific) way. Using some interface that queries each irq chip
> for how many irqs it supports, or something.
hope we can kill nr_irqs/NR_IRQS
YH
Yinghai Lu wrote:
> like system vectors with smp-ipi etc?
>
More or less. Xen currently uses 6 per-cpu event channels, and it maps
each to its own irq. My plan is to allocate just 6 irqs, map 6 vectors
to them, and bind each event channel to a vector/cpu pair. If nothing
else, it will make /proc/interrupts somewhat sane again.
>> Also, implementing probe_nr_irqs() in a more generic (rather than
>> io_apic-specific) way. Using some interface that queries each irq chip
>> for how many irqs it supports, or something.
>>
>
> hope we can kill nr_irqs/NR_IRQS
>
When irqs are truely dynamically allocated?
J
On Thu, Aug 21, 2008 at 5:02 PM, Jeremy Fitzhardinge <[email protected]> wrote:
> Yinghai Lu wrote:
>> like system vectors with smp-ipi etc?
>>
>
> More or less. Xen currently uses 6 per-cpu event channels, and it maps
> each to its own irq. My plan is to allocate just 6 irqs, map 6 vectors
> to them, and bind each event channel to a vector/cpu pair. If nothing
> else, it will make /proc/interrupts somewhat sane again.
>
>
>>> Also, implementing probe_nr_irqs() in a more generic (rather than
>>> io_apic-specific) way. Using some interface that queries each irq chip
>>> for how many irqs it supports, or something.
>>>
>>
>> hope we can kill nr_irqs/NR_IRQS
>>
>
> When irqs are truely dynamically allocated?
>
need more test with sparse_irq
YH
* Jeremy Fitzhardinge <[email protected]> wrote:
> > hope we can kill nr_irqs/NR_IRQS
>
> When irqs are truely dynamically allocated?
they in essence already are truly dynamic in tip/master. Not yet
unconditionally so on x86, we chose to also have a !SPARSEIRQ build
model to make sure dense IRQ arrays still work fine as well. (that's far
easier to test and validate than to keep a farm of cross compilers to
other arches alive.)
But by all means SPARSEIRQ is the primary and only model on x86.
We could even plug in the dynamic vector allocator from Alan Mayer now,
both on 32-bit and on 64-bit x86: Yinghai, any complications expected
there? I guess we need to unify the local APIC code first.
Ingo
* Jeremy Fitzhardinge <[email protected]> wrote:
> Yinghai Lu wrote:
> > otherwise Xen is _completely_ unusable with 5 or more VCPUs.
> > ( when !CONFIG_HAVE_SPARSE_IRQ)
> >
> > based on Alex's patch
> >
> > also add +1 offset after redir_entries
> >
> > Signed-off-by: Yinghai Lu <[email protected]>
> > Cc: Alex Nixon <[email protected]>
> >
>
> Looks good to me. We can fix this in a better way later.
>
> Acked-by: Jeremy Fitzhardinge <[email protected]>
applied to tip/irq/sparseirq.
Alex, does this approach work for you too?
Ingo
On Thu, Aug 21, 2008 at 9:26 PM, Ingo Molnar <[email protected]> wrote:
>
> * Jeremy Fitzhardinge <[email protected]> wrote:
>
>> > hope we can kill nr_irqs/NR_IRQS
>>
>> When irqs are truely dynamically allocated?
>
> they in essence already are truly dynamic in tip/master. Not yet
> unconditionally so on x86, we chose to also have a !SPARSEIRQ build
> model to make sure dense IRQ arrays still work fine as well. (that's far
> easier to test and validate than to keep a farm of cross compilers to
> other arches alive.)
>
> But by all means SPARSEIRQ is the primary and only model on x86.
>
> We could even plug in the dynamic vector allocator from Alan Mayer now,
> both on 32-bit and on 64-bit x86: Yinghai, any complications expected
> there? I guess we need to unify the local APIC code first.
apic_32.c and apic_64.c
irqinit_32.c and irqinit_64.c
irq_32.c and irq_64.c
Cyrill, is still working on merging apic_32.c apic_64.c.?
YH
* Yinghai Lu <[email protected]> wrote:
> > But by all means SPARSEIRQ is the primary and only model on x86.
> >
> > We could even plug in the dynamic vector allocator from Alan Mayer
> > now, both on 32-bit and on 64-bit x86: Yinghai, any complications
> > expected there? I guess we need to unify the local APIC code first.
>
> apic_32.c and apic_64.c
> irqinit_32.c and irqinit_64.c
> irq_32.c and irq_64.c
>
> Cyrill, is still working on merging apic_32.c apic_64.c.?
there's a few weeks of inactivity expected in that area i think - so if
you'd like to pick it up from there, feel free ...
but apic_32/64.c is looking already pretty good in tip/master:
1 file changed, 391 insertions(+), 358 deletions(-)
the most significant bit of that delta seems to be around x2apic,
calibration differences and some older 32-bit legacies which are not on
the 64-bit side (but which we can add safely).
Ingo
Ingo Molnar wrote:
> * Jeremy Fitzhardinge <[email protected]> wrote:
>
>> Yinghai Lu wrote:
>>> otherwise Xen is _completely_ unusable with 5 or more VCPUs.
>>> ( when !CONFIG_HAVE_SPARSE_IRQ)
>>>
>>> based on Alex's patch
>>>
>>> also add +1 offset after redir_entries
>>>
>>> Signed-off-by: Yinghai Lu <[email protected]>
>>> Cc: Alex Nixon <[email protected]>
>>>
>> Looks good to me. We can fix this in a better way later.
>>
>> Acked-by: Jeremy Fitzhardinge <[email protected]>
>
> applied to tip/irq/sparseirq.
>
> Alex, does this approach work for you too?
>
> Ingo
Yep.
- Alex
[Ingo Molnar - Fri, Aug 22, 2008 at 07:16:56AM +0200]
|
| * Yinghai Lu <[email protected]> wrote:
|
| > > But by all means SPARSEIRQ is the primary and only model on x86.
| > >
| > > We could even plug in the dynamic vector allocator from Alan Mayer
| > > now, both on 32-bit and on 64-bit x86: Yinghai, any complications
| > > expected there? I guess we need to unify the local APIC code first.
| >
| > apic_32.c and apic_64.c
| > irqinit_32.c and irqinit_64.c
| > irq_32.c and irq_64.c
| >
| > Cyrill, is still working on merging apic_32.c apic_64.c.?
|
| there's a few weeks of inactivity expected in that area i think - so if
| you'd like to pick it up from there, feel free ...
|
| but apic_32/64.c is looking already pretty good in tip/master:
|
| 1 file changed, 391 insertions(+), 358 deletions(-)
|
| the most significant bit of that delta seems to be around x2apic,
| calibration differences and some older 32-bit legacies which are not on
| the 64-bit side (but which we can add safely).
|
| Ingo
|
Yes - I will be busy for a few weeks with other duties, sorry for
that. There not that much diff rest:
- apic clock calibration
- x2apic operations
- interrupt handlers (different declarations)
- command line handling
ugh... maybe not that small heap of todo's :)
Anyway - before sending patches I'll fetch tip/x86/apic
branch so no problem if someone bring any changes into
the branch.
- Cyrill -