> So, we still have to wait for the exact explanation...
>
> Thanks very much Marcin!
>
> I think, there is this one possible for your testing yet?:
> Subject: [patch] genirq: temporary fix for level-triggered IRQ resend
> Date: Wed, 8 Aug 2007 13:00:37 +0200
>
> If it's not a great problem it would be interesting to try this with
> different CONFIG_HZ too e.g. you could start with 100 (I guess, you
> tested very similar thing in 2.6.23-rc2 with 1000(?) already).
>
> Jean-Baptiste: you can skip/break testing of this 'experimental'
ok
I was still testing on -rc2:
Subject: [patch] genirq: temporary fix for level-triggered IRQ resend
Date: Wed, 8 Aug 2007 13:00:37 +0200
For me after 1day 20hours, the network is still up, with more than 1To
of network traffic. HZ was 1000, i restart with HZ=100.
Jb
On Fri, Aug 10, 2007 at 10:15:53AM +0200, Jean-Baptiste Vignaud wrote:
...
> I was still testing on -rc2:
> Subject: [patch] genirq: temporary fix for level-triggered IRQ resend
> Date: Wed, 8 Aug 2007 13:00:37 +0200
>
> For me after 1day 20hours, the network is still up, with more than 1To
> of network traffic. HZ was 1000, i restart with HZ=100.
For me it's enough too but Thomas seems to doubt.
You've written earlier that you've 2.6.23-rc1 with HARDIRQS_SW_RESEND
prepared too. So, if this is not a great problem maybe you could try
this first. Tomorrow Thomas may send something, so this 100HZ could
wait yet, I hope?
Many thanks,
Jarek P.
* Jarek Poplawski <[email protected]> wrote:
> On Fri, Aug 10, 2007 at 10:15:53AM +0200, Jean-Baptiste Vignaud wrote:
> ...
> > I was still testing on -rc2:
> > Subject: [patch] genirq: temporary fix for level-triggered IRQ resend
> > Date: Wed, 8 Aug 2007 13:00:37 +0200
> >
> > For me after 1day 20hours, the network is still up, with more than
> > 1To of network traffic. HZ was 1000, i restart with HZ=100.
>
> For me it's enough too but Thomas seems to doubt.
seem to doubt what? That rc2 fixes the symptom? That is a sure thing,
and we never doubted that. I think you might have misunderstood what
Thomas said and meant, so please just state your opinion unambiguously
so that we can fix any mis-communication :)
Ingo
On Fri, Aug 10, 2007 at 10:48:41AM +0200, Ingo Molnar wrote:
>
> * Jarek Poplawski <[email protected]> wrote:
>
> > On Fri, Aug 10, 2007 at 10:15:53AM +0200, Jean-Baptiste Vignaud wrote:
> > ...
> > > I was still testing on -rc2:
> > > Subject: [patch] genirq: temporary fix for level-triggered IRQ resend
> > > Date: Wed, 8 Aug 2007 13:00:37 +0200
> > >
> > > For me after 1day 20hours, the network is still up, with more than
> > > 1To of network traffic. HZ was 1000, i restart with HZ=100.
> >
> > For me it's enough too but Thomas seems to doubt.
>
> seem to doubt what? That rc2 fixes the symptom? That is a sure thing,
> and we never doubted that. I think you might have misunderstood what
> Thomas said and meant, so please just state your opinion unambiguously
> so that we can fix any mis-communication :)
>
> Ingo
>
On 25-07-2007 02:19, Thomas Gleixner wrote:
...
> Actually we only need the resend for edge type interrupts. Level type
> interrupts come back once enable_irq() re-enables the interrupt line.
>
On 10-08-2007 10:05, Thomas Gleixner wrote:
...
> But suppressing the resend is not fixing the driver problem. The problem
> can show up with spurious interrupts and with interrupts on a shared PCI
> interrupt line at any time. It just might take weeks instead of minutes.
Maybe I miss something but it's not the same!
So, should Jean-Baptiste or Marcin test this for weeks or it's enough?
Jarek P.
* Jarek Poplawski <[email protected]> wrote:
> On 10-08-2007 10:05, Thomas Gleixner wrote:
> ...
> > But suppressing the resend is not fixing the driver problem. The
> > problem can show up with spurious interrupts and with interrupts on
> > a shared PCI interrupt line at any time. It just might take weeks
> > instead of minutes.
>
> Maybe I miss something but it's not the same!
_now_ i finally understand what you probably meant: because sw-resend
worked and hw-resend didnt, it's hw-resend that is causing the breakage,
not any driver or irqflow bug - correct?
Ingo
On Fri, Aug 10, 2007 at 11:08:33AM +0200, Ingo Molnar wrote:
>
> * Jarek Poplawski <[email protected]> wrote:
>
> > On 10-08-2007 10:05, Thomas Gleixner wrote:
> > ...
> > > But suppressing the resend is not fixing the driver problem. The
> > > problem can show up with spurious interrupts and with interrupts on
> > > a shared PCI interrupt line at any time. It just might take weeks
> > > instead of minutes.
> >
> > Maybe I miss something but it's not the same!
>
> _now_ i finally understand what you probably meant: because sw-resend
> worked and hw-resend didnt, it's hw-resend that is causing the breakage,
> not any driver or irqflow bug - correct?
All correct! There was also checked a possibility it can be not
hw itself, but wrong way of handling after hw (acking too late). This
was false idea (or bad implementation), so it looks like hw vs lapic
problem.
Jarek P.
* Jarek Poplawski <[email protected]> wrote:
> All correct! There was also checked a possibility it can be not hw
> itself, but wrong way of handling after hw (acking too late). This was
> false idea (or bad implementation), so it looks like hw vs lapic
> problem.
i think the problem is that local APIC 'self vectors' might be
edge-triggered by default. I'm not exactly sure whether passing in
APIC_INT_LEVELTRIG to send_IPI_self() will truly be interpreted by the
local APIC into any external IO-APIC ACK sequence (the local APIC might
just treat self-vectors as always-edge) - and it might also be that the
pure act of mixing self-triggered vectors with level-triggered external
irqs sometimes confuses the IO-APIC <-> local-APIC messaging. One more
test of the patch below will tell us a bit more about this part of the
story.
Ingo
Index: linux/arch/i386/kernel/io_apic.c
===================================================================
--- linux.orig/arch/i386/kernel/io_apic.c
+++ linux/arch/i386/kernel/io_apic.c
@@ -735,7 +735,8 @@ void fastcall send_IPI_self(int vector)
* Wait for idle.
*/
apic_wait_icr_idle();
- cfg = APIC_DM_FIXED | APIC_DEST_SELF | vector | APIC_DEST_LOGICAL;
+ cfg = APIC_DM_FIXED | APIC_DEST_SELF | vector | APIC_DEST_LOGICAL |
+ APIC_INT_LEVELTRIG;
/*
* Send the IPI. The write to APIC_ICR fires this off.
*/