Maciej W. Rozycki wrote:
> On Sat, 28 Jul 2007, Russell King wrote:
>
>> Essentially, any complex interrupt handler (such as an IDE interrupt
>> doing a multi-sector PIO transfer _in interrupt context_) can cause this
>> kind of starvation. That's why Linux 1.x had bottom halves - so that
>> the time consuming work could be moved out of the interrupt handler,
>> thereby causing minimal the blockage of other interrupts.
>>
>> Unfortunately, that kind of design has been long since forgotten.
>> Apparantly modern machines are fast enough that it doesn't have to be
>> worried about anymore... Or are they?
>
> I would guess it is not that the machines are fast enough, but that this
> two-level processing makes things more complicated. Enough that most
> people would not bother digging into it unless really forced. Only
> occasional latency problems are probably not enough of a force.
I don't believe the speed of the machine has much to do with it,
as IDE PIO is always at pretty much the same speed (or slower)
regardless of the CPU speed.
Best case is about .120 usec per 16-bit word, but that doesn't often pan out
in practice. More typical is something closer to 1 usec per 16-bit word.
So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491 usec,
plus extra overhead for reading the IDE status register (another usec or so),
and other stuff. Figure maybe 500usec total per interrupt for multcount=16
in the best case, or 4000usec in the worst case.
At 115200bps, we get a byte every 86 usec or so. Assuming the UART FIFO
is set to interrupt (warn) us at 12/16 full, we have 4*86 = 344 usec to
respond and de-assert RTS. Less than that in practice.
Conclusion: using IDE multisector PIO is not a good idea with high speed
serial transfers happening, since we cannot respond quickly enough.
It might be possible to set the buffer underrun threshold lower in the UART (?).
All that said, I doubt that his system is using IDE PIO in the first place.
Dunno how long IDE DMA interrupts take, but it's probably in the 20-50 usec range.
Mark Lord wrote:
> I don't believe the speed of the machine has much to do with it,
> as IDE PIO is always at pretty much the same speed (or slower)
> regardless of the CPU speed.
>
> Best case is about .120 usec per 16-bit word, but that doesn't often pan
> out
> in practice. More typical is something closer to 1 usec per 16-bit word.
>
> So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491
> usec,
> plus extra overhead for reading the IDE status register (another usec or
> so),
> and other stuff. Figure maybe 500usec total per interrupt for multcount=16
> in the best case, or 4000usec in the worst case.
>
> At 115200bps, we get a byte every 86 usec or so. Assuming the UART FIFO
> is set to interrupt (warn) us at 12/16 full, we have 4*86 = 344 usec to
> respond and de-assert RTS. Less than that in practice.
>
> Conclusion: using IDE multisector PIO is not a good idea with high speed
> serial transfers happening, since we cannot respond quickly enough.
>
> It might be possible to set the buffer underrun threshold lower in the
> UART (?).
>
> All that said, I doubt that his system is using IDE PIO in the first place.
> Dunno how long IDE DMA interrupts take, but it's probably in the 20-50
> usec range.
I think that PIO transfers only have to be done with interrupts disabled
on really old, evil controllers (without unmask set). I don't think
libata ever disables interrupts during transfers(?)
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
Robert Hancock wrote:
> Mark Lord wrote:
>> I don't believe the speed of the machine has much to do with it,
>> as IDE PIO is always at pretty much the same speed (or slower)
>> regardless of the CPU speed.
>>
>> Best case is about .120 usec per 16-bit word, but that doesn't often
>> pan out
>> in practice. More typical is something closer to 1 usec per 16-bit word.
>>
>> So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491
>> usec,
>> plus extra overhead for reading the IDE status register (another usec
>> or so),
>> and other stuff. Figure maybe 500usec total per interrupt for
>> multcount=16
>> in the best case, or 4000usec in the worst case.
>>
>> At 115200bps, we get a byte every 86 usec or so. Assuming the UART FIFO
>> is set to interrupt (warn) us at 12/16 full, we have 4*86 = 344 usec to
>> respond and de-assert RTS. Less than that in practice.
>>
>> Conclusion: using IDE multisector PIO is not a good idea with high speed
>> serial transfers happening, since we cannot respond quickly enough.
>>
>> It might be possible to set the buffer underrun threshold lower in the
>> UART (?).
>>
>> All that said, I doubt that his system is using IDE PIO in the first
>> place.
>> Dunno how long IDE DMA interrupts take, but it's probably in the 20-50
>> usec range.
>
> I think that PIO transfers only have to be done with interrupts disabled
> on really old, evil controllers (without unmask set). I don't think
> libata ever disables interrupts during transfers(?)
That's what "hdparm -u1" (or -u0) controls.
But it doesn't matter a whit here. The problem is that the IDE interrupt
handling can take a long time, regardless of whether it unmasks IRQs or not.
And if that IDE interrupt interrupts a serial interrupt, then the serial
stuff won't get handled until the IDE stuff completes. Thus the problem.
The "fix" could be to have the serial IRQ handler never unmask interrupts,
but that's a bit unsociable to others. The IDE stuff really needs to not
do so much during the actual IRQ handler.
Ingo's RT patches would probably fix all of this.
Mark Lord wrote:
>> I think that PIO transfers only have to be done with interrupts
>> disabled on really old, evil controllers (without unmask set). I don't
>> think libata ever disables interrupts during transfers(?)
>
> That's what "hdparm -u1" (or -u0) controls.
>
> But it doesn't matter a whit here. The problem is that the IDE interrupt
> handling can take a long time, regardless of whether it unmasks IRQs or
> not.
> And if that IDE interrupt interrupts a serial interrupt, then the serial
> stuff won't get handled until the IDE stuff completes. Thus the problem.
>
> The "fix" could be to have the serial IRQ handler never unmask interrupts,
> but that's a bit unsociable to others. The IDE stuff really needs to not
> do so much during the actual IRQ handler.
>
> Ingo's RT patches would probably fix all of this.
libata also doesn't do the actual PIO transfer from the interrupt
handler like old IDE does, either, and it only disables interrupts for
the transfer if it's transferring to/from high memory..
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
> I think that PIO transfers only have to be done with interrupts disabled
> on really old, evil controllers (without unmask set). I don't think
> libata ever disables interrupts during transfers(?)
Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
some work on that but its actually very hard to fix without doing polled
PIO.
Alan Cox wrote:
>> I think that PIO transfers only have to be done with interrupts disabled
>> on really old, evil controllers (without unmask set). I don't think
>> libata ever disables interrupts during transfers(?)
>
> Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
> some work on that but its actually very hard to fix without doing polled
> PIO.
Ah, right. Misread the code.
> That's what "hdparm -u1" (or -u0) controls.
Only some of the time.
> Ingo's RT patches would probably fix all of this.
The worst case IDE times we've seen for executing a single indivisible
un-interruptible I/O cycle with a drive are around 1mS. Thats a hardware
limit.
Alan
On Thu, 2 Aug 2007, Alan Cox wrote:
> Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
> some work on that but its actually very hard to fix without doing polled
> PIO.
Hmm, when the drive signals it is ready for a PIO data transfer can't
just the interrupt handler mask the originating interrupt and post a
softirq to handle the case? That should be rather straightforward.
Maciej
Mark Lord wrote:
> The "fix" could be to have the serial IRQ handler never unmask
> interrupts,
> but that's a bit unsociable to others. The IDE stuff really needs to not
> do so much during the actual IRQ handler.
>
> Ingo's RT patches would probably fix all of this.
I did a Fedora 7 installation and installed Ingo's kernel from here:
http://people.redhat.com/mingo/realtime-preempt/yum-testing/yum/i686/kernel-rt-2.6.21-0182.rt11cfsv17.i686.rpm
Even then, the problem still occurs, unfortunately.
Thanks,
Lee.