Hello,
I'm writing a driver for a custom pci card with an FPGA
(Xilinx Spartan 2 (XC2S150-6) with PCI 32 LogiCore),
which can act as a pci bus master. The device is designed
to do DMA transfers with high bandwidth. One task is to
send image data to a printer which already works quite
well, but sometimes there are randomly occuring
problems concerning the timing between two DMA
transfers. The issue seems to be something like interrupt
latency in hardware. Measuring some signals with an
oscilloscope shows, that the delay from generating the
interrupt, which signals a finished transfer, to the time
when the interrupt register on the card is reset (i.e. the
beginning of the ISR) sometimes increases to more
than 500 microseconds, which is dimensions too high.
I already tried with other hardware deactivated, which
could cause traffic on the pci bus or generate many interrupts
(except hard disk). I also increased the priority of the IRQ
used by the pci board (with a tool called irqtune) to the
maximum possible value. Another consideration is, that
another driver could lock all interrupts for too long (but for
500 us??).
As my experience on DMA stuff is not yet too
great, I would be very glad if somebody could give me some
advice to solve this problem. Below there is some further
information about my environment and how I set up the DMA
transfers.
Kind regards,
Burkhard Sch?lpen
Here is how I set up dma transfers from RAM to the pci device:
while (down_interruptible(my_device->write_semaphore));
my_device->dma_write_complete = 0;
my_device->dma_direction = PCI_DMA_TODEVICE;
writel (cpu_to_le32 (virt_to_phys(dma_buffer)), MY_DMA_ADDR_REGISTER);
writel (cpu_to_le32 (my_device->dma_size/4), MY_DMA_COUNT_REGISTER); //triggers dma transfer
if (wait_event_interruptible(write_wait_queue, my_device->dma_write_complete))
{
//handle error...
}
//test, if MY_DMA_COUNT_REGISTER contains 0
up(my_device->write_semaphore);
Inside the Interrupt-handler I do the following:
my_device->dma_write_complete = 1;
wake_up_interruptible(&write_wait_queue);
return IRQ_HANDLED;
Here is some information about the PC:
- Gigabyte GA-8I945GMF mainboard with Pentium D processor
- custom pci board with Xilinx FPGA Spartan 2 (XC2S150-6) with PCI 32 LogiCore
- Debian Linux with 2.6.13.4 SMP kernel
______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS!
Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193
Burkhard Sch?lpen wrote:
> The issue seems to be something like interrupt
> latency in hardware. Measuring some signals with an
> oscilloscope shows, that the delay from generating the
> interrupt, which signals a finished transfer, to the time
> when the interrupt register on the card is reset (i.e. the
> beginning of the ISR) sometimes increases to more
> than 500 microseconds, which is dimensions too high.
>
> ... Another consideration is, that
> another driver could lock all interrupts for too long (but for
> 500 us??).
I also have implemented bus master devices based
on the Spartan 2 + Xilinx PCI core and wrote the
associated Linux driver.
My observations of interrupt latency using
a similar setup to what you describe showed
50usec is common but rare events on
the order of milliseconds are possible.
You are probably correct that the
problem as a poorly behaved driver.
If you have complete control over every system
your device is installed in, you can
find and eliminate any device that
causes high interrupt latency.
If you don't have that control, your hardware
needs to tolerate interrupt latency of
that magnitude.
You describe an implementation with a single
DMA buffer. You could switch to multiple
buffers (ring structure or 2 alternating buffers)
with an interrupt triggered after each
buffer is exhausted (I use a ring of buffers).
The remaining buffers allow the transfer to
continue while waiting for ISR execution.
Or, more crudely, trigger the interrupt when
the single buffer reaches some low water mark
and poll in the ISR for actual completion.
--
Paul Fulghum
Microgate Systems, Ltd.
Burkhard Sch?lpen wrote:
> Hello,
>
> I'm writing a driver for a custom pci card with an FPGA
> (Xilinx Spartan 2 (XC2S150-6) with PCI 32 LogiCore),
> which can act as a pci bus master. The device is designed
> to do DMA transfers with high bandwidth. One task is to
> send image data to a printer which already works quite
> well, but sometimes there are randomly occuring
> problems concerning the timing between two DMA
> transfers. The issue seems to be something like interrupt
> latency in hardware. Measuring some signals with an
> oscilloscope shows, that the delay from generating the
> interrupt, which signals a finished transfer, to the time
> when the interrupt register on the card is reset (i.e. the
> beginning of the ISR) sometimes increases to more
> than 500 microseconds, which is dimensions too high.
Most likely some driver is disabling interrupts for that period, which
is really longer than it should be. However, if your card/driver require
such tight interrupt latency to function correctly, that seems too
fragile and may not be reliable. Some kind of ringbuffer arrangement
would likely work better, so that the interrupt does not have to be
serviced so soon.
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
On Tue, 2006-01-10 at 17:59 -0600, Robert Hancock wrote:
> Most likely some driver is disabling interrupts for that period, which
> is really longer than it should be. However, if your card/driver require
> such tight interrupt latency to function correctly, that seems too
> fragile and may not be reliable. Some kind of ringbuffer arrangement
> would likely work better, so that the interrupt does not have to be
> serviced so soon.
>
You can easily tell if this is the case by applying these patches:
http://people.redhat.com/mingo/latency-tracing-patches/latency-tracing-patches-2.6.15-rc7.tar.gz
It says -rc7 but they apply cleanly and work with 2.6.15 final.
Lee