2001-11-09 21:45:17

by Chris Friesen

[permalink] [raw]
Subject: how is processor cache coherency maintained for device drivers


I'm trying to help track down some infrequent and difficult to reproduce pci bus
parity errors that we're seeing on a cPCI card, and one of the things that has
been suggested is that it may have something to do with DMA coherency between
devices and the processor.

Can someone point me to the proper code/information that deals with how the
processor knows that the memory corresponding to the ethernet device is no
longer up-to-date? Is it somehow marked as non-cacheable, or is it snooped,
explicitly flushed, or what?

The platform in question is a Motorola MCPN765 card, with a PPC7400 processor,
running a modified 2.2.17 kernel.

Thanks,

Chris


--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]


2001-11-11 16:56:20

by James Bottomley

[permalink] [raw]
Subject: Re: how is processor cache coherency maintained for device drivers

> I'm trying to help track down some infrequent and difficult to
> reproduce pci bus parity errors that we're seeing on a cPCI card, and
> one of the things that has been suggested is that it may have
> something to do with DMA coherency between devices and the processor.

> Can someone point me to the proper code/information that deals with
> how the processor knows that the memory corresponding to the ethernet
> device is no longer up-to-date? Is it somehow marked as
> non-cacheable, or is it snooped, explicitly flushed, or what?

This is extremely architecture specific. Some, like the x86, have a fully
coherent architecture, so the caches snoop the bus for invalidations. Others
(older parisc) are completely incoherent and won't invalidate the CPU cache
unless explicitly told to do so.

The code for all this is in:

asm/io.h for the basic dma_cache_* functions

asm/pci.h for the pci bus pci_dma_* functions

See Documentation/DMA-mapping.txt for a partial explanation of the
implementation.

> The platform in question is a Motorola MCPN765 card, with a PPC7400
> processor, running a modified 2.2.17 kernel.

I believe (although you should take better advice from the PPC people) that
the powerpc is completely cache coherent, so any coherency problems you may be
having would be due to the hardware, not the kernel.

James Bottomley


2001-11-12 21:59:44

by Mike Fedyk

[permalink] [raw]
Subject: Re: how is processor cache coherency maintained for device drivers

On Sun, Nov 11, 2001 at 08:55:52AM -0800, James Bottomley wrote:
> I believe (although you should take better advice from the PPC people) that
> the powerpc is completely cache coherent, so any coherency problems you may be
> having would be due to the hardware, not the kernel.
>

IIRC (probably wrong though) the PPC is not coherent for one of its caches,
but is for another. PPC people would definately know the details, I'm just
thinking of a thread I read on this a few weeks ago...

Mike