>>>>> "David" == David S Miller <[email protected]> writes:
David> Jes Sorensen writes:
>> The dma_mask in struct pci_dev tells you whether you are DAC
>> capable. We pass a pointer to this struct when we call the pci_*
>> functions so the required information needed to make the decision
>> whether to return a SAC or a DAC address is already available.
David> The decision is not based upon "device capable of DAC", that is
David> precisely my point.
I understand that, it's part of the equation.
David> The decision must be based upon a number of considerations.
David> 1) Can it do DAC
David> 2) Is DAC more efficient than SAC on this platform
That sounds to me like a 'static' decision at compile time or at least
something decided upon in the pci_* code as it will be the same for
all devices on the bus. If your IOMMU is very complex to program
compared to the overhead of DAC cycles you pick DAC etc.
David> 3) Does the devices _need_ DAC even if it is slower because it
David> requires referencing large portions of the DMA address space
David> simultaneously
Sure or because the IOMMU is starved. Most devices will do SAC/DAC
based on the address you throw at them, ie. it's perfectly valid to
mix and match DAC and SAC addresses handed to a device.
David> Sure, you could imply all of this complexity in the driver by
David> making them consider all of these issues when setting the mask,
David> but that isn't a nice interface at all.
David> And this still leaves the 64-bit dma_addr_t overhead issue.
The 64 bit dma_addr_t is only an issue on 32 bit architectures with
highmem enabled. I never suggested making dma_addr_t 64 bit on 32 bit
architectures as a general thing.
Jes
Jes Sorensen writes:
> David> And this still leaves the 64-bit dma_addr_t overhead issue.
>
> The 64 bit dma_addr_t is only an issue on 32 bit architectures with
> highmem enabled. I never suggested making dma_addr_t 64 bit on 32 bit
> architectures as a general thing.
What about for drivers of SAC-only devices, they eat the overhead
when highmem is enabled too?
This says nothing about the real reason the IA64 solution is
unacceptable, the inputs to the mapping functions which must
be "page+offset+len" triplets as there is no logical "virtual
address" to pass into the mapping routines on 32-bit systems.
Face it, the ia64 stuff is not what we can use across the board. It
simply doesn't deal with all the necessary issues. Therefore,
encouraging driver author's to use this ia64 hacked up scheme is
not such a hot idea until we have a real API implemented.
Later,
David S. Miller
[email protected]
On Fri, 6 Jul 2001, David S. Miller wrote:
> What about for drivers of SAC-only devices, they eat the overhead
> when highmem is enabled too?
Yes. It's not an unreasonable overhead considering that it's configured
out for all the non-highmem kernels that will be shipped. Keep in mind
that the expected lifespan for 32 bit systems is now less than 3 years, so
elaborate planning that delays implementation buys us nothing more than a
smaller window of usefulness.
> This says nothing about the real reason the IA64 solution is
> unacceptable, the inputs to the mapping functions which must
> be "page+offset+len" triplets as there is no logical "virtual
> address" to pass into the mapping routines on 32-bit systems.
On x86 a 64 bit DMA address cookie is fine. If you've got concerns, tell
us what you have in mind for a design.
> Face it, the ia64 stuff is not what we can use across the board. It
> simply doesn't deal with all the necessary issues. Therefore,
> encouraging driver author's to use this ia64 hacked up scheme is
> not such a hot idea until we have a real API implemented.
So what's the API?
-ben
Ben LaHaise writes:
> Yes. It's not an unreasonable overhead considering that it's configured
> out for all the non-highmem kernels that will be shipped. Keep in mind
> that the expected lifespan for 32 bit systems is now less than 3 years, so
> elaborate planning that delays implementation buys us nothing more than a
> smaller window of usefulness.
Maybe by then only 64-bit cpus will matter. Who knows.
> > This says nothing about the real reason the IA64 solution is
> > unacceptable, the inputs to the mapping functions which must
> > be "page+offset+len" triplets as there is no logical "virtual
> > address" to pass into the mapping routines on 32-bit systems.
>
> On x86 a 64 bit DMA address cookie is fine. If you've got concerns, tell
> us what you have in mind for a design.
Things along the lines of what Jens Axboe's patches are what I'm
thinking about.
> So what's the API?
See Jens's patches.
First it has to pass in page/off/len triplets, on all platforms.
This is addressed by Jens's interfaces.
Secondly it has to provide a query mechanism to delineate the
three cases:
1) DAC is faster and always preferred
2) SAC is faster
3) DAC may be slower but more desirable for certain devices
due to large amounts of parallel address space usage
I have not designed an interface for this, but it ought to be
quite simple.
Thirdly seperate 32-bit/64-bit DMA address types. Added to the
overhead concerns, I also think it sucks big donkey balls to cast the
things around, especially since different platforms would potentially
require different casts to eliminate the warnings. In fact, with your
suggested scheme, the setting of highmem would determine a core
type.
In fact, I'm not going to bother to code one single bit of this myself
until I am convinced I have thought the whole problem over properly.
This is the part nobody else wants to do, but it is a prerequisite for
this sort of API.
Later,
David S. Miller
[email protected]
> > that the expected lifespan for 32 bit systems is now less than 3 years, so
> > elaborate planning that delays implementation buys us nothing more than a
> > smaller window of usefulness.
> Maybe by then only 64-bit cpus will matter. Who knows.
Reality check.
Embedded PCI 32bit processors are going to be very common
People are only now retiring 486's
So add another seven or eight years to your estimate
Alan Cox wrote:
>
> > > that the expected lifespan for 32 bit systems is now less than 3 years, so
> > > elaborate planning that delays implementation buys us nothing more than a
> > > smaller window of usefulness.
> > Maybe by then only 64-bit cpus will matter. Who knows.
>
> Reality check.
>
> Embedded PCI 32bit processors are going to be very common
> People are only now retiring 486's
>
> So add another seven or eight years to your estimate
Given a little more context, I thought we were talking specifically
about 64bit-PCI-on-32bit-machines?
Assuming that, AFAICS Ben's statement seems more correct.
And IMHO we definitely should not optimize for 64-bit-on-32-bit case.
Let CONFIG_HIGHMEM grow dma_addr_t to 64-bits, for that case only...
--
Jeff Garzik | A recent study has shown that too much soup
Building 1024 | can cause malaise in laboratory mice.
MandrakeSoft |
> > So add another seven or eight years to your estimate
>
> Given a little more context, I thought we were talking specifically
> about 64bit-PCI-on-32bit-machines?
>
> Assuming that, AFAICS Ben's statement seems more correct.
>
> And IMHO we definitely should not optimize for 64-bit-on-32-bit case.
> Let CONFIG_HIGHMEM grow dma_addr_t to 64-bits, for that case only...
I see no good way to optimise for 64bit dma on a 32bit box. The existing API
is extremely clean, easy to understand and I don't agree with Dave's desire
to write another whole concoction.
We do need pci_set_dma_mask_bits(), which in itself lets drivers indicate
SAC/DAC capable. We do want a way for nosey or fine tuned drivers to query
SAC/DAC properties, but most drivers should be letting arch code make arch
policy.
Alan
Alan Cox writes:
> I see no good way to optimise for 64bit dma on a 32bit box.
I'm actually not only talking about DAC device on 32-bit cpus. Just
as much, I'm talking about drivers for SAC-only devices even on 64-bit
cpus.
I took a lot of crap from driver authors when we started pushing the
PCI dma stuff on people, because of the dma_addr_t people now had to
keep around to unmap the thing later.
To a certain extent I agreed with these folks. I'll be gutting myself
if I make everyone eat twice as much space just to add DAC support to
the kernel :-)
>From yet another perspective, my proposals have also to do with what
API can actually work on all platforms. This is pretty important to
me. I remember yesteryear when I used to give myself the privilege
of being self-arch-centric in my work, a Sparc hack here, a Sparc hack
there. But I simply cannot operate this way anymore. My conscious
will no longer allow me to crap up things like that :-)
> I don't agree with Dave's desire to write another whole concoction.
It needs to be a new set of interfaces (and at that point, why not use
a different dma64_addr_t type and save overhead for SAC-only devices
while we're at it :-) because the proper inputs for a DAC mapping
involve page/off/len pairs.
Ignoring addressing limits of 32-bit cpus for a moment, consider that
this page/off/len triplet is the natural currency the kernel uses for
this kind of stuff anyways.
I think it is interesting to note that Jens noticed this immediately,
him being the first person to actually try and implement something
that would work on 32-bit platforms.
Later,
David S. Miller
[email protected]
>>>>> "David" == David S Miller <[email protected]> writes:
David> Alan Cox writes:
>> I see no good way to optimise for 64bit dma on a 32bit box.
David> I'm actually not only talking about DAC device on 32-bit cpus.
David> Just as much, I'm talking about drivers for SAC-only devices
David> even on 64-bit cpus.
David> I took a lot of crap from driver authors when we started
David> pushing the PCI dma stuff on people, because of the dma_addr_t
David> people now had to keep around to unmap the thing later.
David> To a certain extent I agreed with these folks. I'll be gutting
David> myself if I make everyone eat twice as much space just to add
David> DAC support to the kernel :-)
The overhead is going be negligeble, the overhead of highmem itself is
much worse. Not to mention that today some dma_addr_t's might not be
packed properly in data structure hence they ending up taking 8 bytes
anyway.
Jes
On Wed, Jul 11, 2001 at 09:16:50PM +0200, Jes Sorensen wrote:
The overhead is going be negligeble, the overhead of highmem itself is
much worse. Not to mention that today some dma_addr_t's might not be
packed properly in data structure hence they ending up taking 8 bytes
anyway.
What kind of packing makes a 32-bit value take 8-bytes on any
currently supported archicture? The worst-case I can think of is
7-bytes in the case of misaligned by 3 (e.g. __attribute__((packed))
struct blah { char foo[3]; long bar }; sort of thing).
--cw
Jes Sorensen writes:
> The overhead is going be negligeble, the overhead of highmem itself is
> much worse.
Once Jens's block layer stuff goes in, a lot of that overhead simply
disappears since the page cache need not bounce buffers. The
networking can already technically cope with this too.
> Not to mention that today some dma_addr_t's might not be
> packed properly in data structure hence they ending up taking 8 bytes
> anyway.
Not on x86 which is the current main benefactor of highmem.
Later,
David S. Miller
[email protected]
Chris Wedgwood writes:
> What kind of packing makes a 32-bit value take 8-bytes on any
> currently supported archicture? The worst-case I can think of is
> 7-bytes in the case of misaligned by 3 (e.g. __attribute__((packed))
> struct blah { char foo[3]; long bar }; sort of thing).
If you have this:
struct {
u32 foo;
void *bar;
};
"bar" will be at offset 8 on a 64-bit platforms since it must be
aligned on a 64-byte boundary, so what Jes is saying is that for:
struct {
dma_addr_t foo;
void *bar;
};
the "dma_addr_t" is already consuming 8 bytes of space on 64-bit
systems.
Later,
David S. Miller
[email protected]