> #ifdef OUT_OF_LINE_MMIO
> #define res_readb(res, adr) (res->access_ops->readb(res, adr)
> #else
> #define res_readb(res, adr) readb(adr)
> #endif
I think the second #define should be:
#define res_readb(res, adr) readb(res->start+adr)
for consistency.
David
[email protected] said:
> I think the second #define should be:
> #define res_readb(res, adr) readb(res->start+adr)
> for consistency.
You're right that it should be consistent. But it doesn't really matter
whether we pass an offset within the resource, or whether we continue to
pass the full 'bus address'. The driver doesn't even need to care - it just
adds the register offset to whatever opaque cookie it's given as the address
of that resource anyway.
That's really an orthogonal issue. The _important_ bit is that we pass the
resource to the I/O functions, so that in the case where they're
out-of-line, they don't need to play silly buggers with the numbers they're
given just to work out which bus they should be talking to.
--
dwmw2
> [email protected] said:
> > I think the second #define should be:
> > #define res_readb(res, adr) readb(res->start+adr)
> > for consistency.
>
> You're right that it should be consistent. But it doesn't really matter
> whether we pass an offset within the resource, or whether we continue to
The question I think being ignored here is. Why not leave things as is. The
multiple bus stuff is a port specific detail hidden behind readb() and friends.
On the HP PA32 its already hiding controller number encodings and generating
multiple cycles under spinlocks for PCI I/O space and the devices dont know
about it
[email protected] said:
> The question I think being ignored here is. Why not leave things as
> is.
Because if we just pass in this one extra piece of information which is
normally already available in the driver, we can avoid a whole lot of ugly
cruft in the out-of-line functions by plugging in the correct out-of-line
function to match the resource.
> The multiple bus stuff is a port specific detail hidden behind
> readb() and friends.
The alternative view is that the _single_ bus stuff is a port-specific
detail which has permeated all the drivers and forced the non-i386
architectures' I/O functions to have to try to work out which bus they're
talking to when the driver could have just passed that information to them.
--
dwmw2
> Because if we just pass in this one extra piece of information which is
> normally already available in the driver, we can avoid a whole lot of ugly
> cruft in the out-of-line functions by plugging in the correct out-of-line
> function to match the resource.
Case 1:
You pass a single cookie to the readb code
Odd platforms decode it
Case 2:
You carry around bus number information all throughout
each driver
You keep putting it on/off the stack
You keep it in structures
You do complex generic locking for hotplug 'just in case'
I think I prefer case 1.
On Mon, Jul 02, 2001 at 05:56:56PM +0100, Alan Cox wrote:
> Case 1:
> You pass a single cookie to the readb code
> Odd platforms decode it
Last time I checked, ioremap didn't work for inb() and outb().
--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html
Russell King wrote:
>
> On Mon, Jul 02, 2001 at 05:56:56PM +0100, Alan Cox wrote:
> > Case 1:
> > You pass a single cookie to the readb code
> > Odd platforms decode it
>
> Last time I checked, ioremap didn't work for inb() and outb().
It should :)
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
> > > You pass a single cookie to the readb code
> > > Odd platforms decode it
> >
> > Last time I checked, ioremap didn't work for inb() and outb().
>
> It should :)
it doesnt need to.
pci_find_device returns the io address and can return a cookie, ditto
isapnp etc
>Last time I checked, ioremap didn't work for inb() and outb().
ioremap itself cannot work for inb/outb as they are different
address spaces with potentially overlapping addresses, I don't
see how a single function would handle both... except if we
pass it a struct resource instead of the address.
Ben.
>> > Last time I checked, ioremap didn't work for inb() and outb().
>>
>> It should :)
>
>it doesnt need to.
>
>pci_find_device returns the io address and can return a cookie, ditto
>isapnp etc
Yes, but doing that require 2 annoying things:
- Parsing of this cookie on each inx/outx access, which can
take a bit of time (typically looking up the host bridge)
- On machines with PIO mapped in CPU mem space and several
(or large) IO regions, they must all be mapped all the time,
which is a waste of kernel virtual space.
Why not, at least for 2.5, define a kind of pioremap that
would be the equivalent of ioremap for PIO ?
In fact, I'd rather have all this abstracted in a
ioremap_resource(struct resource *, int flags)
iounmap_resource(struct resource *)
("flags" is just an idea that could be used to pass things
like specific caching attributes, or whatever makes sense to
a given arch).
The distinction between inx/oux & readx/writex would still
make sense at least for x86.
Ben.
> - Parsing of this cookie on each inx/outx access, which can
> take a bit of time (typically looking up the host bridge)
It depends on the implementation obviously, but its typically something like
take lock
writew(port&0xFFFF, port&0xFFFF0000);
writew(data, port&0xFFFF0000+1);
drop lock
Assuming you can drop the bridges on 64K boundaries in pci mem space, or
one extra deref and a register load if not.
Can you give me an idea of what sort of cookie decoding a PPC/PMac would need
and why - Im working off things like pa-risc so I dont have a full picture.
>
>Can you give me an idea of what sort of cookie decoding a PPC/PMac would need
>and why - Im working off things like pa-risc so I dont have a full picture.
Each domain provide an IO space (size depends on the bridge, recent Apple
UniNorth hosts have 16Mb per domain).
That IO space can be in any location (depends on the box, bridge config,
..), so basically, we must assume that each host bridge can have it's IO
space anywhere in CPU mem space.
Currently, we store the physical address of those in our pci_controller
structure, and ioremap all of them. One is picked up as the "ISA" io base
(for VGA and such things as legacy devices on non-pmac PPCs). That
isa_io_base is used as an offset to inx/outx, and all PCI IO_RESOURCES
are fixed up to be their real virtual address offset'ed with isa_io_base.
(A bit weird but works and we have only an addition in inx/outx).
I'm more concerned about having all that space mapped permanently in
kernel virtual space. I'd prefer mapping on-demand, and that would
require a specific ioremap for IOs.
Ben.
Alan Cox wrote:
>
> > > > You pass a single cookie to the readb code
> > > > Odd platforms decode it
> > >
> > > Last time I checked, ioremap didn't work for inb() and outb().
> >
> > It should :)
>
> it doesnt need to.
>
> pci_find_device returns the io address and can return a cookie, ditto
> isapnp etc
Is the idea here to mitigate the amount of driver code changes, or
something else?
If you are sticking a cookie in there behind the scenes, why go ahead
and use ioremap?
We -already- have a system which does remapping and returns cookies and
such for PCI mem regions. Why not use it for I/O regions too?
Jeff
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
> The question I think being ignored here is. Why not leave things as is. The
> multiple bus stuff is a port specific detail hidden behind readb() and
> friends.
This isn't so much for the case where the address generation is done by a
simple addition. That could be optimised away by the compiler with an entirely
inline function (as per David Woodhouse's suggestion).
It's far more important for non-x86 platforms which only have a single address
space and have to fold multiple external address spaces into it.
For example, one board I've got doesn't allow you to do a straight
memory-mapped I/O access to your PCI device directly, but have to reposition a
window in the CPU's memory space over part of the PCI memory space first, and
then hold a spinlock whilst you do it.
David
David Howells wrote:
> For example, one board I've got doesn't allow you to do a straight
> memory-mapped I/O access to your PCI device directly, but have to reposition a
> window in the CPU's memory space over part of the PCI memory space first, and
> then hold a spinlock whilst you do it.
Yuck. Does that wind up making MMIO slower than PIO, on this board?
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
> Case 1:
> You pass a single cookie to the readb code
> Odd platforms decode it
As opposed to passing a cookie (struct resource) and an offset, and letting
the compiler do the addition it'd do anyway or eliminate the cookie directly
on platforms where this is suitable.
> Case 2:
> You carry around bus number information all throughout
> each driver
Eh? Who said anything about bus number info? Just the information in the
resource structure.
> You keep putting it on/off the stack
Why should I want to do that? You've got to keep the base address of your
resource space somewhere anyway, so you could just replace it with a pointer
to the resource struct (which you've already got). Plus, I can pass this in a
register to any behind the scenes function.
In my example code, in the really simple cases (most of them), there were no
pushes and pops.
> You keep it in structures
Doesn't everyone? Apart from those that use global variables, I suppose, but
surely they're limited in reusability.
> You do complex generic locking for hotplug 'just in case'
Eh? No I wasn't, but under some circumstances one might have to do that
anyway, and so the out-of-line functions may be the best place to do that.
David
Jeff Garzik <[email protected]> wrote:
> Russell King wrote:
> >
> > On Mon, Jul 02, 2001 at 05:56:56PM +0100, Alan Cox wrote:
> > > Case 1:
> > > You pass a single cookie to the readb code
> > > Odd platforms decode it
> >
> > Last time I checked, ioremap didn't work for inb() and outb().
>
> It should :)
Surely it shouldn't... ioremap() is for mapping "memory-mapped I/O" resources
into the kernel's virtual memory scheme (at least on the i386 arch). There's
no way to tell the CPU/MMU that a particular pages should assert the IO access
pin rather than memory access pin (or however it is done externally).
David
David Howells wrote:
>
> Jeff Garzik <[email protected]> wrote:
> > Russell King wrote:
> > >
> > > On Mon, Jul 02, 2001 at 05:56:56PM +0100, Alan Cox wrote:
> > > > Case 1:
> > > > You pass a single cookie to the readb code
> > > > Odd platforms decode it
> > >
> > > Last time I checked, ioremap didn't work for inb() and outb().
> >
> > It should :)
>
> Surely it shouldn't... ioremap() is for mapping "memory-mapped I/O" resources
> into the kernel's virtual memory scheme (at least on the i386 arch). There's
> no way to tell the CPU/MMU that a particular pages should assert the IO access
> pin rather than memory access pin (or however it is done externally).
The "at least on the i386 arch" part is the key caveat. On PPC AFAIK,
PIO is remapped and treated very similarly to MMIO. ioremap on x86, for
PIO, could probably be a no-op, simply returning the same address it was
given. For other arches which want to do more complex mappings, ioremap
is IMHO the perfect part of the API for the job.
Basically I don't understand the following train of thought:
* We needed to remap MMIO, therefore ioremap was created.
* Now, we need to remap PIO too [on some arches]. Let's hide the
remapping in arch-specific code.
That's an understandable train of thought from an
implement-it-now-in-2.4 standpoint, but not from a
2.5-design-something-better standpoint.
Jeff
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
I also point out that using ioremap for PIO adds flexibility while
keeping most drivers relatively unchanged. Everyone uses a base address
anyway, so whether its obtained directly (address from PCI BAR) or
indirectly (via ioremap), you already store it and use it.
Further, code lacking ioremap for PIO (100% of PIO code, at present)
does not require a flag day. Drivers can be transitioned as foreign
arches start supporting ioremap for PIO... if ioremap is no-op on x86,
drivers continue to work on x86 before and after the update. Assuming a
stored not hardcoded base address (common case), the only change to a
driver is in probe and remove, nowhere else.
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
> I also point out that using ioremap for PIO adds flexibility while
> keeping most drivers relatively unchanged. Everyone uses a base address
> anyway, so whether its obtained directly (address from PCI BAR) or
> indirectly (via ioremap), you already store it and use it.
I see what you're getting at at last:-) I didn't quite pick up on the fact
that you'd still have to go through readb/writeb and their ilk to access the
code.
Of course, however, this still requires cookie decoding to be done in readb
and writeb (even on the i386). So why not use resource struct?
David
David Howells wrote:
> Of course, however, this still requires cookie decoding to be done in readb
> and writeb (even on the i386). So why not use resource struct?
IMHO that makes the operation too heavyweight on architectures where
that level of abstraction is not needed.
--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner
> For example, one board I've got doesn't allow you to do a straight
> memory-mapped I/O access to your PCI device directly, but have to reposition a
> window in the CPU's memory space over part of the PCI memory space first, and
> then hold a spinlock whilst you do it.
What does this prove. PA-RISC has this problem in reverse for I/O cycle access
to PCI slots on hppa1.1 at least. Cookies work _fine_
And by the time you are taking a spinlock who cares about the add, you can do
that while the bus transactions for the atomic op are completing
On the other hand each call, each push of resource * pointers costs real clocks
on x86
> I'm more concerned about having all that space mapped permanently in
> kernel virtual space. I'd prefer mapping on-demand, and that would
> require a specific ioremap for IOs.
I have no problem with the idea of a function to indicate which I/O maps you
are and are not using. But passing resource structs around is way too heavy
Alan
>> I'm more concerned about having all that space mapped permanently in
>> kernel virtual space. I'd prefer mapping on-demand, and that would
>> require a specific ioremap for IOs.
>
>I have no problem with the idea of a function to indicate which I/O maps you
>are and are not using. But passing resource structs around is way too heavy
Too heavy for inx/outx, I agree, but why too heavy to ioremap ? That would
make a clean abstract implementation, with a semantic like "prepare this
resource for use by the driver". A kind of generic ioremap for IOs, resources,
and whatever another bus type may want to define, returning a token that is
to be passed to readx/writex in all cases but PIO, where it's passed to
inx/outx. That souds to me like the most flexible mecanism, and the small
bit of overhead of passing the resource pointer is done _once_, usually
at driver init.
Something like
iomap_resource(struct resource *);
iounmap_resource(struct resource *);
Eventually, we can have it more fine grained in case where the driver don't
need the entire resource (maybe useful for framebuffers exporting very large
double-endian apertures where only one half is needed).
iomap_resource(struct resource *, unsigned long offset, unsigned long
size);
iounmap_resource(struct resource *, unsigned long offset, unsigned
long size);
The implementation would just call ioremap/iounmap for memory type resources,
and the identity for IO resources on x86. Other archs can then play whatever
tricks, like placing cookies in there.
One thing I have in mind here is the ability for things like embedded that
can have weird bus types, to have additional flags in the resources taken
into account by iomap/unmap to locate the proper bus, or build the proper
cookie that will be used by inx/outx, or define some access attributes
depending on other resource flags (write combine ?).
Ben.
On Tue, 3 Jul 2001, David Howells wrote:
> > The question I think being ignored here is. Why not leave things as is. The
> > multiple bus stuff is a port specific detail hidden behind readb() and
> > friends.
>
> This isn't so much for the case where the address generation is done by a
> simple addition. That could be optimised away by the compiler with an entirely
> inline function (as per David Woodhouse's suggestion).
>
> It's far more important for non-x86 platforms which only have a single address
> space and have to fold multiple external address spaces into it.
>
> For example, one board I've got doesn't allow you to do a straight
> memory-mapped I/O access to your PCI device directly, but have to reposition a
> window in the CPU's memory space over part of the PCI memory space first, and
> then hold a spinlock whilst you do it.
This is a common practice on NEC PCI host bridges: usually you have 2 `windows'
to the PCI bus only, so you can have direct access to only two of PCI memory,
PCI I/O and PCI config spaces at the same time. If you need access to the
third, you have to reconfigure the windows. Usually you configure the windows
to have direct access to PCI memory and PCI I/O spaces. So PCI config space
takes the hit. If you have only one window, YMMV.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds