I'm looking at adding dom0 support to the pv-ops kernel. One of the
obvious things we need is to support real device drivers, and the
associated p->m translations for devices.
I'm thinking the cleanest thing to do is make x86-64's dma-mapping.h
with its dma_mapping_ops common to i386 and x86-64, so we can hook the
Xen translations in there. Presumably we'll need to do this anyway to
support VTd for 32-bit (but I don't know if that's a reasonable thing to
do anyway).
What do you think?
J
On Tuesday 26 June 2007 21:59, Jeremy Fitzhardinge wrote:
> I'm looking at adding dom0 support to the pv-ops kernel. One of the
> obvious things we need is to support real device drivers, and the
> associated p->m translations for devices.
>
> I'm thinking the cleanest thing to do is make x86-64's dma-mapping.h
> with its dma_mapping_ops common to i386 and x86-64, so we can hook the
> Xen translations in there. Presumably we'll need to do this anyway to
> support VTd for 32-bit (but I don't know if that's a reasonable thing to
> do anyway).
>
> What do you think?
Ok, if you can do it without ifdefs.
And no swiotlb on i386; that is something that is completely broken
in upstream Xen and needs to be fixed properly anyways.
-Andi
Andi Kleen wrote:
> Ok, if you can do it without ifdefs.
>
That should be OK. All the existing i386 mapping operations would just
have their own ops structure, right?
> And no swiotlb on i386; that is something that is completely broken
> in upstream Xen and needs to be fixed properly anyways.
>
Hm, OK. I'm not really familiar with the issues here. What are they?
Looks like Jan has made a number of Xen-ish changes to lib/swiotlb.c;
are more changes be needed?
J
On Wednesday 27 June 2007 16:15:17 Jeremy Fitzhardinge wrote:
> Andi Kleen wrote:
> > Ok, if you can do it without ifdefs.
> >
>
> That should be OK. All the existing i386 mapping operations would just
> have their own ops structure, right?
I just mention it because many people's ideas of merging files
seem to add lots of ifdefs which is imho the totally wrong thing
to do.
> > And no swiotlb on i386; that is something that is completely broken
> > in upstream Xen and needs to be fixed properly anyways.
> >
>
> Hm, OK. I'm not really familiar with the issues here. What are they?
> Looks like Jan has made a number of Xen-ish changes to lib/swiotlb.c;
> are more changes be needed?
See the recent "quiet down swiotlb warnings" thread which uncovered
quite some corpses in Xen's current IO setup.
Xen apparently bounces for multi page IOs which get merged from block
lists because the block layer doesn't know they are not really
continuous in machine memory.
Proper fix is to tell the block layer to not merge in the first
place instead.
And probably some similar mechanism for network drivers that limits
MTUs.
Maybe we'll still need a simple bouncing mechanism for other obscure
devices with large IOs then, but I would very much prefer if it wasn't
swiotlb and could be solved some other way.
-Andi
Andi Kleen wrote:
> On Wednesday 27 June 2007 16:15:17 Jeremy Fitzhardinge wrote:
>
>> Andi Kleen wrote:
>>
>>> Ok, if you can do it without ifdefs.
>>>
>>>
>> That should be OK. All the existing i386 mapping operations would just
>> have their own ops structure, right?
>>
>
> I just mention it because many people's ideas of merging files
> seem to add lots of ifdefs which is imho the totally wrong thing
> to do.
>
>
>>> And no swiotlb on i386; that is something that is completely broken
>>> in upstream Xen and needs to be fixed properly anyways.
>>>
>>>
>> Hm, OK. I'm not really familiar with the issues here. What are they?
>> Looks like Jan has made a number of Xen-ish changes to lib/swiotlb.c;
>> are more changes be needed?
>>
>
> See the recent "quiet down swiotlb warnings" thread which uncovered
> quite some corpses in Xen's current IO setup.
>
> Xen apparently bounces for multi page IOs which get merged from block
> lists because the block layer doesn't know they are not really
> continuous in machine memory.
>
> Proper fix is to tell the block layer to not merge in the first
> place instead.
>
> And probably some similar mechanism for network drivers that limits
> MTUs.
>
Well, I think there are two issues here. One is that two
pseudo-physical pages won't necessarily be contigious in bus space,
because of the pseudo-phys to machine mapping.
The second problem is that devices which can't address all machine
physical memory (ie, 32-bit PCI devices on machines with >4G memory)
will need to have bouncebuffers established for them. Device drivers
won't necessarily be able to do it because they're not really aware of
machine addresses.
> Maybe we'll still need a simple bouncing mechanism for other obscure
> devices with large IOs then, but I would very much prefer if it wasn't
> swiotlb and could be solved some other way.
>
I think 32-bit-only devices are a bigger concern, no?
J
On Wed, Jun 27, 2007 at 05:26:50PM +0200, Andi Kleen wrote:
> See the recent "quiet down swiotlb warnings" thread which uncovered
> quite some corpses in Xen's current IO setup.
>
> Xen apparently bounces for multi page IOs which get merged from block
> lists because the block layer doesn't know they are not really
> continuous in machine memory.
>
> Proper fix is to tell the block layer to not merge in the first
> place instead.
>
> And probably some similar mechanism for network drivers that limits
> MTUs.
Will that guarantee that block and net IOs will not straddle a page
boundary?
Cheers,
Muli
Muli Ben-Yehuda <[email protected]> wrote:
>
>> And probably some similar mechanism for network drivers that limits
>> MTUs.
>
> Will that guarantee that block and net IOs will not straddle a page
> boundary?
Mostly. There is the thorny case of slab debugging that breaks
these nice assumptions.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt