On 10 Jan, Peter Antoniac wrote:
[...]
> Problem is: how to get the VMALLOC_RESERVED value for the kernel that is
> running? I couldn't find any standard way to do that (something to apply to
> GNU Linux and the like). All the things I could get were the default value
> being 128MiB :) and that is it. Now, I could just put 128, but what if
> somebody changes that, or in some new distro suddenly decides to make it
> different? Even worse, what if it is an old kernel with 64 setting?
[...]
Maybe somebody at LKML has answers?
--
Stefan Richter
-=====-=-=== ---= -===-
http://arcgraph.de/sr/
On Sun, 2007-01-14 at 20:19 +0100, Stefan Richter wrote:
> On 10 Jan, Peter Antoniac wrote:
> [...]
> > Problem is: how to get the VMALLOC_RESERVED value for the kernel that is
> > running? I couldn't find any standard way to do that (something to apply to
> > GNU Linux and the like). All the things I could get were the default value
> > being 128MiB :) and that is it. Now, I could just put 128, but what if
> > somebody changes that, or in some new distro suddenly decides to make it
> > different? Even worse, what if it is an old kernel with 64 setting?
> [...]
>
> Maybe somebody at LKML has answers?
vmalloc space is limited; you really can't assume you can get any more
than 64Mb or so (and even then it's thight on some systems already); it
really sounds like vmalloc space isn't the right solution for your
problem whatever it is (context is lost in the quoted mail)...
can you restate the problem to see if there's a better solution
possible?
Greetings,
Arjan van de Ven
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On 14 Jan, Arjan van de Ven wrote:
> vmalloc space is limited; you really can't assume you can get any more
> than 64Mb or so (and even then it's thight on some systems already);
I suppose "grep VmallocChunk /proc/meminfo" shows what is available?
> it really sounds like vmalloc space isn't the right solution for your
> problem whatever it is (context is lost in the quoted mail)...
> can you restate the problem to see if there's a better solution
> possible?
Thanks. Below is Peter's message to linux1394-devel. The previous
discussion went over libdc1394-devel which I don't receive. Obviously he
wants a really large buffer for reception of an isochronous stream. I
guess his reason is highly application specific...
| Hi all,
|
| I've been trying to get a resolution to the problem with vmalloc error (the
| <<allocation failed: out of vmalloc space - use vmalloc=<size> to increase
| size.>> kernel error message thing). The plan how to resolve the issue is
| simple; get the buffer size that we try to allocate (vmmap.nb_buffers *
| vmmap.buf_size) and compare it to the VMALLOC_RESERVED. If too big, error
| with explanation how to fix it. If small, other error (usual out of mem).
| Problem is: how to get the VMALLOC_RESERVED value for the kernel that is
| running? I couldn't find any standard way to do that (something to apply to
| GNU Linux and the like). All the things I could get were the default value
| being 128MiB :) and that is it. Now, I could just put 128, but what if
| somebody changes that, or in some new distro suddenly decides to make it
| different? Even worse, what if it is an old kernel with 64 setting?
|
| Currently, in the SVN version, Damien was kind to change so that a message
| gets printed with a full explanation of how to treat it. Still, this is a
| compromise solution and not the elegant one that I was looking for. I believe
| and hope that maybe somebody had this issue before and could help with some
| suggestions...
|
| So, my question is: anybody knows the way to get to the kernel value like
| VMALLOC_RESERVED or something around this area (a function like getpagesize
| or sysconf)? It will do a great deal to solve the problematic error treatment
| in the library...
|
| Thank you.
|
| For your reference, this is in response to this line of thinking or the
| libdc1394-devel thread:
| [...]
| > > When I set NUM_BUFFERS (number of DMA buffers) to a value greater than 5
| > > the program dies like this:
| > >
| > > (dc1394_capture.c) VIDEO1394_IOC_LISTEN_CHANNEL ioctl failed!
| > > Libdc1394 error (dc1394_capture.c:dc1394_capture_setup_dma:382): Capture
| > > is not set : Could not setup DMA capture
| [...]
| > > [17723533.496000] video1394_0: Iso receive DMA: 8 buffers of size
| > > 6627328 allocated for a frame size 6624000, each with 1619 prgs
| > > [17723533.516000] video1394_0: iso context 0 listen on channel 1
| > > [17723533.712000] ieee1394: Node [1-01:1023] wants to release broadcast
| > > channel 31. ?Ignoring.
| > > [17723534.448000] video1394_1: mask: 0000000000000004 usage:
| > > 0000000000000000
| > > [17723534.448000]
| > > [17723534.508000] video1394_1: Iso receive DMA: 8 buffers of size
| > > 6627328 allocated for a frame size 6624000, each with 1619 prgs
| > > [17723534.532000] video1394_1: iso context 0 listen on channel 2
| > > [17723534.728000] ieee1394: Node [2-01:1023] wants to release broadcast
| > > channel 31. ?Ignoring.
| > > [17723535.464000] video1394_2: mask: 0000000000000008 usage:
| > > 0000000000000000
| > > [17723535.464000]
| > > [17723535.464000] printk: 11 messages suppressed.
| > > [17723535.464000] allocation failed: out of vmalloc space - use
| > > vmalloc=<size> to increase size.
| > > [17723535.464000] dma_region_alloc: vmalloc_32() failed
| > > [17723535.464000] video1394_2: Failed to allocate dma buffer
| > > [17723535.464000] video1394_2: Couldn't allocate ir context
| > > [17723535.668000] video1394_0: On release: Iso receive context 0 stop
| > > listening on channel 1
| > > [17723535.676000] video1394_1: On release: Iso receive context 0 stop
| > > listening on channel 2
| [...]
| > ------------------------------
| >
| > Message: 2
| > Date: Fri, 05 Jan 2007 17:30:39 +0100
| > From: Martin Peris <xxxxxxxxxxxxxxxxxxx>
| > Subject: Re: [libdc1394-devel] [SPAM RBL] Re:
| > ????????VIDEO1394_IOC_LISTEN_CHANNEL????ioctl failed! and Bad images
| > To: Damien Douxchamps <xxxxxxxxxxxxxxxxxxxxx>
| > Cc: libdc1394-devel <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
| [...]
| > I think I have some answers...
| >
| > I've been investigating a bit, and the ?problem with the limit in the
| > size of the allocated DMA buffer ?is not so obscure.
| >
| > vmalloc_32() allocate virtually contiguous memory (32bit addressable),
| > the default maximum amount of memory reserved for this depends on each
| > kernel compilation, and in my case it was set to 128Mbytes that's why I
| > had an error if tried to allocate too many buffers.
| >
| > ?but at boot time you can specify how much virtually contiguous memory
| > you want with the parameter vmalloc, so if you want about 512Mbytes of
| > memory for vmalloc you should add the parameter vmalloc=536870912 to the
| > line that defines the kernel parameters. (If you use grub there should
| > be a line like this on /boot/grub/menu.lst)
| >
| > kernel ? ? ? ? ?/boot/vmlinuz-2.6.15-27-686 root=/dev/sda2
| > vmalloc=536870912 ro quiet splash
| >
| >
| > That killed my problem with:
| >
| > dc1394_capture.c) VIDEO1394_IOC_LISTEN_CHANNEL ioctl failed!
| > Libdc1394 error (dc1394_capture.c:dc1394_capture_setup_dma:382): Capture
| > is not set : Could not setup DMA capture
| [...]
--
Stefan Richter
-=====-=-=== ---= -===-
http://arcgraph.de/sr/
On Sun, 2007-01-14 at 21:31 +0100, Stefan Richter wrote:
> On 14 Jan, Arjan van de Ven wrote:
> > vmalloc space is limited; you really can't assume you can get any more
> > than 64Mb or so (and even then it's thight on some systems already);
>
> I suppose "grep VmallocChunk /proc/meminfo" shows what is available?
>
> > it really sounds like vmalloc space isn't the right solution for your
> > problem whatever it is (context is lost in the quoted mail)...
> > can you restate the problem to see if there's a better solution
> > possible?
>
> Thanks. Below is Peter's message to linux1394-devel. The previous
> discussion went over libdc1394-devel which I don't receive. Obviously he
> wants a really large buffer for reception of an isochronous stream. I
> guess his reason is highly application specific...
but why does that even use vmalloc? You can just do a scatter gather
thing instead... and keep a list of pages that you're mapping into
userspace. vmalloc isn't really a requirement for that....
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Monday 15 January 2007 05:31, Stefan Richter wrote:
> On 14 Jan, Arjan van de Ven wrote:
> > vmalloc space is limited; you really can't assume you can get any more
> > than 64Mb or so (and even then it's thight on some systems already);
>
> I suppose "grep VmallocChunk /proc/meminfo" shows what is available?
This is more the answer that I expect. But is there a way, function or
constant from libdc that can give you the answer, or you have to get it from
the /proc/meminfo? Now, if you can only get it from the /proc/meminfo, is
this info always there? As far as I remember, you can compile kernels without
the /proc/meminfo...
And yes, probably we would have to get rid of this issue and look for an
alternative way. Still, from a technical point of view, I am really curious
if this information on the size of VmallocChunk is available to the developer
in other forms...
Thank you all!
Cheers,
Peter
> > it really sounds like vmalloc space isn't the right solution for your
> > problem whatever it is (context is lost in the quoted mail)...
> > can you restate the problem to see if there's a better solution
> > possible?
>
> Thanks. Below is Peter's message to linux1394-devel. The previous
> discussion went over libdc1394-devel which I don't receive. Obviously he
> wants a really large buffer for reception of an isochronous stream. I
> guess his reason is highly application specific...
>
> | Hi all,
> |
> | I've been trying to get a resolution to the problem with vmalloc error
> | (the <<allocation failed: out of vmalloc space - use vmalloc=<size> to
> | increase size.>> kernel error message thing). The plan how to resolve the
> | issue is simple; get the buffer size that we try to allocate
> | (vmmap.nb_buffers * vmmap.buf_size) and compare it to the
> | VMALLOC_RESERVED. If too big, error with explanation how to fix it. If
> | small, other error (usual out of mem). Problem is: how to get the
> | VMALLOC_RESERVED value for the kernel that is running? I couldn't find
> | any standard way to do that (something to apply to GNU Linux and the
> | like). All the things I could get were the default value being 128MiB :)
> | and that is it. Now, I could just put 128, but what if somebody changes
> | that, or in some new distro suddenly decides to make it different? Even
> | worse, what if it is an old kernel with 64 setting?
> |
> | Currently, in the SVN version, Damien was kind to change so that a
> | message gets printed with a full explanation of how to treat it. Still,
> | this is a compromise solution and not the elegant one that I was looking
> | for. I believe and hope that maybe somebody had this issue before and
> | could help with some suggestions...
> |
> | So, my question is: anybody knows the way to get to the kernel value like
> | VMALLOC_RESERVED or something around this area (a function like
> | getpagesize or sysconf)? It will do a great deal to solve the problematic
> | error treatment in the library...
> |
> | Thank you.
> |
> | For your reference, this is in response to this line of thinking or the
> | libdc1394-devel thread:
> | [...]
> |
> | > > When I set NUM_BUFFERS (number of DMA buffers) to a value greater
> | > > than 5 the program dies like this:
> | > >
> | > > (dc1394_capture.c) VIDEO1394_IOC_LISTEN_CHANNEL ioctl failed!
> | > > Libdc1394 error (dc1394_capture.c:dc1394_capture_setup_dma:382):
> | > > Capture is not set : Could not setup DMA capture
> |
> | [...]
> |
> | > > [17723533.496000] video1394_0: Iso receive DMA: 8 buffers of size
> | > > 6627328 allocated for a frame size 6624000, each with 1619 prgs
> | > > [17723533.516000] video1394_0: iso context 0 listen on channel 1
> | > > [17723533.712000] ieee1394: Node [1-01:1023] wants to release
> | > > broadcast channel 31. ?Ignoring.
> | > > [17723534.448000] video1394_1: mask: 0000000000000004 usage:
> | > > 0000000000000000
> | > > [17723534.448000]
> | > > [17723534.508000] video1394_1: Iso receive DMA: 8 buffers of size
> | > > 6627328 allocated for a frame size 6624000, each with 1619 prgs
> | > > [17723534.532000] video1394_1: iso context 0 listen on channel 2
> | > > [17723534.728000] ieee1394: Node [2-01:1023] wants to release
> | > > broadcast channel 31. ?Ignoring.
> | > > [17723535.464000] video1394_2: mask: 0000000000000008 usage:
> | > > 0000000000000000
> | > > [17723535.464000]
> | > > [17723535.464000] printk: 11 messages suppressed.
> | > > [17723535.464000] allocation failed: out of vmalloc space - use
> | > > vmalloc=<size> to increase size.
> | > > [17723535.464000] dma_region_alloc: vmalloc_32() failed
> | > > [17723535.464000] video1394_2: Failed to allocate dma buffer
> | > > [17723535.464000] video1394_2: Couldn't allocate ir context
> | > > [17723535.668000] video1394_0: On release: Iso receive context 0 stop
> | > > listening on channel 1
> | > > [17723535.676000] video1394_1: On release: Iso receive context 0 stop
> | > > listening on channel 2
> |
> | [...]
> |
> | > ------------------------------
> | >
> | > Message: 2
> | > Date: Fri, 05 Jan 2007 17:30:39 +0100
> | > From: Martin Peris <xxxxxxxxxxxxxxxxxxx>
> | > Subject: Re: [libdc1394-devel] [SPAM RBL] Re:
> | > ????????VIDEO1394_IOC_LISTEN_CHANNEL????ioctl failed! and Bad images
> | > To: Damien Douxchamps <xxxxxxxxxxxxxxxxxxxxx>
> | > Cc: libdc1394-devel <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
> |
> | [...]
> |
> | > I think I have some answers...
> | >
> | > I've been investigating a bit, and the ?problem with the limit in the
> | > size of the allocated DMA buffer ?is not so obscure.
> | >
> | > vmalloc_32() allocate virtually contiguous memory (32bit addressable),
> | > the default maximum amount of memory reserved for this depends on each
> | > kernel compilation, and in my case it was set to 128Mbytes that's why I
> | > had an error if tried to allocate too many buffers.
> | >
> | > ?but at boot time you can specify how much virtually contiguous memory
> | > you want with the parameter vmalloc, so if you want about 512Mbytes of
> | > memory for vmalloc you should add the parameter vmalloc=536870912 to
> | > the line that defines the kernel parameters. (If you use grub there
> | > should be a line like this on /boot/grub/menu.lst)
> | >
> | > kernel ? ? ? ? ?/boot/vmlinuz-2.6.15-27-686 root=/dev/sda2
> | > vmalloc=536870912 ro quiet splash
> | >
> | >
> | > That killed my problem with:
> | >
> | > dc1394_capture.c) VIDEO1394_IOC_LISTEN_CHANNEL ioctl failed!
> | > Libdc1394 error (dc1394_capture.c:dc1394_capture_setup_dma:382):
> | > Capture is not set : Could not setup DMA capture
> |
> | [...]
On Monday 15 January 2007 13:14, Peter Antoniac wrote:
> This is more the answer that I expect. But is there a way, function or
> constant from **libdc** that can give you the answer, or you have to get it
...
What was I thinking... not from libdc but from LIBC :)
Arjan van de Ven wrote:
> On Sun, 2007-01-14 at 20:19 +0100, Stefan Richter wrote:
>> On 10 Jan, Peter Antoniac wrote:
>> [...]
>>> Problem is: how to get the VMALLOC_RESERVED value for the kernel that is
>>> running? I couldn't find any standard way to do that (something to apply to
>>> GNU Linux and the like). All the things I could get were the default value
>>> being 128MiB :) and that is it. Now, I could just put 128, but what if
>>> somebody changes that, or in some new distro suddenly decides to make it
>>> different? Even worse, what if it is an old kernel with 64 setting?
>> [...]
>>
>> Maybe somebody at LKML has answers?
>
> vmalloc space is limited; you really can't assume you can get any more
> than 64Mb or so (and even then it's thight on some systems already); it
> really sounds like vmalloc space isn't the right solution for your
> problem whatever it is (context is lost in the quoted mail)...
> can you restate the problem to see if there's a better solution
> possible?
>
I've used vmalloc in the past, and not had a problem, but it is a fair
question, how do you find out how much space is available? Other than a
binary vmalloc/release loop.
--
bill davidsen <[email protected]>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
> >
> I've used vmalloc in the past, and not had a problem, but it is a fair
> question, how do you find out how much space is available? Other than a
> binary vmalloc/release loop.
that's a really hard question since it's dynamic;
I suppose a good rule of thumb is "never expect more than 16Mb to work"
(when I said "64Mb" that was for the kernel as a whole, and there's
other users than just "you" :)
if you need that much you probably should redesign your algorithms to
not need vmalloc in the first place....
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Mon, 2007-01-15 at 10:20 -0800, Arjan van de Ven wrote:
> if you need that much you probably should redesign your algorithms to
> not need vmalloc in the first place....
I think you've convinced me that vmalloc is not a good choice when a
driver needs a large buffer (many megabytes) for DMA.
In this case, we need a large ring buffer for reception of isochronous
packets from a firewire device. If I understand you correctly, you are
suggesting that this buffer be obtained as followed:
1. Application performs malloc() in user-space and mmap()s it.
2. Driver uses vmalloc_to_page() on every page of the malloc'ed memory
and constructs a scatter-gather list.
3. Map the sg list with pci_map_sg().
4. Commence DMA.
Is that correct? In particular, does it do the right thing in terms of
pinning the memory and dealing with high memory?
I notice that the block I/O API has some convenience functions for this,
but this is not a block device. Are there some other convenience
functions that can be used?
Forgive me if these are obvious questions -- I'm not the developer of
video1394, but I'd still like get it right for the new firewire stack
that's being developed.
Thanks,
David
On 1/15/07, David Moore <[email protected]> wrote:
> On Mon, 2007-01-15 at 10:20 -0800, Arjan van de Ven wrote:
> > if you need that much you probably should redesign your algorithms to
> > not need vmalloc in the first place....
>
> I think you've convinced me that vmalloc is not a good choice when a
> driver needs a large buffer (many megabytes) for DMA.
>
> In this case, we need a large ring buffer for reception of isochronous
> packets from a firewire device. If I understand you correctly, you are
> suggesting that this buffer be obtained as followed:
>
> 1. Application performs malloc() in user-space and mmap()s it.
> 2. Driver uses vmalloc_to_page() on every page of the malloc'ed memory
> and constructs a scatter-gather list.
> 3. Map the sg list with pci_map_sg().
> 4. Commence DMA.
>
> Is that correct? In particular, does it do the right thing in terms of
> pinning the memory and dealing with high memory?
>
> I notice that the block I/O API has some convenience functions for this,
> but this is not a block device. Are there some other convenience
> functions that can be used?
>
> Forgive me if these are obvious questions -- I'm not the developer of
> video1394, but I'd still like get it right for the new firewire stack
> that's being developed.
David, thanks for bringing this up. Indeed if vmalloc is not the
right way for allocating big buffers, we need to figure something else
out. My impression was that the vmalloc group of functions could be
used for big allocations since they don't require the underlying
memory to be physically contiguous.
What I'm doing currently in the new firewire stack is to vmalloc the
memory to be used for isochronous payload and then use
remap_vmalloc_range() to map the memory to the user. I never access
the contents from the kernel side, I just use vmalloc so I can pass
the pointer to remap_vmalloc_range(). Maybe this is overkill and a
better way to do this is to call get_page() a number of times and
manually add these pages to the process address space without ever
setting up a kernel side mapping for these.
However, what I'd really like to do is to leave it to user space to
allocate the memory as David describes. In the transmit case, user
space allocates memory (malloc or mmap) and loads the payload into
that buffer. Then is does an ioctl() on the firewire control device
to indicate the location of this buffer, describe how that payload is
to be split into packets, and optionally a header per packet to
prepend. The kernel side driver then converts the user space
addresses to pages, pins the pages in question, and sets up dma
programs to transmit the packets.
Likewise for reception, user space allocates buffers for receiving the
data and then instructs the kernel the receive a certain amount of
data into this buffer. The kernel pins the pages backing the user
space buffer and sets up dma to received into those pages. Once a
page it full, it's unpinned and userspace is notified.
It's not too difficult from what I'm doing now, I'd just like to give
user space more control over the buffers it uses for streaming (i.e.
letting user space allocate them). What I'm missing here is: how do I
actually pin a page in memory? I'm sure it's not too difficult, but I
haven't yet figured it out and I'm sure somebody knows it off the top
of his head.
cheers,
Kristian
> However, what I'd really like to do is to leave it to user space to
> allocate the memory as David describes. In the transmit case, user
> space allocates memory (malloc or mmap) and loads the payload into
> that buffer.
there is a lot of pain involved with doing things this way, it is a TON
better if YOU provide the memory via a custom mmap handler for a device
driver.
(there are a lot of security nightmares involved with the opposite
model, like the user can put any kind of memory there, even pci mmio
space)
> Then is does an ioctl() on the firewire control device
ioctls are evil ;) esp an "mmap me" ioctl
> It's not too difficult from what I'm doing now, I'd just like to give
> user space more control over the buffers it uses for streaming (i.e.
> letting user space allocate them). What I'm missing here is: how do I
> actually pin a page in memory? I'm sure it's not too difficult, but I
> haven't yet figured it out and I'm sure somebody knows it off the top
> of his head.
again the best way is for you to provide an mmap method... you can then
fill in the pages and keep that in some sort of array; this is for
example also what the DRI/DRM layer does for textures etc...
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On 1/15/07, Arjan van de Ven <[email protected]> wrote:
>
> > However, what I'd really like to do is to leave it to user space to
> > allocate the memory as David describes. In the transmit case, user
> > space allocates memory (malloc or mmap) and loads the payload into
> > that buffer.
>
> there is a lot of pain involved with doing things this way, it is a TON
> better if YOU provide the memory via a custom mmap handler for a device
> driver.
> (there are a lot of security nightmares involved with the opposite
> model, like the user can put any kind of memory there, even pci mmio
> space)
OK, point taken. I don't have a strong preference for the opposite
model, it just seems elegant that you can let user space handle
allocation and pin and map the pages as needed. But you're right, it
certainly is easier to give safe memory to user space in the first
place rather than try to make sure user space isn't trying to trick
us.
> > Then is does an ioctl() on the firewire control device
>
> ioctls are evil ;) esp an "mmap me" ioctl
Ah, I'm not mmap'ing it from the ioctl, I do implement the mma file
operation for this. However, you have to do an ioctl before mapping
the device to configure the dma context.
Other than that what is the problem with ioctls, and more interesting,
what is the alternative? I don't expect (or want) a bunch of syscalls
to be added for this, so I don't really see what other mechanism I
should use for this.
> > It's not too difficult from what I'm doing now, I'd just like to give
> > user space more control over the buffers it uses for streaming (i.e.
> > letting user space allocate them). What I'm missing here is: how do I
> > actually pin a page in memory? I'm sure it's not too difficult, but I
> > haven't yet figured it out and I'm sure somebody knows it off the top
> > of his head.
>
> again the best way is for you to provide an mmap method... you can then
> fill in the pages and keep that in some sort of array; this is for
> example also what the DRI/DRM layer does for textures etc...
That sounds a lot like what I have now (mmap method, array of pages)
so I'll just stick with that.
thanks,
Kristian
On Tuesday 16 January 2007 06:43, Kristian Høgsberg wrote:
> On 1/15/07, Arjan van de Ven <[email protected]> wrote:
> > there is a lot of pain involved with doing things this way, it is a TON
> > better if YOU provide the memory via a custom mmap handler for a device
> > driver.
> > (there are a lot of security nightmares involved with the opposite
> > model, like the user can put any kind of memory there, even pci mmio
> > space)
>
> OK, point taken. I don't have a strong preference for the opposite
> model, it just seems elegant that you can let user space handle
> allocation and pin and map the pages as needed. But you're right, it
> certainly is easier to give safe memory to user space in the first
> place rather than try to make sure user space isn't trying to trick
> us.
I am glad that the discussion is heading to the right place thanks to David.
Yes. Probably that is the best solution. In the case of the ring buffers,
based on my discussion with Damien, 4 buffers are probably optimal. If the
user is allocating them, in case of normal cameras, this is somewhere around
4 MiB, lets say maxim 16 MiB. So, everything should be ok for normal people,
at least for now. The problem is when the cameras require bigger images (we
are thinking about the future, right) and maybe also more buffers in the DMA
ring buffer. If you leave that to the user, it will require some hacking
skills if we are using the current model from libdc1394 and video1394. Why?
Because if you use 10 buffers with some big images it is likely you are going
out of the 64 MiB. In that case, we were thinking to give a nice error (that
is why we needed to know the amount available for mmap/vmalloc) and instruct
the user to change the kernel boot time allocation of memory in a way that
will fit the range (the vmalloc=xxx at startup - the "hacking"). So, in a
way, it will be nice to have the solution close to the one proposed by David.
Do you think that if the user allocates small buffers (instead of the big
ring buffer) and sends the list to the driver, this will help in breaking the
64 limit? I have doubts about it, but I am not good at this level of VMA.
Anyway, I hope that something can be done to allow bigger DMA ring buffers
without the user needing to reboot the system with some parameter.
> > > Then is does an ioctl() on the firewire control device
> >
> > ioctls are evil ;) esp an "mmap me" ioctl
>
> Ah, I'm not mmap'ing it from the ioctl, I do implement the mma file
> operation for this. However, you have to do an ioctl before mapping
> the device to configure the dma context.
>
> Other than that what is the problem with ioctls, and more interesting,
> what is the alternative? I don't expect (or want) a bunch of syscalls
> to be added for this, so I don't really see what other mechanism I
> should use for this.
>
> > > It's not too difficult from what I'm doing now, I'd just like to give
> > > user space more control over the buffers it uses for streaming (i.e.
> > > letting user space allocate them). What I'm missing here is: how do I
> > > actually pin a page in memory? I'm sure it's not too difficult, but I
> > > haven't yet figured it out and I'm sure somebody knows it off the top
> > > of his head.
> >
> > again the best way is for you to provide an mmap method... you can then
> > fill in the pages and keep that in some sort of array; this is for
> > example also what the DRI/DRM layer does for textures etc...
>
> That sounds a lot like what I have now (mmap method, array of pages)
> so I'll just stick with that.
>
> thanks,
> Kristian
On Mon, 2007-01-15 at 16:43 -0500, Kristian H?gsberg wrote:
> On 1/15/07, Arjan van de Ven <[email protected]> wrote:
> > again the best way is for you to provide an mmap method... you can then
> > fill in the pages and keep that in some sort of array; this is for
> > example also what the DRI/DRM layer does for textures etc...
>
> That sounds a lot like what I have now (mmap method, array of pages)
> so I'll just stick with that.
It sounds like the distinction Arjan is getting at is that the buffer
should exist in the process's virtual address space instead of the
kernel's virtual address space so that we have plenty of space available
to us.
Thus, we should use get_user_pages() instead of vmalloc(). I think
get_user_pages() will also automatically pin the memory. And we'll also
need to call get_user_pages() from a custom mmap() handler so that we
know what process virtual address to assign to the region.
Is that right Arjan?
Thanks,
David
Hi,
> I think you've convinced me that vmalloc is not a good choice when a
> driver needs a large buffer (many megabytes) for DMA.
Yep. bttv used to do that long time ago, and some people used to run
into trouble because the nvidia driver used lots of vmalloc address
space ...
> In this case, we need a large ring buffer for reception of isochronous
> packets from a firewire device. If I understand you correctly, you are
> suggesting that this buffer be obtained as followed:
>
> 1. Application performs malloc() in user-space and mmap()s it.
> 2. Driver uses vmalloc_to_page() on every page of the malloc'ed memory
> and constructs a scatter-gather list.
> 3. Map the sg list with pci_map_sg().
> 4. Commence DMA.
video4linux drivers do this:
(1) app opens /dev/video0
(2) app issues some ioctls to set buffer count, format and size
(3) app uses mmap(/dev/video0) to map the buffers.
Buffer memory is simple userspace memory, it is allocated by the nopage
handler. Memory is pinned down by get_user_pages().
> Are there some other convenience
> functions that can be used?
Look at drivers/media/video/video-buf.c, that is the buffer management
code shared by v4l drivers. It should at least give an idea how it
works, not sure you can actually reuse the code. Chances are not too
bad though, it is used not only for video buffers, but also for audio
and mpeg2 data, so it could be generic enough to fit your needs too.
cheers,
Gerd
--
Gerd Hoffmann <[email protected]>
On Tue, 2007-01-16 at 00:21 -0500, David Moore wrote:
> On Mon, 2007-01-15 at 16:43 -0500, Kristian Høgsberg wrote:
> > On 1/15/07, Arjan van de Ven <[email protected]> wrote:
> > > again the best way is for you to provide an mmap method... you can then
> > > fill in the pages and keep that in some sort of array; this is for
> > > example also what the DRI/DRM layer does for textures etc...
> >
> > That sounds a lot like what I have now (mmap method, array of pages)
> > so I'll just stick with that.
>
> It sounds like the distinction Arjan is getting at is that the buffer
> should exist in the process's virtual address space instead of the
> kernel's virtual address space so that we have plenty of space available
> to us.
exactly! and either way the user would need that space allocated ANYWAY.
>
> Thus, we should use get_user_pages() instead of vmalloc(). I think
> get_user_pages() will also automatically pin the memory.
actually if you provide the mmap method yourself you're not going to
need get_user_pages(), because it's kernel allocated memory already!
> And we'll also
> need to call get_user_pages() from a custom mmap() handler so that we
> know what process virtual address to assign to the region.
see above; it's one or the other. Personally I'd think the mmap method
is simpler, because there's less conditions to consider (again, the
malicious user passing you memory that is mmap'd PCI MMIO space is a
"fun" example, but there's a lot more cafes which are... funky)
Greetings,
Arjan van de Ven
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org