I'm trying to allocate a buffer to be used for ISA DMA and I'm
experiencing some difficulties.
I'm allocating a 64kB buffer (max size for low ISA DMA) using:
kmalloc(65536, GFP_KERNEL | GFP_DMA);
The choice of flags are from another driver that does ISA DMA so I
didn't put too much thought into them at first.
The problem is now that this allocation doesn't always succeed. When it
fails I get:
insmod: page allocation failure. order:4, mode:0x11
and a nice little stack dump.
Digging around in gfp.h to see if I have the proper flags I find that I
currently have the following:
* __GFP_WAIT : This seems to indicate that the process should be put to
sleep until the allocation can succeed. Doesn't seem to work that way
though.
* __GFP_IO : What is meant with physical IO? PCI DMA? This buffer needs
only be read by the ISA DMA controller and the driver in kernel space.
Any useful data is copied to other buffers.
* __GFP_FS : Since the data is copied before use this probably isn't needed.
* __GFP_DMA : From what I've been told, this flags causes the allocator
to do the magic required for the buffer to end up i memory accessible
from the ISA DMA controller. So this seems to be the only flag that
actually does anything useful.
My question is now, why does the allocation fail (sometimes) and what
should I do about it?
Memory fragmentation and overusage seems like reasons to why but why
doesn't the kernel throw out cache pages and reorganise user pages so
that the allocation can succeed?
As for solutions I've tried using __GFP_REPEAT which seems to do the
trick. But the double underscore indicates (at least to me) that these
are internal defines that shouldn't be used except for very special
cases. What is the policy about these?
Rgds
Pierre
On Tue, Nov 02, 2004 at 04:49:20PM +0100, Pierre Ossman wrote:
> I'm trying to allocate a buffer to be used for ISA DMA and I'm
> experiencing some difficulties.
>
> I'm allocating a 64kB buffer (max size for low ISA DMA) using:
>
> kmalloc(65536, GFP_KERNEL | GFP_DMA);
>
> The choice of flags are from another driver that does ISA DMA so I
> didn't put too much thought into them at first.
>
> The problem is now that this allocation doesn't always succeed. When it
> fails I get:
>
> insmod: page allocation failure. order:4, mode:0x11
This is a big allocation and the kernel is having problem finding such a
big page, due to memory fragmentation (as you mention below).
What kernel version are you using?
-mm contains a series of patches from Nick which should make the situation
better, have you tried it? Currently kswapd doenst honour high order
page shortage.
> and a nice little stack dump.
>
> Digging around in gfp.h to see if I have the proper flags I find that I
> currently have the following:
>
> * __GFP_WAIT : This seems to indicate that the process should be put to
> sleep until the allocation can succeed. Doesn't seem to work that way
> though.
>
> * __GFP_IO : What is meant with physical IO? PCI DMA? This buffer needs
> only be read by the ISA DMA controller and the driver in kernel space.
> Any useful data is copied to other buffers.
>
> * __GFP_FS : Since the data is copied before use this probably isn't needed.
>
> * __GFP_DMA : From what I've been told, this flags causes the allocator
> to do the magic required for the buffer to end up i memory accessible
> from the ISA DMA controller. So this seems to be the only flag that
> actually does anything useful.
>
> My question is now, why does the allocation fail (sometimes) and what
> should I do about it?
>
> Memory fragmentation and overusage seems like reasons to why but why
> doesn't the kernel throw out cache pages and reorganise user pages so
> that the allocation can succeed?
We're working on that.
> As for solutions I've tried using __GFP_REPEAT which seems to do the
> trick. But the double underscore indicates (at least to me) that these
> are internal defines that shouldn't be used except for very special
> cases. What is the policy about these?
Its OK to use these flags externally. They might change in future major kernel
versions though, or even future v2.6 release. ie its not a stable API.
Marcelo Tosatti wrote:
>On Tue, Nov 02, 2004 at 04:49:20PM +0100, Pierre Ossman wrote:
>
>
>>
>>The problem is now that this allocation doesn't always succeed. When it
>>fails I get:
>>
>>insmod: page allocation failure. order:4, mode:0x11
>>
>>
>
>This is a big allocation and the kernel is having problem finding such a
>big page, due to memory fragmentation (as you mention below).
>
>What kernel version are you using?
>
>
I'm currently running 2.6.9. No external patches (except for my own
stuff related to this driver).
>-mm contains a series of patches from Nick which should make the situation
>better, have you tried it? Currently kswapd doenst honour high order
>page shortage.
>
>
>
No I haven't. Only saw it today and I usually don't use the -mm tree.
I've gotten the impression it's a bit too bleeding edge for me ;)
What do these patches add to the mix?
I'm also not familiar what the order means. I guess it's some kind of
priority system? Is there a way I can raise my priority to get access to
the memory that kswapd actually keeps available?
>>As for solutions I've tried using __GFP_REPEAT which seems to do the
>>trick. But the double underscore indicates (at least to me) that these
>>are internal defines that shouldn't be used except for very special
>>cases. What is the policy about these?
>>
>>
>
>Its OK to use these flags externally. They might change in future major kernel
>versions though, or even future v2.6 release. ie its not a stable API.
>
>
Is there any other way of increasing the chances of actually getting the
pages I need? Since it is DMA it needs to be one big block.
Rgds
Pierre
On Tue, Nov 02, 2004 at 07:01:55PM +0100, Pierre Ossman wrote:
> Marcelo Tosatti wrote:
>
> >On Tue, Nov 02, 2004 at 04:49:20PM +0100, Pierre Ossman wrote:
> >
> >
> >>
> >>The problem is now that this allocation doesn't always succeed. When it
> >>fails I get:
> >>
> >>insmod: page allocation failure. order:4, mode:0x11
> >>
> >>
> >
> >This is a big allocation and the kernel is having problem finding such a
> >big page, due to memory fragmentation (as you mention below).
> >
> >What kernel version are you using?
> >
> >
> I'm currently running 2.6.9. No external patches (except for my own
> stuff related to this driver).
>
> >-mm contains a series of patches from Nick which should make the situation
> >better, have you tried it? Currently kswapd doenst honour high order
> >page shortage.
> >
> >
> >
> No I haven't. Only saw it today and I usually don't use the -mm tree.
> I've gotten the impression it's a bit too bleeding edge for me ;)
> What do these patches add to the mix?
They make kswapd free pages when high order allocations fail.
> I'm also not familiar what the order means. I guess it's some kind of
> priority system? Is there a way I can raise my priority to get access to
> the memory that kswapd actually keeps available?
order indicates the order which 2 will be elevated, to indicate how many
physical pages this "high order page" is, in size.
"2 ^ 0" = 1 page, order 0 , "2 ^ 1" = 2 pages, order 1,
"2 ^ 2" = 4 pages, order 2 and so on.
>
> >>As for solutions I've tried using __GFP_REPEAT which seems to do the
> >>trick. But the double underscore indicates (at least to me) that these
> >>are internal defines that shouldn't be used except for very special
> >>cases. What is the policy about these?
> >>
> >>
> >
> >Its OK to use these flags externally. They might change in future major
> >kernel
> >versions though, or even future v2.6 release. ie its not a stable API.
> >
> >
> Is there any other way of increasing the chances of actually getting the
> pages I need? Since it is DMA it needs to be one big block.
__GFP_NOFAIL, from gfp.h:
* Action modifiers - doesn't change the zoning
*
* __GFP_REPEAT: Try hard to allocate the memory, but the allocation attempt
* _might_ fail. This depends upon the particular VM implementation.
*
* __GFP_NOFAIL: The VM implementation _must_ retry infinitely: the caller
* cannot handle allocation failures.
*
* __GFP_NORETRY: The VM implementation must not retry indefinitely.
*/
Marcelo Tosatti wrote:
>On Tue, Nov 02, 2004 at 07:01:55PM +0100, Pierre Ossman wrote:
>
>
>>Is there any other way of increasing the chances of actually getting the
>>pages I need? Since it is DMA it needs to be one big block.
>>
>>
>
>__GFP_NOFAIL, from gfp.h:
>
> * Action modifiers - doesn't change the zoning
> *
> * __GFP_REPEAT: Try hard to allocate the memory, but the allocation attempt
> * _might_ fail. This depends upon the particular VM implementation.
> *
> * __GFP_NOFAIL: The VM implementation _must_ retry infinitely: the caller
> * cannot handle allocation failures.
> *
> * __GFP_NORETRY: The VM implementation must not retry indefinitely.
> */
>
>
>
Yes, I've browsed through these. __GFP_NOFAIL seems like it can hang for
a very long time (I don't know if there is an upper bound on how long it
will have to wait for a free page). __GFP_REPEAT seems to work good
enough in this case.
My question was meant to be more along the lines of "Is there anything I
can do without resorting to unstable/interal API:s?".
Rgds
Pierre
On Tue, Nov 02, 2004 at 07:31:39PM +0100, Pierre Ossman wrote:
> Marcelo Tosatti wrote:
>
> >On Tue, Nov 02, 2004 at 07:01:55PM +0100, Pierre Ossman wrote:
> >
> >
> >>Is there any other way of increasing the chances of actually getting the
> >>pages I need? Since it is DMA it needs to be one big block.
> >>
> >>
> >
> >__GFP_NOFAIL, from gfp.h:
> >
> >* Action modifiers - doesn't change the zoning
> >*
> >* __GFP_REPEAT: Try hard to allocate the memory, but the allocation attempt
> >* _might_ fail. This depends upon the particular VM implementation.
> >*
> >* __GFP_NOFAIL: The VM implementation _must_ retry infinitely: the caller
> >* cannot handle allocation failures.
> >*
> >* __GFP_NORETRY: The VM implementation must not retry indefinitely.
> >*/
> >
> >
> >
> Yes, I've browsed through these. __GFP_NOFAIL seems like it can hang for
> a very long time (I don't know if there is an upper bound on how long it
> will have to wait for a free page). __GFP_REPEAT seems to work good
> enough in this case.
> My question was meant to be more along the lines of "Is there anything I
> can do without resorting to unstable/interal API:s?".
Not really.
They are not that unstable, I shouldnt mean that.
These defines are not as stable as system calls - VM internals might change
in v2.7 and the flags also - but for v2.6 they are very likely to remain
untouched.
Its just like any driver API in Linux - they change.
Just keep an eye.
Marcelo Tosatti wrote:
>On Tue, Nov 02, 2004 at 07:31:39PM +0100, Pierre Ossman wrote:
>
>
>>
>>Yes, I've browsed through these. __GFP_NOFAIL seems like it can hang for
>>a very long time (I don't know if there is an upper bound on how long it
>>will have to wait for a free page). __GFP_REPEAT seems to work good
>>enough in this case.
>>My question was meant to be more along the lines of "Is there anything I
>>can do without resorting to unstable/interal API:s?".
>>
>>
>
>Not really.
>
>They are not that unstable, I shouldnt mean that.
>
>These defines are not as stable as system calls - VM internals might change
>in v2.7 and the flags also - but for v2.6 they are very likely to remain
>untouched.
>
>Its just like any driver API in Linux - they change.
>
>Just keep an eye.
>
>
Ok. I'll stick with __GFP_REPEAT for now then. If things break in the
future it should be fairly obvious that this is the problem. Any users
of the driver will probably point out that it's broken rather quickly :)
Thanks for the help!
Rgds
Pierre