I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
from kernel. A consequence is that DMA zone now has size 0. That causes
many drivers to grief (OOMs).
I see two ways out:
1. Modify individual drivers and convince them not to alloc with GFP_DMA.
I have been trying to do this but do not seem to see an end of it. :)
2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
the whole memory region belongs to DMA zone.
#2 sounds pretty hackish. I am sure something bad will happen
sooner or later (like what?). But so far it appears to be working fine.
The fundamental question is: Has anybody tried to run Linux without 0 sized
DMA zone before? Am I doing something that nobody has done before (which is
something really hard to believe these days with Linux :P)?
Cheers.
Jun
It seems a good idea.
Is dma zone is still necessay on most modern computers?
Best regards,
Conke @ AMD, Inc.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jun Sun
Sent: 2006??11??2?? 10:16
To: [email protected]
Subject: Can Linux live without DMA zone?
I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
from kernel. A consequence is that DMA zone now has size 0. That causes
many drivers to grief (OOMs).
I see two ways out:
1. Modify individual drivers and convince them not to alloc with GFP_DMA.
I have been trying to do this but do not seem to see an end of it. :)
2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
the whole memory region belongs to DMA zone.
#2 sounds pretty hackish. I am sure something bad will happen
sooner or later (like what?). But so far it appears to be working fine.
The fundamental question is: Has anybody tried to run Linux without 0 sized
DMA zone before? Am I doing something that nobody has done before (which is
something really hard to believe these days with Linux :P)?
Cheers.
Jun
On Thu, 2006-11-02 at 11:43 +0800, Conke Hu wrote:
> It seems a good idea.
> Is dma zone is still necessay on most modern computers?
(don't top post!)
if you would have used google, you'd have found this:
http://www.mail-archive.com/[email protected]/msg01623.html
cheers
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Jun Sun
> Sent: 2006年11月2日 10:16
> To: [email protected]
> Subject: Can Linux live without DMA zone?
>
>
> I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
> from kernel. A consequence is that DMA zone now has size 0. That causes
> many drivers to grief (OOMs).
>
> I see two ways out:
>
> 1. Modify individual drivers and convince them not to alloc with GFP_DMA.
> I have been trying to do this but do not seem to see an end of it. :)
>
> 2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
> the whole memory region belongs to DMA zone.
>
> #2 sounds pretty hackish. I am sure something bad will happen
> sooner or later (like what?). But so far it appears to be working fine.
>
> The fundamental question is: Has anybody tried to run Linux without 0 sized
> DMA zone before? Am I doing something that nobody has done before (which is
> something really hard to believe these days with Linux :P)?
>
> Cheers.
>
> Jun
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On Wed, 2006-11-01 at 18:15 -0800, Jun Sun wrote:
> I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
> from kernel. A consequence is that DMA zone now has size 0. That causes
> many drivers to grief (OOMs).
>
> I see two ways out:
>
> 1. Modify individual drivers and convince them not to alloc with GFP_DMA.
> I have been trying to do this but do not seem to see an end of it. :)
>
> 2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
> the whole memory region belongs to DMA zone.
>
> #2 sounds pretty hackish. I am sure something bad will happen
> sooner or later (like what?). But so far it appears to be working fine.
>
> The fundamental question is: Has anybody tried to run Linux without 0 sized
> DMA zone before? Am I doing something that nobody has done before (which is
> something really hard to believe these days with Linux :P)?
on a PC there are still devices that need memory in the lower 16Mb.....
(like floppy)
Maybe you should reserve another area of memory instead!
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Wed, Nov 01, 2006 at 06:15:47PM -0800, Jun Sun wrote:
> The fundamental question is: Has anybody tried to run Linux without 0 sized
> DMA zone before? Am I doing something that nobody has done before (which is
> something really hard to believe these days with Linux :P)?
>
There's patches that rip out ZONE_DMA for platforms that don't have a DMA
limitation (and were pretty much putting all of ZONE_NORMAL in ZONE_DMA),
all of these are already in -mm.
Most PCs do not have ISA or floppy, so maybe we could add an option to enable DMA zone or not.
Best regards,
Conke @ AMD, Inc.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Arjan van de Ven
Sent: 2006??11??2?? 17:16
To: Jun Sun
Cc: [email protected]
Subject: Re: Can Linux live without DMA zone?
On Wed, 2006-11-01 at 18:15 -0800, Jun Sun wrote:
> I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
> from kernel. A consequence is that DMA zone now has size 0. That causes
> many drivers to grief (OOMs).
>
> I see two ways out:
>
> 1. Modify individual drivers and convince them not to alloc with GFP_DMA.
> I have been trying to do this but do not seem to see an end of it. :)
>
> 2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
> the whole memory region belongs to DMA zone.
>
> #2 sounds pretty hackish. I am sure something bad will happen
> sooner or later (like what?). But so far it appears to be working fine.
>
> The fundamental question is: Has anybody tried to run Linux without 0 sized
> DMA zone before? Am I doing something that nobody has done before (which is
> something really hard to believe these days with Linux :P)?
on a PC there are still devices that need memory in the lower 16Mb.....
(like floppy)
Maybe you should reserve another area of memory instead!
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Thu, 2006-11-02 at 18:33 +0800, Conke Hu wrote:
> Most PCs do not have ISA or floppy, so maybe we could add an option to enable DMA zone or not.
>
please don't top-post.
also floppy is still there a lot unfortunately ... as are some of the
more crappy soundcards. ZONE_DMA is a 32 bit linux thing; most modern
systems are 64 bit capable now....
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
Ar Iau, 2006-11-02 am 18:33 +0800, ysgrifennodd Conke Hu:
> Most PCs do not have ISA or floppy, so maybe we could add an option to enable DMA zone or not.
Lots of hardware still needs memory below the 2GB boundary, or for
floppy etc (which is common) in the low 16MB. People are still shipping
wireless hardware with these kind of limits even today.
That code is in 2.6.19-rc4-mm1. If you clear CONFIG_ZONE_DMA then you
wont have a DMA zone anymore.
Shouldn't only ancient ISA drivers be using GFP_DMA? You know, ones
that actually require it? PCI drivers should not have this limit.
Jun Sun wrote:
> I am trying to reserve a block of memory (>16MB) starting from 0 and hide it
> from kernel. A consequence is that DMA zone now has size 0. That causes
> many drivers to grief (OOMs).
>
> I see two ways out:
>
> 1. Modify individual drivers and convince them not to alloc with GFP_DMA.
> I have been trying to do this but do not seem to see an end of it. :)
>
> 2. Simply lie and increase MAX_DMA_ADDRESS to really big (like 1GB) so that
> the whole memory region belongs to DMA zone.
>
> #2 sounds pretty hackish. I am sure something bad will happen
> sooner or later (like what?). But so far it appears to be working fine.
>
> The fundamental question is: Has anybody tried to run Linux without 0 sized
> DMA zone before? Am I doing something that nobody has done before (which is
> something really hard to believe these days with Linux :P)?
>
> Cheers.
>
> Jun
On Thu, 2006-11-02 at 11:32 -0500, Phillip Susi wrote:
> Shouldn't only ancient ISA drivers be using GFP_DMA? You know, ones
> that actually require it? PCI drivers should not have this limit.
that is a nice theory, but unfortunately there is just a lot of "PCI"
hardware out there for which the designers decided to save a bit of
copper and only wire up the lower X address lines (for various values of
X)
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
Ar Iau, 2006-11-02 am 11:32 -0500, ysgrifennodd Phillip Susi:
> Shouldn't only ancient ISA drivers be using GFP_DMA? You know, ones
> that actually require it? PCI drivers should not have this limit.
"Should" and "Do not" are different things. Many PCI drivers have
interesting little limits.
Arjan van de Ven wrote:
> that is a nice theory, but unfortunately there is just a lot of "PCI"
> hardware out there for which the designers decided to save a bit of
> copper and only wire up the lower X address lines (for various values of
> X)
Yea, but shouldn't PCI drivers be using another means than allocating
from GFP_DMA? Wasn't there some sort of bounce buffers call I can't
quite remember the details of? That performs any required translations
to bus hardware addresses, and copies the buffer to a more appropriate
location if required, based on the specific requirements of that device?
I know that most 32 bit PCI devices can't handle addresses above the 4
GB mark on 64 bit machines, but those drivers should NOT be limiting DMA
to the first 16 MB. Especially since most machines don't have over 4 GB
of ram anyhow, but quite often original buffers will be above 16 MB.
On Thu, 2006-11-02 at 14:08 -0500, Phillip Susi wrote:
> Arjan van de Ven wrote:
> > that is a nice theory, but unfortunately there is just a lot of "PCI"
> > hardware out there for which the designers decided to save a bit of
> > copper and only wire up the lower X address lines (for various values of
> > X)
>
> Yea, but shouldn't PCI drivers be using another means than allocating
> from GFP_DMA? Wasn't there some sort of bounce buffers call I can't
> quite remember the details of? That performs any required translations
> to bus hardware addresses, and copies the buffer to a more appropriate
> location if required, based on the specific requirements of that device?
that's for the 32 bit boundary. THe problem is that there are 31, 30, 28
and 26 bit devices as well, and those are in more trouble, and will
eventually fall back to GFP_DMA (inside the x86 PCI code; the driver
just uses the pci dma allocation routines) if they can't get suitable
memory otherwise....
It's all nice in theory. But then there is the reality that not all
devices are nice pci device that implement the entire spec;)
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
Arjan van de Ven wrote:
> that's for the 32 bit boundary. THe problem is that there are 31, 30, 28
> and 26 bit devices as well, and those are in more trouble, and will
> eventually fall back to GFP_DMA (inside the x86 PCI code; the driver
> just uses the pci dma allocation routines) if they can't get suitable
> memory otherwise....
>
> It's all nice in theory. But then there is the reality that not all
> devices are nice pci device that implement the entire spec;)
>
Right, but doesn't the bounce/allocation routine take as a parameter the
limit that the device can handle? If the device can handle 28 bit
addresses, then the kernel should not limit it to only 24 bits.
On Thu, 2006-11-02 at 16:26 -0500, Phillip Susi wrote:
> Arjan van de Ven wrote:
> > that's for the 32 bit boundary. THe problem is that there are 31, 30, 28
> > and 26 bit devices as well, and those are in more trouble, and will
> > eventually fall back to GFP_DMA (inside the x86 PCI code; the driver
> > just uses the pci dma allocation routines) if they can't get suitable
> > memory otherwise....
> >
> > It's all nice in theory. But then there is the reality that not all
> > devices are nice pci device that implement the entire spec;)
> >
>
> Right, but doesn't the bounce/allocation routine take as a parameter the
> limit that the device can handle? If the device can handle 28 bit
> addresses, then the kernel should not limit it to only 24 bits.
you're right in theory, but the kernel only has a few pools of memory
available, but not at every bit boundary. there is a 32 bit pool
(GFP_DMA32) on some, a 30-ish bit pool (GFP_KERNEL) on others, and a 24
bit pool (GFP_DMA) with basically nothing inbetween.
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Thu, Nov 02, 2006 at 11:19:05PM +0100, Arjan van de Ven wrote:
> On Thu, 2006-11-02 at 16:26 -0500, Phillip Susi wrote:
> > Arjan van de Ven wrote:
> > > that's for the 32 bit boundary. THe problem is that there are 31, 30, 28
> > > and 26 bit devices as well, and those are in more trouble, and will
> > > eventually fall back to GFP_DMA (inside the x86 PCI code; the driver
> > > just uses the pci dma allocation routines) if they can't get suitable
> > > memory otherwise....
> > >
> > > It's all nice in theory. But then there is the reality that not all
> > > devices are nice pci device that implement the entire spec;)
> > >
> >
> > Right, but doesn't the bounce/allocation routine take as a parameter the
> > limit that the device can handle? If the device can handle 28 bit
> > addresses, then the kernel should not limit it to only 24 bits.
>
> you're right in theory, but the kernel only has a few pools of memory
> available, but not at every bit boundary. there is a 32 bit pool
> (GFP_DMA32) on some, a 30-ish bit pool (GFP_KERNEL) on others, and a 24
> bit pool (GFP_DMA) with basically nothing inbetween.
>
Perhaps a better solution is to
1. get rid of DMA zone
2. have another alloc funciton (e.g., kmalloc_range()) which takes an
extra pair of parameters to indicate the desired range for the
allocated memory. Most DMA buffers are allocated during start-up.
So the alloc operations should generally be successful.
3. convert drivers over to use the new function.
Cheers.
Jun
are allocated at start-up time.
On Thu, 2 Nov 2006 15:17:15 -0800
Jun Sun <[email protected]> wrote:
> On Thu, Nov 02, 2006 at 11:19:05PM +0100, Arjan van de Ven wrote:
> > On Thu, 2006-11-02 at 16:26 -0500, Phillip Susi wrote:
> > > Arjan van de Ven wrote:
> > > > that's for the 32 bit boundary. THe problem is that there are 31, 30, 28
> > > > and 26 bit devices as well, and those are in more trouble, and will
> > > > eventually fall back to GFP_DMA (inside the x86 PCI code; the driver
> > > > just uses the pci dma allocation routines) if they can't get suitable
> > > > memory otherwise....
> > > >
> > > > It's all nice in theory. But then there is the reality that not all
> > > > devices are nice pci device that implement the entire spec;)
> > > >
> > >
> > > Right, but doesn't the bounce/allocation routine take as a parameter the
> > > limit that the device can handle? If the device can handle 28 bit
> > > addresses, then the kernel should not limit it to only 24 bits.
> >
> > you're right in theory, but the kernel only has a few pools of memory
> > available, but not at every bit boundary. there is a 32 bit pool
> > (GFP_DMA32) on some, a 30-ish bit pool (GFP_KERNEL) on others, and a 24
> > bit pool (GFP_DMA) with basically nothing inbetween.
> >
>
> Perhaps a better solution is to
>
> 1. get rid of DMA zone
>
> 2. have another alloc funciton (e.g., kmalloc_range()) which takes an
> extra pair of parameters to indicate the desired range for the
> allocated memory. Most DMA buffers are allocated during start-up.
> So the alloc operations should generally be successful.
>
Network devices don't allocate buffer until they are brought up.
By then a lot of memory allocation has happened. You could add an
interface that allows a device to say:
kmalloc_range_intent(unsigned long mask, unsigned count, unsigned size)
to cause reservation before use.
--
Stephen Hemminger <[email protected]>
Arjan van de Ven <[email protected]> writes:
> you're right in theory, but the kernel only has a few pools of memory
> available, but not at every bit boundary. there is a 32 bit pool
> (GFP_DMA32) on some, a 30-ish bit pool (GFP_KERNEL) on others, and a 24
> bit pool (GFP_DMA) with basically nothing inbetween.
Perhaps naive question, but... what's wrong with allocating memory
from the top (within given address mask[1], size-wise)? I think we
don't allocate more than 1 page with kmalloc anymore, do we?
[1] - some devices (a specific StrongARM only?) need non-continuous
masks due to a hardware bug(s).
--
Krzysztof Halasa
Jun Sun wrote:
> Perhaps a better solution is to
>
> 1. get rid of DMA zone
>
> 2. have another alloc funciton (e.g., kmalloc_range()) which takes an
> extra pair of parameters to indicate the desired range for the
> allocated memory. Most DMA buffers are allocated during start-up.
> So the alloc operations should generally be successful.
>
> 3. convert drivers over to use the new function.
>
> Cheers.
>
> Jun
> are allocated at start-up time.
That is what I was thinking. You don't need lots of separate pools, you
just need the standard allocator to prefer higher addresses, and then
the bounce routines need to simply check if the existing user buffer
happens to already be within the area the hardware can address ( which
it often will be ), and if not, copy the data to pages allocated in
lower memory.