Things change. In particular Linux since 2.x.x. After spending several
hours peering at prolifiration of functions and constants in mm.h I
decided to take an easy way out and ask people responsible for this
sophistication:
How does one do the following task: obtain a bunch of free pages (around
300K) with physical addresses between certain bounds (more then
0x4000000, but it is likely this is not constant) reserver them and map to
kernel space so that the driver can access them directly ?
thanks !
Vladimir Dergachev
> How does one do the following task: obtain a bunch of free pages (around
> 300K) with physical addresses between certain bounds (more then
> 0x4000000, but it is likely this is not constant) reserver them and map to
> kernel space so that the driver can access them directly ?
We support allocating pages below 16Mb, below 4Gb, or anywhere within RAM
on x86. If you want to within a range or your 300K must be a single 300K
block then you need to allocate it at boot time
On Mon, 10 Dec 2001, Alan Cox wrote:
> > How does one do the following task: obtain a bunch of free pages (around
> > 300K) with physical addresses between certain bounds (more then
> > 0x4000000, but it is likely this is not constant) reserver them and map to
> > kernel space so that the driver can access them directly ?
>
> We support allocating pages below 16Mb, below 4Gb, or anywhere within RAM
> on x86. If you want to within a range or your 300K must be a single 300K
> block then you need to allocate it at boot time
>
It does not have to be contiguous. I was thinking of simply starting with
the smallest address and trying to free the pages until the needed amount
is available. But I have no idea how to properly do locking or force the
page to be swapped out or something.
As for doing this during boot time this does not seem like a good idea for
two reasons:
a) I need at least _two_ buffers 300K each
b) the actual amount of RAM needed changes depending on video standard
(this is for video capture).
thanks
Vladimir Dergachev
> It does not have to be contiguous. I was thinking of simply starting with
> the smallest address and trying to free the pages until the needed amount
> is available. But I have no idea how to properly do locking or force the
> page to be swapped out or something.
You can simply get_free_page() 300K of pages. However you can't land them
in a given band other than the existing "below 16Mb" "below 4Gb" "anywhere"
bands.
Alan
On Mon, 10 Dec 2001, Alan Cox wrote:
> > It does not have to be contiguous. I was thinking of simply starting with
> > the smallest address and trying to free the pages until the needed amount
> > is available. But I have no idea how to properly do locking or force the
> > page to be swapped out or something.
>
> You can simply get_free_page() 300K of pages. However you can't land them
> in a given band other than the existing "below 16Mb" "below 4Gb" "anywhere"
> bands.
Right, but then my card refuses to dma into anything with address smaller
than 04000000.
I have thought of doing get_free_page until I have enough of pages with
large addresses (and then free all the small ones as 64mb is a finite
amount) but this would place a big load on the system during buffer
allocation.
I was hoping for something more elegant, but I am not adverse to writing
my own get_free_page_from_range().
Vladimir Dergachev
>
> Alan
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
> Right, but then my card refuses to dma into anything with address smaller
> than 04000000.
What was your board designer on when they decided to bar DMA below 64Mb ?
> amount) but this would place a big load on the system during buffer
> allocation.
And might never terminate
> I was hoping for something more elegant, but I am not adverse to writing
> my own get_free_page_from_range().
Thats not a trivial task.
On Mon, 10 Dec 2001, Alan Cox wrote:
> > I was hoping for something more elegant, but I am not adverse to writing
> > my own get_free_page_from_range().
>
> Thats not a trivial task.
Especially because we never quite know the users of a
physical page, so moving data around is somewhat hard.
cheers,
Rik
--
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
http://www.surriel.com/ http://distro.conectiva.com/
On Mon, 10 Dec 2001, Alan Cox wrote:
> > Right, but then my card refuses to dma into anything with address smaller
> > than 04000000.
>
> What was your board designer on when they decided to bar DMA below 64Mb ?
Not mine (unfortunately ?). The card in question is ATI All-in-Wonder
Radeon - and I am trying to get video capture working. Unfortunately the
documentation is quite terse - for example the dma table register
description mentions its width, offset and "No description is provided".
Additionally I have to tiptoe my way around DRI driver which also uses
busmastering.
At the moment I have a working driver with one bug: DMA transfer just does
not happen into pages with address less than 0x4000000 which is the same
value as physical address of the ring buffer.
I suspect that this is somehow connected with the amount of available
"AGP addressable" memory which is ~64meg less than the total amount on my
machine. However, looking around in AGP driver or AGP specs does not seem
to indicate any restriction of the sort and, moreover, I do not need AGP
for this DMA transfer (it is PCI only).
>
> > amount) but this would place a big load on the system during buffer
> > allocation.
>
> And might never terminate
I thought of establishing a timeout and giving up after a while. The
buffer is allocated once for each open on /dev/videoX so it is not too
critical - though I would not want to cause and OOM condition.
>
> > I was hoping for something more elegant, but I am not adverse to writing
> > my own get_free_page_from_range().
>
> Thats not a trivial task.
>
Better than giving up.. Unfortunately looking around in
linux/Documentation and drivers did not yield much in terms of
explanation. I know I can use mem_map_reserve to reserve a page but I
don't know how to get page struct from a physical address nor which lock
to use when messing with this.
thanks
Vladimir Dergachev
On Mon, 10 Dec 2001, Rik van Riel wrote:
> On Mon, 10 Dec 2001, Alan Cox wrote:
>
> > > I was hoping for something more elegant, but I am not adverse to writing
> > > my own get_free_page_from_range().
> >
> > Thats not a trivial task.
>
> Especially because we never quite know the users of a
> physical page, so moving data around is somewhat hard.
I don't want to move them - I just want to collect all that are free and
then try to free some more.
Vladimir Dergachev
>
> cheers,
>
> Rik
> --
> DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
>
> http://www.surriel.com/ http://distro.conectiva.com/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Mon, 10 Dec 2001 [email protected] wrote:
> On Mon, 10 Dec 2001, Rik van Riel wrote:
> > On Mon, 10 Dec 2001, Alan Cox wrote:
> >
> > > > I was hoping for something more elegant, but I am not adverse to writing
> > > > my own get_free_page_from_range().
> > >
> > > Thats not a trivial task.
> >
> > Especially because we never quite know the users of a
> > physical page, so moving data around is somewhat hard.
>
> I don't want to move them - I just want to collect all that are free
> and then try to free some more.
I could put it on the TODO list for my VM stuff, but it's
not exactly near the top of the list so it might take quite
a while more before I get around to this...
http://linuxvm.bkbits.net/
cheers,
Rik
--
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
http://www.surriel.com/ http://distro.conectiva.com/
On Mon, 10 Dec 2001, Rik van Riel wrote:
> On Mon, 10 Dec 2001 [email protected] wrote:
> > On Mon, 10 Dec 2001, Rik van Riel wrote:
> > > On Mon, 10 Dec 2001, Alan Cox wrote:
> > >
> > > > > I was hoping for something more elegant, but I am not adverse to writing
> > > > > my own get_free_page_from_range().
> > > >
> > > > Thats not a trivial task.
> > >
> > > Especially because we never quite know the users of a
> > > physical page, so moving data around is somewhat hard.
> >
> > I don't want to move them - I just want to collect all that are free
> > and then try to free some more.
>
> I could put it on the TODO list for my VM stuff, but it's
> not exactly near the top of the list so it might take quite
> a while more before I get around to this...
Thanks, but I was more hoping for the advice on how I can make it myself..
the same way as bt848 driver has its own memory allocation functions.
Vladimir Dergachev
>
> http://linuxvm.bkbits.net/
>
> cheers,
>
> Rik
> --
> DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
>
> http://www.surriel.com/ http://distro.conectiva.com/
>
> I don't want to move them - I just want to collect all that are free and
> then try to free some more.
How will you free them, you don't know who owns them.
>> > I was hoping for something more elegant, but I am not adverse to writing
>> > my own get_free_page_from_range().
>>
>> Thats not a trivial task.
>
> Better than giving up.. Unfortunately looking around in
> linux/Documentation and drivers did not yield much in terms of
> explanation. I know I can use mem_map_reserve to reserve a page but I
> don't know how to get page struct from a physical address nor which lock
> to use when messing with this.
If you don't have any ISA DMA going on in the system, you might consider
bastardising the ZONE_DMA page range by moving the boundary up to
64Mb, then fixing the allocator not to fail back ZONE_NORMAL et al
allocations to ZONE_DMA. Thus what was originally ZONE_DMA becomes
a sort of ZONE_NO_DMA. Not in the slightest bit pretty, but it might be easier
to implement. Depends if you ever want it to get back into the main tree,
I guess ;-)
M.
> "AGP addressable" memory which is ~64meg less than the total amount on my
> machine. However, looking around in AGP driver or AGP specs does not seem
> to indicate any restriction of the sort and, moreover, I do not need AGP
> for this DMA transfer (it is PCI only).
Can the transfer go to pages mapped into the AGP gart, using their gart
side mapping ?
> Better than giving up.. Unfortunately looking around in
> linux/Documentation and drivers did not yield much in terms of
> explanation. I know I can use mem_map_reserve to reserve a page but I
> don't know how to get page struct from a physical address nor which lock
> to use when messing with this.
You have to grab them at boot time via bootmem to get them in a range of
your choice. Otherwise you can use
get_free_page - grab a page
virt_to_page - page struct of page
virt_to_bus - bus addr of page
virt_to_bus isnt portable because real world pci bus mapping on non x86 is
deeply murky and mysterious. But you probably want to worry about that
after it works.
Alan
>>> > I was hoping for something more elegant, but I am not adverse to writing
>>> > my own get_free_page_from_range().
>>>
>>> Thats not a trivial task.
>>
>> Better than giving up.. Unfortunately looking around in
>> linux/Documentation and drivers did not yield much in terms of
>> explanation. I know I can use mem_map_reserve to reserve a page but I
>> don't know how to get page struct from a physical address nor which lock
>> to use when messing with this.
>
> If you don't have any ISA DMA going on in the system, you might consider
> bastardising the ZONE_DMA page range by moving the boundary up to
> 64Mb, then fixing the allocator not to fail back ZONE_NORMAL et al
> allocations to ZONE_DMA. Thus what was originally ZONE_DMA becomes
> a sort of ZONE_NO_DMA. Not in the slightest bit pretty, but it might be easier
> to implement. Depends if you ever want it to get back into the main tree,
> I guess ;-)
Of course, if you just did that, you'd never use the ZONE_DMA memory, so
that's pretty pointless ;-) You need to create an alloc call with an *option*
to force alloc out of ZONE_NORMAL, not make no fallback the default.
M.
On Mon, 10 Dec 2001, Martin J. Bligh wrote:
> >> > I was hoping for something more elegant, but I am not adverse to writing
> >> > my own get_free_page_from_range().
> >>
> >> Thats not a trivial task.
> >
> > Better than giving up.. Unfortunately looking around in
> > linux/Documentation and drivers did not yield much in terms of
> > explanation. I know I can use mem_map_reserve to reserve a page but I
> > don't know how to get page struct from a physical address nor which lock
> > to use when messing with this.
>
> If you don't have any ISA DMA going on in the system, you might consider
> bastardising the ZONE_DMA page range by moving the boundary up to
> 64Mb, then fixing the allocator not to fail back ZONE_NORMAL et al
> allocations to ZONE_DMA. Thus what was originally ZONE_DMA becomes
> a sort of ZONE_NO_DMA. Not in the slightest bit pretty, but it might be easier
> to implement. Depends if you ever want it to get back into the main tree,
> I guess ;-)
Won't work - this is for general public..
Vladimir Dergachev
>
> M.
>
On Mon, 10 Dec 2001, Alan Cox wrote:
> > I don't want to move them - I just want to collect all that are free and
> > then try to free some more.
>
> How will you free them, you don't know who owns them.
I think you misunderstood me - this allocation happens in response to the
system call _not_ in an interrupt handler. So it is ok to wait - as long
as needed. I was thinking of calling page swapper or something and perhaps
going after I/O buffers first.
Vladimir Dergachev
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Mon, 10 Dec 2001, Alan Cox wrote:
> > "AGP addressable" memory which is ~64meg less than the total amount on my
> > machine. However, looking around in AGP driver or AGP specs does not seem
> > to indicate any restriction of the sort and, moreover, I do not need AGP
> > for this DMA transfer (it is PCI only).
>
> Can the transfer go to pages mapped into the AGP gart, using their gart
> side mapping ?
Yes, but agpgart will not let more then one driver use it. So it will be
_either_ 3d or video capture with switching upon Xserver restart. Sucks !
>
> > Better than giving up.. Unfortunately looking around in
> > linux/Documentation and drivers did not yield much in terms of
> > explanation. I know I can use mem_map_reserve to reserve a page but I
> > don't know how to get page struct from a physical address nor which lock
> > to use when messing with this.
>
> You have to grab them at boot time via bootmem to get them in a range of
Does this imply that the driver must be compiled into the kernel ? In that
case I'll need to allocate at least 2meg of ram for each capture card..
> your choice. Otherwise you can use
>
> get_free_page - grab a page
> virt_to_page - page struct of page
> virt_to_bus - bus addr of page
>
> virt_to_bus isnt portable because real world pci bus mapping on non x86 is
> deeply murky and mysterious. But you probably want to worry about that
> after it works.
Yes, I almost went all the way rewriting the code in accordance with
DMA-mapping.txt thinking I am using "non-DMAable" memory or something
until I noticed that all the functions are inlined to trivial for x86.
Vladimir Dergachev
>
> Alan
>
> Yes, but agpgart will not let more then one driver use it. So it will be
> _either_ 3d or video capture with switching upon Xserver restart. Sucks !
That sounds like agpgart needs fixing. Its going to be easier than hacking
the vm code
On Mon, 10 Dec 2001, Alan Cox wrote:
> > Yes, but agpgart will not let more then one driver use it. So it will be
> > _either_ 3d or video capture with switching upon Xserver restart. Sucks !
>
> That sounds like agpgart needs fixing. Its going to be easier than hacking
> the vm code
>
Well, I was trying to avoid that and simply distribute additional memory
management routines with the driver.
Vladimir Dergachev
> > That sounds like agpgart needs fixing. Its going to be easier than hacking
> > the vm code
>
> Well, I was trying to avoid that and simply distribute additional memory
> management routines with the driver.
Thats going to be a no go. To create a new memory zone as you need means
mucking around in the very depths of the mm code in the kernel core.
On Mon, 10 Dec 2001 [email protected] wrote:
> On Mon, 10 Dec 2001, Alan Cox wrote:
>
> > > I don't want to move them - I just want to collect all that are free and
> > > then try to free some more.
> >
> > How will you free them, you don't know who owns them.
>
> I think you misunderstood me - this allocation happens in response to
> the system call _not_ in an interrupt handler. So it is ok to wait -
> as long as needed. I was thinking of calling page swapper or something
> and perhaps going after I/O buffers first.
Even if you have a handle on a physical page, you don't know
what processes are using the page, nor if there are additional
users besides the processes.
This makes it rather hard to go around trying to free pages
within a certain physical range.
cheers,
Rik
--
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
http://www.surriel.com/ http://distro.conectiva.com/
On Mon, 10 Dec 2001, Rik van Riel wrote:
> On Mon, 10 Dec 2001 [email protected] wrote:
> > On Mon, 10 Dec 2001, Alan Cox wrote:
> >
> > > > I don't want to move them - I just want to collect all that are free and
> > > > then try to free some more.
> > >
> > > How will you free them, you don't know who owns them.
> >
> > I think you misunderstood me - this allocation happens in response to
> > the system call _not_ in an interrupt handler. So it is ok to wait -
> > as long as needed. I was thinking of calling page swapper or something
> > and perhaps going after I/O buffers first.
>
> Even if you have a handle on a physical page, you don't know
> what processes are using the page, nor if there are additional
> users besides the processes.
>
> This makes it rather hard to go around trying to free pages
> within a certain physical range.
Well, what does kernel do when it runs out of memory ? For example when I
mmap a large file and start reading it back and force ?
Vladimir Dergachev
>
> cheers,
>
> Rik
> --
> DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
>
> http://www.surriel.com/ http://distro.conectiva.com/
>
> > This makes it rather hard to go around trying to free pages
> > within a certain physical range.
>
> Well, what does kernel do when it runs out of memory ? For example when I
> mmap a large file and start reading it back and force ?
It doesn't care which physical page it gets. Processes being freeing
up/swapping pages they have mappings to. The map counts hit zero and the
page is discarded.
Alan
On Mon, 10 Dec 2001 [email protected] wrote:
> On Mon, 10 Dec 2001, Rik van Riel wrote:
> > Even if you have a handle on a physical page, you don't know
> > what processes are using the page, nor if there are additional
> > users besides the processes.
>
> Well, what does kernel do when it runs out of memory ? For example
> when I mmap a large file and start reading it back and force ?
For the full horror, see mm/vmscan.c, do_try_to_free_memory()
cheers,
Rik
--
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
http://www.surriel.com/ http://distro.conectiva.com/
On Mon, 10 Dec 2001, Alan Cox wrote:
> > > This makes it rather hard to go around trying to free pages
> > > within a certain physical range.
> >
> > Well, what does kernel do when it runs out of memory ? For example when I
> > mmap a large file and start reading it back and force ?
>
> It doesn't care which physical page it gets. Processes being freeing
> up/swapping pages they have mappings to. The map counts hit zero and the
> page is discarded.
Right, but instead of trying to balance cache available memory and swap
my swapper will only be concerned whether the page can be evicted and
whether it is from the address range I want.
The scheme is like:
open -> request buffer allocation -> start region_swapper ->
-> wait for freed memory to accumulate and reserve as it appears ->
-> when enough is available stop swapper and declare allocation finished
Vladimir Dergachev
>
> Alan
>
On Mon, 10 Dec 2001, Rik van Riel wrote:
> On Mon, 10 Dec 2001 [email protected] wrote:
> > On Mon, 10 Dec 2001, Rik van Riel wrote:
>
> > > Even if you have a handle on a physical page, you don't know
> > > what processes are using the page, nor if there are additional
> > > users besides the processes.
> >
> > Well, what does kernel do when it runs out of memory ? For example
> > when I mmap a large file and start reading it back and force ?
>
> For the full horror, see mm/vmscan.c, do_try_to_free_memory()
Wonderful !This is the kind of advice I was looking for :))
So if I copy it over to, say, do_try_to_free_region_memory1() and start
hacking - will this violate some grand scheme of kernel development ?
I.e. is there anything inherently wrong with two different kswapd going on ?
thanks !
Vladimir Dergachev
>
> cheers,
>
> Rik
> --
> DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/
>
> http://www.surriel.com/ http://distro.conectiva.com/
>
> Right, but instead of trying to balance cache available memory and swap
> my swapper will only be concerned whether the page can be evicted and
> whether it is from the address range I want.
You want to rewrite the entire vm to have back pointers ? Right now you
can't find pages in an address range. Its all driven from the virtual side
without reverse lookup tables.
On Mon, 10 Dec 2001, Alan Cox wrote:
> > Right, but instead of trying to balance cache available memory and swap
> > my swapper will only be concerned whether the page can be evicted and
> > whether it is from the address range I want.
>
> You want to rewrite the entire vm to have back pointers ? Right now you
> can't find pages in an address range. Its all driven from the virtual side
> without reverse lookup tables.
>
You are right I don't want to rewrite vm. But I can go thru virtual side
taking note of the physical address. I.e. base the decision to try and
free pages not on how old the page is but on what it's physical address
is.
You see, I don't want to find a few pages in 16mb range in 512mb system.
I want to find a few pages _outside_ 64mb range in a 512mb system.
So if I free 70mb I _will_ be able to find at least 2mb in my desired
range. In fact I won't have to free that much as they it will work is
"try to free the page", "if succeed do not return to memory pool but
instead give to the 'special region list'"
Does this make sense ?
Vladimir Dergachev
[email protected] writes:
> On Mon, 10 Dec 2001, Alan Cox wrote:
>
> > > Right, but instead of trying to balance cache available memory and swap
> > > my swapper will only be concerned whether the page can be evicted and
> > > whether it is from the address range I want.
> >
> > You want to rewrite the entire vm to have back pointers ? Right now you
> > can't find pages in an address range. Its all driven from the virtual side
> > without reverse lookup tables.
> >
>
> You are right I don't want to rewrite vm. But I can go thru virtual side
> taking note of the physical address. I.e. base the decision to try and
> free pages not on how old the page is but on what it's physical address
> is.
>
> You see, I don't want to find a few pages in 16mb range in 512mb system.
>
> I want to find a few pages _outside_ 64mb range in a 512mb system.
> So if I free 70mb I _will_ be able to find at least 2mb in my desired
> range. In fact I won't have to free that much as they it will work is
> "try to free the page", "if succeed do not return to memory pool but
> instead give to the 'special region list'"
>
> Does this make sense ?
There is actually a cheap trick that will achieve what you want.
Allocate pages. If you allocate a page in the 0-64mb range keep
it allocated until you have allocated your 300KB > 64mb. After
you have all of the pages you want free the extra pages in 0-64mb that
you didn't want...
Eric
On 11 Dec 2001, Eric W. Biederman wrote:
> [email protected] writes:
>
> > On Mon, 10 Dec 2001, Alan Cox wrote:
> >
> > > > Right, but instead of trying to balance cache available memory and swap
> > > > my swapper will only be concerned whether the page can be evicted and
> > > > whether it is from the address range I want.
> > >
> > > You want to rewrite the entire vm to have back pointers ? Right now you
> > > can't find pages in an address range. Its all driven from the virtual side
> > > without reverse lookup tables.
> > >
> >
> > You are right I don't want to rewrite vm. But I can go thru virtual side
> > taking note of the physical address. I.e. base the decision to try and
> > free pages not on how old the page is but on what it's physical address
> > is.
> >
> > You see, I don't want to find a few pages in 16mb range in 512mb system.
> >
> > I want to find a few pages _outside_ 64mb range in a 512mb system.
> > So if I free 70mb I _will_ be able to find at least 2mb in my desired
> > range. In fact I won't have to free that much as they it will work is
> > "try to free the page", "if succeed do not return to memory pool but
> > instead give to the 'special region list'"
> >
> > Does this make sense ?
>
> There is actually a cheap trick that will achieve what you want.
> Allocate pages. If you allocate a page in the 0-64mb range keep
> it allocated until you have allocated your 300KB > 64mb. After
> you have all of the pages you want free the extra pages in 0-64mb that
> you didn't want...
Yes, I thought of that, but this might produce more memory pressure than
needed.
Regardless, it looks like I won't need this after all - the device has
internal memory controller which was misprogrammed. I think I corrected
this so it looks to be working now.
However, the question of how to get pages from a given range is
interesting in itself..
big thanks to everyone who responded :))
Vladimir Dergachev
>
> Eric
>
On December 11, 2001 12:07 pm, [email protected] wrote:
> On 11 Dec 2001, Eric W. Biederman wrote:
> > There is actually a cheap trick that will achieve what you want.
> > Allocate pages. If you allocate a page in the 0-64mb range keep
> > it allocated until you have allocated your 300KB > 64mb. After
> > you have all of the pages you want free the extra pages in 0-64mb that
> > you didn't want...
>
> Yes, I thought of that, but this might produce more memory pressure than
> needed.
>
> Regardless, it looks like I won't need this after all - the device has
> internal memory controller which was misprogrammed. I think I corrected
> this so it looks to be working now.
>
> However, the question of how to get pages from a given range is
> interesting in itself..
Yes, particularly numa folks.
--
Daniel