Hi everyone!
I have a strange problem where I don't even know if there is a solution to it
at the moment:
I'm working on a new way of doing memory-management and currently I allocate
memory mappings (at non-fixed locations) in user-memory (i.e. < TASK_SIZE) in
addition to the regular pages mapped for the apps (e.g. heap memory, etc.)
It all works perfectly well (creating & deleting the additional mappings),
however, when the kernel feels like it needs to allocate a mapping in
user-space it sometimes deletes my mapping and overwrites it with the new
one, although there is plenty of free memory at some other location.
typically my mappings are (automatically chosen) located somewhere around
0xb7xxxxxx although there's plenty free space around 0xayyyyyyy. (i know that
this is a bad location because of stack, etc.) but i need the kernel to
choose a good location for me...
is there a way to make sure my mappings are not removed from memory?? or is
there a location where i can put my mappings that they will not be removed
(i.e. by using MAP_FIXED)?
Note that i need a big amount of memory so the range will have to be fairly
big (to be precise: i need a lot of 0x1000byte mappings - they are always
page-sized, not bigger, not smaller, but A LOT of them :-/ )
hope someone can help me with this!
thanks & greets,
Clemens
On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> It all works perfectly well (creating & deleting the additional
> mappings), however, when the kernel feels like it needs to allocate a
> mapping in user-space it sometimes deletes my mapping and overwrites it
> with the new one, although there is plenty of free memory at some other
> location.
Hi Clemens,
what do you mean by "overwrites it"? It just probably merges your vma with
the newly created one, right?
--
Jiri Kosina
On Thursday 30 August 2007 19:07:05 Jiri Kosina wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > It all works perfectly well (creating & deleting the additional
> > mappings), however, when the kernel feels like it needs to allocate a
> > mapping in user-space it sometimes deletes my mapping and overwrites it
> > with the new one, although there is plenty of free memory at some other
> > location.
>
> Hi Clemens,
>
> what do you mean by "overwrites it"? It just probably merges your vma with
> the newly created one, right?
that's exactly the problem... it replaces my mapping with the new one... i.e.
it first deletes my mapping and then creates the new one at the same
location.
i *guess* this all happens in exec.c, lines 1033ff:
munmap_back:
vma = find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
if (vma && vma->vm_start < addr + len) {
if (do_munmap(mm, addr, len))
return -ENOMEM;
goto munmap_back;
}
On Thursday 30 August 2007 19:07:05 you wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > It all works perfectly well (creating & deleting the additional
> > mappings), however, when the kernel feels like it needs to allocate a
> > mapping in user-space it sometimes deletes my mapping and overwrites it
> > with the new one, although there is plenty of free memory at some other
> > location.
>
> Hi Clemens,
>
> what do you mean by "overwrites it"? It just probably merges your vma with
> the newly created one, right?
does really noone have an answer for me?? i'm having the hardest time to find
a work-around for it.
is there no way to tell the kernel, that a certain mapping must not be
removed, no matter what (except of course an explicit call to sys_unmap, of
course)?
On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> is there no way to tell the kernel, that a certain mapping must not be
> removed, no matter what (except of course an explicit call to sys_unmap,
> of course)?
I don't seem to get what is the issue here. Your mapping is not removed,
only the VMAs are merged together into one larger VMA if they have
neighbouring address ranges and compatible protection bits. See
vma_merge().
--
Jiri Kosina
On Thursday 30 August 2007 23:34:52 you wrote:
> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > is there no way to tell the kernel, that a certain mapping must not be
> > removed, no matter what (except of course an explicit call to sys_unmap,
> > of course)?
>
> I don't seem to get what is the issue here. Your mapping is not removed,
> only the VMAs are merged together into one larger VMA if they have
> neighbouring address ranges and compatible protection bits. See
> vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.
consider the user-space code:
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000 bytes
and create one big vma with size 0x5000 bytes.
is there a way to tell it that the second mmap MUST fail?
On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
> On Thursday 30 August 2007 23:34:52 you wrote:
> > On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > > is there no way to tell the kernel, that a certain mapping must not be
> > > removed, no matter what (except of course an explicit call to sys_unmap,
> > > of course)?
> >
> > I don't seem to get what is the issue here. Your mapping is not removed,
> > only the VMAs are merged together into one larger VMA if they have
> > neighbouring address ranges and compatible protection bits. See
> > vma_merge().
>
> the thing is that they are not. the kernel chooses to REPLACE my mapping.
>
> consider the user-space code:
>
> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
>
> here, the second call to mmap will shorten the first mapping to 0x1000 bytes
> and create one big vma with size 0x5000 bytes.
>
> is there a way to tell it that the second mmap MUST fail?
There's an LSM exit point for mmap, you could perhaps do something there.
What are you trying to achieve by forcing the second one to fail?
On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> the thing is that they are not. the kernel chooses to REPLACE my
> mapping.
> consider the user-space code:
> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
> here, the second call to mmap will shorten the first mapping to 0x1000 bytes
> and create one big vma with size 0x5000 bytes.
Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
- see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
--
Jiri Kosina
On Thursday 30 August 2007 23:50:21 [email protected] wrote:
> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
> > On Thursday 30 August 2007 23:34:52 you wrote:
> > > On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> > > > is there no way to tell the kernel, that a certain mapping must not
> > > > be removed, no matter what (except of course an explicit call to
> > > > sys_unmap, of course)?
> > >
> > > I don't seem to get what is the issue here. Your mapping is not
> > > removed, only the VMAs are merged together into one larger VMA if they
> > > have neighbouring address ranges and compatible protection bits. See
> > > vma_merge().
> >
> > the thing is that they are not. the kernel chooses to REPLACE my mapping.
> >
> > consider the user-space code:
> >
> > mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
> > mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
> >
> > here, the second call to mmap will shorten the first mapping to 0x1000
> > bytes and create one big vma with size 0x5000 bytes.
> >
> > is there a way to tell it that the second mmap MUST fail?
>
> There's an LSM exit point for mmap, you could perhaps do something there.
>
> What are you trying to achieve by forcing the second one to fail?
puh... that is a good question :-)
I'm writing my master's thesis on a new model of memory protection and need to
have every memory mapping in userspace duplicated. I also have kind of a
second PGD/PTD that allows finding this mirrored mapping.
However, as the number of original mappings grows, I suddenly have the problem
that the kernel tries to allocate a new mapping and picks the address of a
mirrored memory page, which it shouldn't.
Honestly, I don't understand why it does so, but it simply does... so
basically I want these second mappings to stay in memory as long as the
original page.
What do you mean exactly with
> There's an LSM exit point for mmap, you could perhaps do something there.
??
By the way / @Jiri Kosina:
> Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
> - see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
I know... I'm not even saying that this is a bug... I just wonder if there is
some way to avoid this problem :-)
Thanks for your help - i really appreciate that!!
Clemens Kolbitsch wrote:
> On Thursday 30 August 2007 23:50:21 [email protected] wrote:
>
>> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
>>
>>> On Thursday 30 August 2007 23:34:52 you wrote:
>>>
>>>> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
>>>>
>>>>> is there no way to tell the kernel, that a certain mapping must not
>>>>> be removed, no matter what (except of course an explicit call to
>>>>> sys_unmap, of course)?
>>>>>
>>>> I don't seem to get what is the issue here. Your mapping is not
>>>> removed, only the VMAs are merged together into one larger VMA if they
>>>> have neighbouring address ranges and compatible protection bits. See
>>>> vma_merge().
>>>>
>>> the thing is that they are not. the kernel chooses to REPLACE my mapping.
>>>
>>> consider the user-space code:
>>>
>>> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
>>> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
>>>
>>> here, the second call to mmap will shorten the first mapping to 0x1000
>>> bytes and create one big vma with size 0x5000 bytes.
>>>
>>> is there a way to tell it that the second mmap MUST fail?
>>>
>> There's an LSM exit point for mmap, you could perhaps do something there.
>>
>> What are you trying to achieve by forcing the second one to fail?
>>
>
> puh... that is a good question :-)
>
> I'm writing my master's thesis on a new model of memory protection and need to
> have every memory mapping in userspace duplicated. I also have kind of a
> second PGD/PTD that allows finding this mirrored mapping.
>
> However, as the number of original mappings grows, I suddenly have the problem
> that the kernel tries to allocate a new mapping and picks the address of a
> mirrored memory page, which it shouldn't.
>
> Honestly, I don't understand why it does so,
The "why" is easy: Having many mappings is expensive,
so merging them (when this cause no problems) is
a smart thing to do. So that is what linux does.
It means fewer mappings to keep track of. Less resources in
use means that linux moves faster.
If you are doing research, consider these methods:
1. Change vma_merge() so it always fail to merge mappings
or
2. Set up your "mappings duplicated in userspace" so
they too merge in the same way.
Helge Hafting
> If you are doing research, consider these methods:
> 1. Change vma_merge() so it always fail to merge mappings
>
> or
>
> 2. Set up your "mappings duplicated in userspace" so
> they too merge in the same way.
>
> Helge Hafting
Hi!
Thanks for your answer, however you (too) misunderstood: Merging of the vma's
is not the problem and I understand why the kernel does so.
The real problem is that it deletes mappings it should not be deleting... in
my case, I've found out that a possible reason for the deletions is that the
stack tries to grow or other large areas of memory are allocated and my
mapping is taking up the memory that the kernel wants to reserve.
However, this has been dealt with in some other reply - I decided to simply
relocate my mappings to another memory area as soon as the kernel tries to
delete them.
This is not really all that efficient, but it does not matter right now in my
case.
Thanks again for your answer - I appreciate it!
On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
> On Thursday 30 August 2007 23:50:21 [email protected] wrote:
>> On Thu, 30 Aug 2007 23:41:09 +0200, Clemens Kolbitsch said:
>>> On Thursday 30 August 2007 23:34:52 you wrote:
>>>> On Thu, 30 Aug 2007, Clemens Kolbitsch wrote:
>>>>> is there no way to tell the kernel, that a certain mapping must not
>>>>> be removed, no matter what (except of course an explicit call to
>>>>> sys_unmap, of course)?
>>>>
>>>> I don't seem to get what is the issue here. Your mapping is not
>>>> removed, only the VMAs are merged together into one larger VMA if they
>>>> have neighbouring address ranges and compatible protection bits. See
>>>> vma_merge().
>>>
>>> the thing is that they are not. the kernel chooses to REPLACE my mapping.
>>>
>>> consider the user-space code:
>>>
>>> mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
>>> mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
>>>
>>> here, the second call to mmap will shorten the first mapping to 0x1000
>>> bytes and create one big vma with size 0x5000 bytes.
>>>
>>> is there a way to tell it that the second mmap MUST fail?
>>
>> There's an LSM exit point for mmap, you could perhaps do something there.
>>
>> What are you trying to achieve by forcing the second one to fail?
>
> puh... that is a good question :-)
>
> I'm writing my master's thesis on a new model of memory protection and need to
> have every memory mapping in userspace duplicated. I also have kind of a
> second PGD/PTD that allows finding this mirrored mapping.
>
> However, as the number of original mappings grows, I suddenly have the problem
> that the kernel tries to allocate a new mapping and picks the address of a
> mirrored memory page, which it shouldn't.
>
> Honestly, I don't understand why it does so, but it simply does... so
> basically I want these second mappings to stay in memory as long as the
> original page.
>
> What do you mean exactly with
>
>> There's an LSM exit point for mmap, you could perhaps do something there.
>
> ??
>
> By the way / @Jiri Kosina:
>
>> Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
>> - see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
>
> I know... I'm not even saying that this is a bug... I just wonder if there is
> some way to avoid this problem :-)
>
> Thanks for your help - i really appreciate that!!
You might need to keep track of your memory mappings and
not remap something that will overlap. You can mmap
a single page in which you write the addresses. If you
map it shared, all your processes can access it and
call a common memory-mapping procedure (that you write)
which keeps track of its kernel allocations for you.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.30 BogoMips).
My book : http://www.AbominableFirebug.com/
_
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.