2005-04-28 10:29:50

by Hubert Tonneau

[permalink] [raw]
Subject: 2.6.12-rc3 mmap lack of consistency among runs

As a way to freeze then restart processes,
the first shot of the process calls 'mmap' with NULL as 'start',
then restarts of the process will call 'mmap' with the value received at the
first shot, and expect to be assigned the requested area.

This used to work perfectly with 2.6.11 and all previous kernels (unless some
shared libraries have been upgraded in the mean time),
but with 2.6.12-rc3 (I have not tested rc1 and rc2) it fails half time.

I can solve the problem through specifying a 'start' value at the first shot,
but then I will get a more serious problem on the long run because the
application would then have to be awared of the general layout of the address
space enforced by the kernel and so could be disturbed by any change.


2005-04-29 12:48:08

by Andrew Morton

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

Hubert Tonneau <[email protected]> wrote:
>
> As a way to freeze then restart processes,
> the first shot of the process calls 'mmap' with NULL as 'start',
> then restarts of the process will call 'mmap' with the value received at the
> first shot, and expect to be assigned the requested area.
>
> This used to work perfectly with 2.6.11 and all previous kernels (unless some
> shared libraries have been upgraded in the mean time),
> but with 2.6.12-rc3 (I have not tested rc1 and rc2) it fails half time.
>
> I can solve the problem through specifying a 'start' value at the first shot,
> but then I will get a more serious problem on the long run because the
> application would then have to be awared of the general layout of the address
> space enforced by the kernel and so could be disturbed by any change.

Maybe you're being bitten by the address space randomisation.

Try
echo 0 > /proc/sys/kernel/randomize_va_space

2005-04-29 13:15:09

by Hubert Tonneau

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

Andrew Morton wrote:
>
> Maybe you're being bitten by the address space randomisation.
>
> Try
> echo 0 > /proc/sys/kernel/randomize_va_space

Ok, it solves my issue, but:

. desabling it through 'echo 0 > /proc/sys/kernel/randomize_va_space' is not
a solution because only the application knows that it wants it to be desabled,
and the application is not root so cannot write to /proc; morever the
application can only speak for itself so desabling should be on a per process
bias.

I can hardly imagine to publish a warning in the README such as:
This software only works if your Linux kernel is configured so that
/proc/sys/kernel/randomize_va_space = 0

. second, my process restart succeeding roughly in 50% cases means that the
randomisation performed is just a toy. A virus assuming fixed memory layout
will still succeed 50% of times to install.

All in all, I'm not concerned about Linux kernel to randomise or not,
but I need to have a reliable way to request a memory region and be granted
that I can request the same one in a futur run.
What is the proper way to get such a memory area ?

2005-04-29 13:26:56

by Andrew Morton

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

Hubert Tonneau <[email protected]> wrote:
>
> Andrew Morton wrote:
> >
> > Maybe you're being bitten by the address space randomisation.
> >
> > Try
> > echo 0 > /proc/sys/kernel/randomize_va_space
>
> Ok, it solves my issue, but:
>
> . desabling it through 'echo 0 > /proc/sys/kernel/randomize_va_space' is not
> a solution because only the application knows that it wants it to be desabled,
> and the application is not root so cannot write to /proc; morever the
> application can only speak for itself so desabling should be on a per process
> bias.

You can disable randomization on a per-executable basis by setting an ELF
personality. I forget the magic incantation. Arjan?

> I can hardly imagine to publish a warning in the README such as:
> This software only works if your Linux kernel is configured so that
> /proc/sys/kernel/randomize_va_space = 0
>
> . second, my process restart succeeding roughly in 50% cases means that the
> randomisation performed is just a toy. A virus assuming fixed memory layout
> will still succeed 50% of times to install.

Dunno.

> All in all, I'm not concerned about Linux kernel to randomise or not,
> but I need to have a reliable way to request a memory region and be granted
> that I can request the same one in a futur run.
> What is the proper way to get such a memory area ?

MAP_FIXED?

2005-04-29 13:35:09

by Arjan van de Ven

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

On Fri, Apr 29, 2005 at 06:20:53AM -0700, Andrew Morton wrote:
> > application can only speak for itself so desabling should be on a per process
> > bias.
>
> You can disable randomization on a per-executable basis by setting an ELF
> personality. I forget the magic incantation. Arjan?

setarch -R

>
> > I can hardly imagine to publish a warning in the README such as:
> > This software only works if your Linux kernel is configured so that
> > /proc/sys/kernel/randomize_va_space = 0
> >
> > . second, my process restart succeeding roughly in 50% cases means that the
> > randomisation performed is just a toy. A virus assuming fixed memory layout
> > will still succeed 50% of times to install.
>
> Dunno.

It just means that half the time the old value was below the current
boundary, and half the time above. Eg half the time it was in free
space and you succeeded but left a gap, the other half there was a conflict.
Says nothing about the value of randomisation...


>
> > All in all, I'm not concerned about Linux kernel to randomise or not,
> > but I need to have a reliable way to request a memory region and be granted
> > that I can request the same one in a futur run.
> > What is the proper way to get such a memory area ?
>
> MAP_FIXED?

MAP_FIXED is generally a really bad idea though.

2005-04-29 14:58:42

by Hubert Tonneau

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

Arjan van de Ven wrote:
>
> > You can disable randomization on a per-executable basis by setting an ELF
> > personality. I forget the magic incantation. Arjan?
>
> setarch -R

I had no success with it:
/usr/src/setarch-1.7/setarch i386 -R /pliant/fullpliant

I even tried adding the following instruction at the very beginning of my
C program, with no more success:
personality(0x0040000); // ADDR_NO_RANDOMIZE

Basically, the behaviour is not changed, as opposed to if I do:
echo 0 >/proc/sys/kernel/randomize_va_space

> > . second, my process restart succeeding roughly in 50% cases means that the
> > randomisation performed is just a toy. A virus assuming fixed memory layout
> > will still succeed 50% of times to install.
>
> It just means that half the time the old value was below the current
> boundary, and half the time above. Eg half the time it was in free
> space and you succeeded but left a gap, the other half there was a conflict.
> Says nothing about the value of randomisation...

Understood.

> > All in all, I'm not concerned about Linux kernel to randomise or not,
> > but I need to have a reliable way to request a memory region and be granted
> > that I can request the same one in a futur run.
> > What is the proper way to get such a memory area ?
>
> > MAP_FIXED?
>
> MAP_FIXED is generally a really bad idea though.

If I replace
PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0
with
PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0
the call just fails.

2005-04-29 19:06:35

by Hubert Tonneau

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

Hubert Tonneau wrote:
>
> I even tried adding the following instruction at the very beginning of my
> C program, with no more success:
> personality(0x0040000); // ADDR_NO_RANDOMIZE
>
> Basically, the behaviour is not changed, as opposed to if I do:
> echo 0 >/proc/sys/kernel/randomize_va_space

I believe that I understand why calling 'personality' does not work:
it has to be called before the process is loaded as far as I could understand
the Linux source code.

So, at the moment, the only two effective workarounds I'm awared of are:
. switch to calling 'mmap' with specified requested address right from the
beginning (also I find it very dangerous over the long run)
. call 'mmap' to allocate (waste) 1 MB of address space when the process
runs for the first time, so that I'm granted that subsequente 'mmap'
will allocate from a memory area that is always available from run to run

Also about the second solution (the one that I've included in Pliant)
I have two concernes:
. first it's ugly (should I include it in Posix OS agnostic version of
Pliant or just declare that Linux cannot run the generic version anymore)
. second, and most important one, I have experimentaly determined that 1 MB
is the minimum address space to allocate (waste), but I could not understand
where it comes from reading the Kernel source, and I don't understand how
stable will this value be over time:

unsigned long arch_align_stack(unsigned long sp)
{
if (randomize_va_space)
sp -= get_random_int() % 8192;
return sp & ~0xf;
}

2005-04-30 00:39:25

by Arjan van de Ven

[permalink] [raw]
Subject: Re: 2.6.12-rc3 mmap lack of consistency among runs

On Fri, 2005-04-29 at 12:44 +0000, Hubert Tonneau wrote:
> Andrew Morton wrote:
> >
> > Maybe you're being bitten by the address space randomisation.
> >
> > Try
> > echo 0 > /proc/sys/kernel/randomize_va_space
>
> Ok, it solves my issue, but:
>
> . desabling it through 'echo 0 > /proc/sys/kernel/randomize_va_space' is not
> a solution because only the application knows that it wants it to be desabled,
> and the application is not root so cannot write to /proc; morever the
> application can only speak for itself so desabling should be on a per process
> bias.

there is the setarch command that you can use to disable it on a per
process basis.

> . second, my process restart succeeding roughly in 50% cases means that the
> randomisation performed is just a toy. A virus assuming fixed memory layout
> will still succeed 50% of times to install.

actually no.
It just means that half the time the old value was below the current
boundary, and half the time above. Eg half the time it was in free
space.


> All in all, I'm not concerned about Linux kernel to randomise or not,
> but I need to have a reliable way to request a memory region and be granted
> that I can request the same one in a futur run.

glibc also has some randomisations in places (for cache performance) as
did kernels before 2.6.11 on p4 hyperthreading machines (granted that
was very small randomisation)

The only reliable ways I can think of is to either take a predetermined
address, or to mmap a big chunk first, then your real chunk and then
free the first chunk. Neither are clean or pretty