2010-07-15 21:48:51

by Xianghua Xiao

[permalink] [raw]
Subject: anonymous mmap() and random heap allocation

It seems mmap() can not do random allocation for malloc()/heap on
Linux, anyone used pax's 'mmap randonness' option for that? what's the
price for that?

I want to port openbsd's malloc() to linux to avoid heap
crashes(overrun/underrun), openbsd's malloc() can do random allocation
for security reasons, meanwhile it helps to avoid some
overrun/underrun crashes with no extra cost, the latter is what I'm
looking for.

thanks,
xianghua


2010-07-16 09:59:45

by Mikael Pettersson

[permalink] [raw]
Subject: Re: anonymous mmap() and random heap allocation

Xianghua Xiao writes:
> It seems mmap() can not do random allocation for malloc()/heap on
> Linux, anyone used pax's 'mmap randonness' option for that? what's the
> price for that?
>
> I want to port openbsd's malloc() to linux to avoid heap
> crashes(overrun/underrun), openbsd's malloc() can do random allocation
> for security reasons, meanwhile it helps to avoid some
> overrun/underrun crashes with no extra cost, the latter is what I'm
> looking for.

You can implement this in user-space with existing kernel features.
1: place guard pages around mmap():ed data
2: parse /proc/self/maps and mmap(MAP_FIXED)
3: use a "safe" compiler or a dynamic binary instrumenter
4: use x86 segments (ugh!)
5: use a safe programming language
...

2010-07-16 13:35:39

by Xianghua Xiao

[permalink] [raw]
Subject: Re: anonymous mmap() and random heap allocation

On Fri, Jul 16, 2010 at 4:59 AM, Mikael Pettersson <[email protected]> wrote:
> Xianghua Xiao writes:
>  > It seems mmap() can not do random allocation for malloc()/heap on
>  > Linux, anyone used pax's 'mmap randonness' option for that? what's the
>  > price for that?
>  >
>  > I want to port openbsd's malloc() to linux to avoid heap
>  > crashes(overrun/underrun), openbsd's malloc() can do random allocation
>  > for security reasons, meanwhile it helps to avoid some
>  > overrun/underrun crashes with no extra cost, the latter is what I'm
>  > looking for.
>
> You can implement this in user-space with existing kernel features.
> 1: place guard pages around mmap():ed data
> 2: parse /proc/self/maps and mmap(MAP_FIXED)
> 3: use a "safe" compiler or a dynamic binary instrumenter
> 4: use x86 segments (ugh!)
> 5: use a safe programming language
> ...
>

Mikael,
Thanks.
Forgot to mention this is a powerpc platform. This will be enhancement
to an existing system, i.e. language(mostly C) is already set.
Also, this is for runtime, placing guard pages around mmap() will slow
down the system dramatically.
I'm porting OpenBSD's malloc now. Even though linux's mmap(), unlike
OpenBSD, can not randomize its allocations(which offers free
overrun/underrun protections to some extent), however OpenBSD's malloc
does not put its control block on the heap, that's a plus at least.

What do you mean by 'safe' compiler, something like checkergcc? I
think checkergcc is obsolete now.

cheers,
xianghua