2012-05-07 15:54:39

by Robert Święcki

[permalink] [raw]
Subject: mmap/clone returns ENOMEM with lots of free memory

Hi,

I'm fuzzing 3.4.0-rc5 (mostly with
http://code.google.com/p/iknowthis/), and got to the point that new
fuzzing threads/processes don't want to run any more. I have script
that periodically sends SIGCONT to all processes.

while [ 1 ]; do su test -c 'kill -CONT -1'; su test2 -c 'kill -CONT
-1'; su nobody -c 'kill -CONT -1'; sleep 300; done

It doesn't work:

root@ise-test:~/kern-fuz# ./cont.sh
su: Cannot fork user shell
su: Cannot fork user shell
su: Cannot fork user shell

root@ise-test:~/kern-fuz# strace -e mmap,clone su test -c 'kill -CONT
-1' 2>&1 | grep "= \-1"
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7fadf334f9f0) = -1 ENOMEM (Cannot allocate memory)
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = -1 ENOMEM (Cannot allocate memory)

It happens with every user (even newly created one), so it doesn't
seem to be any per-id rlimit.

root@ise-test:~/kern-fuz# su - test3
su: Cannot fork user shell
root@ise-test:~/kern-fuz# su - test3
$ <-- success
root@ise-test:~/kern-fuz# su - test3
su: Cannot fork user shell
root@ise-test:~/kern-fuz# su - test3
su: Cannot fork user shell
root@ise-test:~/kern-fuz# su - test3
su: Cannot fork user shell

Not sure how to debug it yet. I can run kdb/kgdb on it, but before I
dive into mm structures, I though I attach some proc files from your
entertainment, maybe you can spot anything interesting there.

Also, whatever happened on this machine, i.e. any syscall during
fuzzing, was invoked from non-root user.

--
Robert Święcki


Attachments:
_proc_buddyinfo.txt (300.00 B)
_proc_config.txt (106.68 kB)
_proc_meminfo.txt (1.14 kB)
_proc_slabinfo.txt (32.09 kB)
_proc_vmallocinfo.txt (7.48 kB)
_proc_vmstat.txt (2.01 kB)
ps_waux (10.64 kB)
top.txt (4.25 kB)
ulimit_a.txt (692.00 B)
Download all attachments
Subject: Re: mmap/clone returns ENOMEM with lots of free memory

On Mon, 7 May 2012, Robert Święcki wrote:

> root@ise-test:~/kern-fuz# ./cont.sh
> su: Cannot fork user shell
> su: Cannot fork user shell
> su: Cannot fork user shell
>
> root@ise-test:~/kern-fuz# strace -e mmap,clone su test -c 'kill -CONT
> -1' 2>&1 | grep "= \-1"
> clone(child_stack=0,
> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x7fadf334f9f0) = -1 ENOMEM (Cannot allocate memory)
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = -1 ENOMEM (Cannot allocate memory)

Hmmm... That looks like some maximum virtual memory limit was violated.

Check ulimit and the overcommit settings (see /proc/meminfo's commitlimit
etc)

2012-05-07 21:41:53

by Robert Święcki

[permalink] [raw]
Subject: Re: mmap/clone returns ENOMEM with lots of free memory

On Mon, May 7, 2012 at 10:15 PM, Christoph Lameter <[email protected]> wrote:
> On Mon, 7 May 2012, Robert Święcki wrote:
>
>> root@ise-test:~/kern-fuz# ./cont.sh
>> su: Cannot fork user shell
>> su: Cannot fork user shell
>> su: Cannot fork user shell
>>
>> root@ise-test:~/kern-fuz# strace -e mmap,clone su test -c 'kill -CONT
>> -1' 2>&1 | grep "= \-1"
>> clone(child_stack=0,
>> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
>> child_tidptr=0x7fadf334f9f0) = -1 ENOMEM (Cannot allocate memory)
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = -1 ENOMEM (Cannot allocate memory)
>
> Hmmm... That looks like some maximum virtual memory limit was violated.
>
> Check ulimit and the overcommit settings (see /proc/meminfo's commitlimit
> etc)

Yup (btw: I attached dump of some proc files and some debug commands
in the original e-mail - can be found here
http://marc.info/?l=linux-kernel&m=133640623421007&w=2 in case some
MTA removed them)

CommitLimit: 1981528 kB
Committed_AS: 1916788 kB

just not sure if Committed_AS should present this kind of value. Did I
just hit a legitimate condition, or may it suggest a bug? I'm a bit
puzzled cause

root@ise-test:/proc# grep Mem /proc/meminfo
MemTotal: 3963060 kB
MemFree: 3098324 kB

Also, some sysctl values:
vm.overcommit_memory = 2
vm.overcommit_ratio = 50

--
Robert Święcki

Subject: Re: mmap/clone returns ENOMEM with lots of free memory

On Mon, 7 May 2012, Robert Święcki wrote:

> Yup (btw: I attached dump of some proc files and some debug commands
> in the original e-mail - can be found here
> http://marc.info/?l=linux-kernel&m=133640623421007&w=2 in case some
> MTA removed them)
>
> CommitLimit: 1981528 kB
> Committed_AS: 1916788 kB
>
> just not sure if Committed_AS should present this kind of value. Did I
> just hit a legitimate condition, or may it suggest a bug? I'm a bit
> puzzled cause

This is a legitimate condition. No bug.
>
> root@ise-test:/proc# grep Mem /proc/meminfo
> MemTotal: 3963060 kB
> MemFree: 3098324 kB

Physical memory is free in quantity but virtual memory is exhausted.

> Also, some sysctl values:
> vm.overcommit_memory = 2
> vm.overcommit_ratio = 50

Setting overcommit memory to 2 means that the app is strictly policed
for staying within bounds on virtual memory. Dont do that.

See linux source linux/Documentation/vm/overcommit-accounting for more
details.

2012-05-08 14:47:38

by Robert Święcki

[permalink] [raw]
Subject: Re: mmap/clone returns ENOMEM with lots of free memory

On Tue, May 8, 2012 at 4:02 PM, Christoph Lameter <[email protected]> wrote:
> On Mon, 7 May 2012, Robert Święcki wrote:
>
>> Yup (btw: I attached dump of some proc files and some debug commands
>> in the original e-mail - can be found here
>> http://marc.info/?l=linux-kernel&m=133640623421007&w=2 in case some
>> MTA removed them)
>>
>> CommitLimit:     1981528 kB
>> Committed_AS:    1916788 kB
>>
>> just not sure if Committed_AS should present this kind of value. Did I
>> just hit a legitimate condition, or may it suggest a bug? I'm a bit
>> puzzled cause
>
> This is a legitimate condition. No bug.
>>
>> root@ise-test:/proc# grep Mem /proc/meminfo
>> MemTotal:        3963060 kB
>> MemFree:         3098324 kB
>
> Physical memory is free in quantity but virtual memory is exhausted.
>
>> Also, some sysctl values:
>> vm.overcommit_memory = 2
>> vm.overcommit_ratio = 50
>
> Setting overcommit memory to 2 means that the app is strictly policed
> for staying within bounds on virtual memory. Dont do that.
>
> See linux source linux/Documentation/vm/overcommit-accounting for more
> details.

Thanks Christoph.

--
Robert Święcki

2012-05-08 15:29:06

by Alan

[permalink] [raw]
Subject: Re: mmap/clone returns ENOMEM with lots of free memory

> Setting overcommit memory to 2 means that the app is strictly policed
> for staying within bounds on virtual memory. Dont do that.

For a fuzz test you probably do want it at 2 to avoid the box dying in a
swap storm.


Alan