2005-09-01 20:46:06

by Richard Hayden

[permalink] [raw]
Subject: A couple of OOM killer races

Hi all,

It appears there is no protection in badness() (called by
out_of_memory() for each process) when it reads p->mm->total_vm. Another
processor (or a kernel preemption) could presumably run do_exit and then
exit_mm, freeing the process in question's reference to its mm just
after the (!p->mm) check but before it reads p->mm->total_vm, making the
latter reference a null pointer reference.

Also there appears to be no protection when we set p->time_slice in
__oom_kill_task(). Am I right in thinking that this field should be
protected by the appropriate runqueue lock, at least this is what
scheduler_tick() seems to use?

Have I missed anything?

Best regards,

Richard Hayden.


___________________________________________________________
How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com


2005-09-02 01:30:34

by Zhou Yingchao

[permalink] [raw]
Subject: Re: A couple of OOM killer races

On 9/2/05, Richard Hayden <[email protected]> wrote:
> Hi all,
>
> It appears there is no protection in badness() (called by
> out_of_memory() for each process) when it reads p->mm->total_vm. Another
> processor (or a kernel preemption) could presumably run do_exit and then
> exit_mm, freeing the process in question's reference to its mm just
> after the (!p->mm) check but before it reads p->mm->total_vm, making the
> latter reference a null pointer reference.

In badness, the tasklist_lock has been hold. And when an exit signal
delived to it because other thread call do_group_exit, it need to hold
the tasklist_lock first, so we are protected.

--
Yingchao Zhou

2005-09-08 02:09:12

by Coywolf Qi Hunt

[permalink] [raw]
Subject: Re: A couple of OOM killer races

On 9/2/05, Richard Hayden <[email protected]> wrote:
> Hi all,
>
> It appears there is no protection in badness() (called by
> out_of_memory() for each process) when it reads p->mm->total_vm. Another
> processor (or a kernel preemption) could presumably run do_exit and then
> exit_mm, freeing the process in question's reference to its mm just
> after the (!p->mm) check but before it reads p->mm->total_vm, making the
> latter reference a null pointer reference.

We have read_lock(&tasklist_lock); .

>
> Also there appears to be no protection when we set p->time_slice in
> __oom_kill_task(). Am I right in thinking that this field should be
> protected by the appropriate runqueue lock, at least this is what
> scheduler_tick() seems to use?

ditto

--
Coywolf Qi Hunt
http://sosdg.org/~coywolf/