2000-11-15 21:36:52

by pavel-velo

[permalink] [raw]
Subject: RE: KPATCH] Reserve VM for root (was: Re: Looking for better VM)

Hi!

>I've also never said OOM killer should be disabled. In theory the
>non-overcommitting systems deadlock, Linux survives. Ironically
>usually it's just the opposite in practice. Any user can
>deadlock/crash Linux [default install, no quotas] but not an
>non-overcommitting system [root can clean up]. Here is an example code
>"simulating" a leaking daemon that will "deadlock" Linux even with
>your OOM killer patch [that is anyway *MUCH* better than the actually
>non-existing one in 2.2.x kernels]:
>
>main() { while(1) if (fork()) malloc(1); }
>
>With the patch below I could ssh to the host and killall the offending
>processes. To enable reserving VM space for root do

what about main() { while(1) system("ftp localhost &"); }

This. or so,ething similar should allow you to kill your machine even with your
patch from normal user account

Pavel


2000-11-16 14:21:05

by Szabolcs Szakacsits

[permalink] [raw]
Subject: RE: KPATCH] Reserve VM for root (was: Re: Looking for better VM)


On Wed, 1 Jan 1997 [email protected] wrote:

> >main() { while(1) if (fork()) malloc(1); }
> >With the patch below I could ssh to the host and killall the offending
> >processes. To enable reserving VM space for root do
> what about main() { while(1) system("ftp localhost &"); }
> This. or so,ething similar should allow you to kill your machine
> even with your patch from normal user account

This or something similar didn't kill the box [I've tried all local
DoS from Packetstorm that I could find]. Please send a working
example. Of course probably it's possible to trigger root owned
processes to eat memory eagerly by user apps but that's a problem in
the process design running as root and not a kernel issue.

Note, I'm not discussing "local user can kill the box without limits",
I say Linux "deadlocks" [it starts its own autonom life and usually
your only chance is to hit the reset button] when there is continuous
VM pressure by user applications. If you think fork() kills the box
then ulimit the maximum number of user processes (ulimit -u). This is
a different issue and a bad design in the scheduler (see e.g. Tru64
for a better one).

BTW, I have a new version of the patch with that Linux behaves much
better from root's point of view when the memory is more significantly
overcommited. I'll post it if I have time [and there is interest].

Szaka

2000-11-16 16:21:56

by Rik van Riel

[permalink] [raw]
Subject: RE: KPATCH] Reserve VM for root (was: Re: Looking for better VM)

On Thu, 16 Nov 2000, Szabolcs Szakacsits wrote:

[snip exploit that really shouldn't take Linux down]

> This or something similar didn't kill the box [I've tried all local
> DoS from Packetstorm that I could find]. Please send a working
> example. Of course probably it's possible to trigger root owned
> processes to eat memory eagerly by user apps but that's a problem in
> the process design running as root and not a kernel issue.

Not necessarily, but your patch will probably make a difference
for quite a number of people...

> If you think fork() kills the box then ulimit the maximum number
> of user processes (ulimit -u). This is a different issue and a
> bad design in the scheduler (see e.g. Tru64 for a better one).

My fair scheduler catches this one just fine. It hasn't
been integrated in the kernel yet, but both VA Linux and
Conectiva use it in their kernel RPM.

> BTW, I have a new version of the patch with that Linux behaves
> much better from root's point of view when the memory is more
> significantly overcommited. I'll post it if I have time [and
> there is interest].

There is interest, believe me ;)

While this is not one of the sexy new kernel
features, this will help quite a few system
administrators and is destined to a long and
healthy life inside kernel RPMs, maybe even
in the main kernel tree (when 2.5 splits?).

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

2000-11-16 16:34:33

by Christoph Hellwig

[permalink] [raw]
Subject: Re: KPATCH] Reserve VM for root (was: Re: Looking for better VM)

On Thu, Nov 16, 2000 at 01:51:01PM -0200, Rik van Riel wrote:
> > If you think fork() kills the box then ulimit the maximum number
> > of user processes (ulimit -u). This is a different issue and a
> > bad design in the scheduler (see e.g. Tru64 for a better one).
>
> My fair scheduler catches this one just fine. It hasn't
> been integrated in the kernel yet, but both VA Linux and
> Conectiva use it in their kernel RPM.

BTW: do you have a fairsched patch for 2.4?

Christoph

--
Always remember that you are unique. Just like everyone else.

2000-11-16 16:36:22

by Rik van Riel

[permalink] [raw]
Subject: Re: KPATCH] Reserve VM for root (was: Re: Looking for better VM)

On Thu, 16 Nov 2000, Christoph Hellwig wrote:
> On Thu, Nov 16, 2000 at 01:51:01PM -0200, Rik van Riel wrote:
> > > If you think fork() kills the box then ulimit the maximum number
> > > of user processes (ulimit -u). This is a different issue and a
> > > bad design in the scheduler (see e.g. Tru64 for a better one).
> >
> > My fair scheduler catches this one just fine. It hasn't
> > been integrated in the kernel yet, but both VA Linux and
> > Conectiva use it in their kernel RPM.
>
> BTW: do you have a fairsched patch for 2.4?

I haven't updated it yet to the latest kernels yet...

[but I, or someone else, should ;)]

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

2000-11-16 20:59:54

by Szabolcs Szakacsits

[permalink] [raw]
Subject: RE: KPATCH] Reserve VM for root (was: Re: Looking for better VM)


On Thu, 16 Nov 2000, Rik van Riel wrote:
> On Thu, 16 Nov 2000, Szabolcs Szakacsits wrote:
> [snip exploit that really shouldn't take Linux down]

I don't really consider it as an exploit. It's a kind of workload
that's optimized for fast testing simulating many busy user daemons
(e.g. dynamically generating web pages). Everybody knows Slashdot
effect. A system was designed for a workload according to a budget and
other factors. But immediately the load gets *much* higher as it was
ever expected, the system starts to trash and nobody can login or
start new processes. You can pull off the cable but if it's a remote
box then you are really in a bad situation. Or if a local [e.g.
computing] batch job run away you also must hit the reset button.
Happens too many times that it should be really taken seriously now,
who don't believe should just search for typical OOM crash patterns of
user reports on different mailling lists/newsgroups.

> > This or something similar didn't kill the box [I've tried all local
> > DoS from Packetstorm that I could find]. Please send a working
> > example. Of course probably it's possible to trigger root owned
> > processes to eat memory eagerly by user apps but that's a problem in
> > the process design running as root and not a kernel issue.
> Not necessarily, but your patch will probably make a difference
> for quite a number of people...

Could you please explain what you mean? ;) I saw only ONE difference.
The system remains usable for root but not anybody else. Everything
else is the same as before. Of course I think there are still problems
with the patch but to be honest I don't know what they are ... except
those I wrote before -- e.g. the latest, not yet released version
definitely doesn't work with your OOM killer [system just trashes].
Can you explain why you put process killing in do_try_to_free_pages()
instead of the original place, do_page_fault()? I guess putting it in
do_page_fault() [if possible] would fix my current problem.

> > If you think fork() kills the box then ulimit the maximum number
> > of user processes (ulimit -u). This is a different issue and a
> > bad design in the scheduler (see e.g. Tru64 for a better one).
> My fair scheduler catches this one just fine. It hasn't
> been integrated in the kernel yet, but both VA Linux and
> Conectiva use it in their kernel RPM.

I know about two fair schedulers for Linux, one of them is yours but
I couldn't try them out yet. Anyway definitely a must ;)

> While this is not one of the sexy new kernel
> features, this will help quite a few system
> administrators and is destined to a long and
> healthy life inside kernel RPMs, maybe even
> in the main kernel tree (when 2.5 splits?).

Thanks for the feedback,

Szaka

2000-11-18 22:32:18

by Pavel Machek

[permalink] [raw]
Subject: Re: KPATCH] Reserve VM for root (was: Re: Looking for better VM)

Hi!

> > >main() { while(1) if (fork()) malloc(1); }
> > >With the patch below I could ssh to the host and killall the offending
> > >processes. To enable reserving VM space for root do
> > what about main() { while(1) system("ftp localhost &"); }
> > This. or so,ething similar should allow you to kill your machine
> > even with your patch from normal user account
>
> This or something similar didn't kill the box [I've tried all local
> DoS from Packetstorm that I could find]. Please send a working

Sorry, I did not have working example, just feeling that something
like that should be possible.

> Note, I'm not discussing "local user can kill the box without limits",
> I say Linux "deadlocks" [it starts its own autonom life and usually
> your only chance is to hit the reset button] when there is continuous
> VM pressure by user applications. If you think fork() kills the box

That's clear bug, right? It should not deadlock, it should go to
OOM-killer and kill someone.

> BTW, I have a new version of the patch with that Linux behaves much
> better from root's point of view when the memory is more significantly
> overcommited. I'll post it if I have time [and there is interest].

There is interest. Yesterday atrey died due userland process eating
all memory.
Pavel
PS: atrey is machine that gets my mail, so it is kind of important to
me.
--
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]