2006-05-07 19:31:39

by Andries E. Brouwer

[permalink] [raw]
Subject: a Linux swap storm

Earlier this evening I showed someone some pictures under X:

% display -size 300x300 *.jpg

(395 pictures, 315 MB). When display (from ImageMagick)
started to repeat, I exited the program.
At this moment the machine became unusable for twenty minutes
of solid disk activity.
No keystroke seen, not even the Ctrl-Alt-Backspace to kill X,
or Ctrl-Alt-F1 to switch consoles, no mouse movement seen,
vmstat did not produce any output for twenty minutes.

The vmstat 5 output was

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa

0 3 315728 3164 4424 3356 646 382 792 382 2425 182 0 5 0 95
1 1 316356 3856 4428 3572 454 366 507 366 2242 150 0 5 0 95
0 1 317540 3784 4456 3664 530 578 590 578 2403 179 0 7 0 93
0 1 306740 4240 4524 5372 127013 49878 129427 50061 405016 32901 0 4 0 95
1 1 306712 3992 4536 5372 30 0 30 3 450 122 2 1 94 2
0 0 306692 4016 4548 5372 18 0 18 3 402 134 2 2 96 1
0 0 306692 4016 4560 5372 0 0 0 3 257 35 1 1 98 0

The machine is vanilla 2.6.14, 256MB, 550MB swap.

% rpm -qf `which X`
xorg-x11-server-6.8.2-100

I wonder what precisely happened. Is this an X bug? Or a kernel bug?
The effect is reproducible.

Andries


2006-05-07 20:17:39

by Jan Engelhardt

[permalink] [raw]
Subject: Re: a Linux swap storm


>% display -size 300x300 *.jpg

>I wonder what precisely happened. Is this an X bug? Or a kernel bug?
>The effect is reproducible.

Right before it starts to swap, hit Ctrl+Z to suspend 'display' and do a
`ps u`. Does it hog memory?


Jan Engelhardt
--

2006-05-07 20:37:36

by Jesper Juhl

[permalink] [raw]
Subject: Re: a Linux swap storm

On 5/7/06, [email protected] <[email protected]> wrote:
[snip]
> At this moment the machine became unusable for twenty minutes
[snip]
> The machine is vanilla 2.6.14, 256MB, 550MB swap.
[snip]
> The effect is reproducible.
>
[snip]

Reproducible - nice.
Have you tried it with a newer kernel like 2.6.16.14 or
2.6.17-rc3-git13 ? Could be interresting to see if it has already
been fixed since 2.6.14... in any case, it's one more datapoint :-)


--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html

2006-05-08 03:23:22

by Mike Galbraith

[permalink] [raw]
Subject: Re: a Linux swap storm

On Sun, 2006-05-07 at 21:31 +0200, [email protected] wrote:
> Earlier this evening I showed someone some pictures under X:
>
> % display -size 300x300 *.jpg
>
> (395 pictures, 315 MB). When display (from ImageMagick)
> started to repeat, I exited the program.
> At this moment the machine became unusable for twenty minutes
> of solid disk activity.
> No keystroke seen, not even the Ctrl-Alt-Backspace to kill X,
> or Ctrl-Alt-F1 to switch consoles, no mouse movement seen,
> vmstat did not produce any output for twenty minutes.
>
> The vmstat 5 output was
>
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id wa
>
> 0 3 315728 3164 4424 3356 646 382 792 382 2425 182 0 5 0 95
> 1 1 316356 3856 4428 3572 454 366 507 366 2242 150 0 5 0 95
> 0 1 317540 3784 4456 3664 530 578 590 578 2403 179 0 7 0 93
> 0 1 306740 4240 4524 5372 127013 49878 129427 50061 405016 32901 0 4 0 95
> 1 1 306712 3992 4536 5372 30 0 30 3 450 122 2 1 94 2
> 0 0 306692 4016 4548 5372 18 0 18 3 402 134 2 2 96 1
> 0 0 306692 4016 4560 5372 0 0 0 3 257 35 1 1 98 0

This is after ImageMagic exited? If so, and you don't have a userland
hog sitting on that memory, I'd suggest posting /proc/meminfo and any
part of /proc/slabinfo showing large numbers of allocations. (if you
can repeat with latest vanilla kernel that is)

> The machine is vanilla 2.6.14, 256MB, 550MB swap.
>
> % rpm -qf `which X`
> xorg-x11-server-6.8.2-100
>
> I wonder what precisely happened. Is this an X bug? Or a kernel bug?
> The effect is reproducible.

I'd lean toward kernel. If it was thrashing so hard that the box became
a doorstop for 20 minutes, seems to me that's a fine description of oom,
so somebody should have been killed. Does SysRq-M work during the
seizure?

-Mike