Forgive me if I don't provide enough information just yet, or am not
clear enough. I simply don't know what setting to tweak.
I'll explain.
In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
had notice my system, when using X mainly, got terribly slow after some
use. It surprised me that when I tried 2.5.47 this did not happen at
all, since I thought my problem was a lack of memory - the system has
384MB -.
Hence I tried to find where the difference was. What I found is that
2.4.20 kernels - 2.4.19 does the same -, was swapping just too much,
while there was a lot free memory on the system, cached but free.
I disabled all swap and it suddenly began to work smoothly again, yet
with the random kills when memory was a scarce resource on the system.
I've tried different sysctl's vm.overcommit settings but the result is
the same.
I also found a 2.4.x kernel which did not show this behavior, WOLK, in
any version I tried.
Could you please point me toward something I can try tweaking, or some
documentation to read which explains what I can change, unless it's some
kind of kernel problem?
BTW, aa kernels behaved somewhat better on this, only that the last one
I tried -rc2aa1- had some stability problems.
I can provide you with dmesg, /proc/meminfo or whatever might be useful.
Thanks in advance :)
--
Javier Marcet <[email protected]>
I just wanted to say that I experience the same thing using 2.4.20-rc1-ac3.
When I was using 2.4.19-pre9-ac3 I didn't notice the same behavior.
Christer
> In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
> had notice my system, when using X mainly, got terribly slow after some
> use. It surprised me that when I tried 2.5.47 this did not happen at
> all, since I thought my problem was a lack of memory - the system has
> 384MB -.
Same thing happens with RedHat 8.0's latest 2.4.18-18 kernel.
Box has been up for almost 8 days.
top:
Mem: 1031036K av, 1021140K used, 9896K free, 0K shrd, 80560K
buff
Swap: 1052216K av, 5064K used, 1047152K free 627808K
cached
vmstat:
procs memory swap io system
cpu
r b w swpd free buff cache si so bi bo in cs us sy
id
0 0 0 5064 10192 80860 626932 0 0 2 21 3 56 0 0
36
Swapping in these circumstances shows a pathological reluctance to shed
cached pages.
Not critical, but hardly optimal behaviour. Or is it?
Phil
---------------------------------------------
Phil Randal
Network Engineer
Herefordshire Council
Hereford, UK
> -----Original Message-----
> From: Javier Marcet [mailto:[email protected]]
> Sent: 29 November 2002 11:54
> To: [email protected]
> Subject: Exaggerated swap usage
>
>
>
> Forgive me if I don't provide enough information just yet, or am not
> clear enough. I simply don't know what setting to tweak.
>
> I'll explain.
> In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
> had notice my system, when using X mainly, got terribly slow
> after some
> use. It surprised me that when I tried 2.5.47 this did not happen at
> all, since I thought my problem was a lack of memory - the system has
> 384MB -.
> Hence I tried to find where the difference was. What I found is that
> 2.4.20 kernels - 2.4.19 does the same -, was swapping just too much,
> while there was a lot free memory on the system, cached but free.
> I disabled all swap and it suddenly began to work smoothly again, yet
> with the random kills when memory was a scarce resource on the system.
> I've tried different sysctl's vm.overcommit settings but the result is
> the same.
>
> I also found a 2.4.x kernel which did not show this behavior, WOLK, in
> any version I tried.
>
> Could you please point me toward something I can try tweaking, or some
> documentation to read which explains what I can change,
> unless it's some
> kind of kernel problem?
>
> BTW, aa kernels behaved somewhat better on this, only that
> the last one
> I tried -rc2aa1- had some stability problems.
>
> I can provide you with dmesg, /proc/meminfo or whatever might
> be useful.
>
> Thanks in advance :)
>
>
> --
> Javier Marcet <[email protected]>
>
On Fri, 29 Nov 2002, Randal, Phil wrote:
> Mem: 1031036K av, 1021140K used, 9896K free, 0K shrd, 80560K buff
> Swap: 1052216K av, 5064K used, 1047152K free 627808K cached
> Swapping in these circumstances shows a pathological reluctance to shed
> cached pages.
You've got 1 GB of RAM and 5 MB of pages in swap. That's an
almost negligable quantity.
Also, if vmstat doesn't show any swapins, the data is just
sitting there in swap without ever being used ... quite
possible since many distros start up daemons that people
don't really use.
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://guru.conectiva.com/
Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
On Fri, 29 Nov 2002, Javier Marcet wrote:
> In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
> had notice my system, when using X mainly, got terribly slow after some
> use.
First, lets get one thing straight: the problem is the slowness,
not necessarily the swap usage. It is very easy to jump to wrong
conclusions, so lets keep focussed on that part of the problem
which we know to be true before we all start hacking on parts of
the system which aren't related to the slowness...
> I can provide you with dmesg, /proc/meminfo or whatever might be useful.
How about some output (maybe 20 lines) of 'vmstat 1' during the
time where your system is slow, together with a short description
of how large the various processes (X, Mozilla ...) in your system
are ?
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://guru.conectiva.com/
Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
* Rik van Riel <[email protected]> [021129 21:00]:
>> In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
>> had notice my system, when using X mainly, got terribly slow after some
>> use.
>First, lets get one thing straight: the problem is the slowness,
>not necessarily the swap usage. It is very easy to jump to wrong
>conclusions, so lets keep focussed on that part of the problem
>which we know to be true before we all start hacking on parts of
>the system which aren't related to the slowness...
OK, you might be right on this point. But that different kernels show
such a big difference in its willingness to swap out things in memory is
something to think about...
>> I can provide you with dmesg, /proc/meminfo or whatever might be useful.
>How about some output (maybe 20 lines) of 'vmstat 1' during the
>time where your system is slow, together with a short description
>of how large the various processes (X, Mozilla ...) in your system
>are ?
root # vmstat 1
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 265048 5248 32248 119108 6 15 65 62 252 585 21 6 73 0
1 6 266648 4480 32316 120300 0 4656 2152 4652 1348 821 13 8 79 0
1 0 265052 4496 31036 120184 8 336 1668 340 1226 765 15 7 78 0
0 1 265052 4496 31112 121564 4 0 3152 0 1198 894 18 8 74 0
1 0 265052 4504 31076 123112 0 0 3024 8576 1229 857 17 7 76 0
0 1 265052 4496 30020 127816 12 0 3300 0 1283 862 16 6 78 0
2 0 265052 5244 29980 129796 0 0 3520 0 1212 991 17 9 74 0
2 0 265052 4604 30044 130904 0 0 3036 0 1213 903 20 8 72 0
0 9 266664 4704 30068 130836 4 6612 212 6608 1533 558 9 4 87 0
0 1 265048 4668 30080 131120 8 272 1632 308 1246 837 20 11 69 0
0 2 265048 4680 28808 130356 0 0 2308 48 1198 912 18 8 74 0
1 2 265048 4680 27592 133108 12 0 1500 0 1142 753 16 5 79 0
1 2 265048 4756 26776 135432 4 0 1436 0 1151 783 17 8 75 0
0 2 265048 4820 26556 137208 12 0 1488 0 1150 773 18 5 77 0
0 3 265048 4820 26508 137832 0 0 544 728 1256 654 16 5 79 0
1 1 265048 4876 26328 141604 8 0 2980 268 1228 992 20 10 70 0
1 0 265040 4472 25924 140324 92 0 3200 0 1167 1569 58 15 27 0
2 0 265048 4372 25896 141292 8 4808 76 4812 1286 2517 85 15 0 0
1 1 265048 4348 24516 147516 24 0 860 0 1074 2550 83 16 1 0
1 0 265048 4408 22992 154120 28 0 668 0 1078 2455 83 15 2 0
root # cat /proc/meminfo
total: used: free: shared: buffers: cached:
Mem: 394547200 378638336 15908864 0 22396928 258523136
Swap: 542826496 271384576 271441920
MemTotal: 385300 kB
MemFree: 15536 kB
MemShared: 0 kB
Buffers: 21872 kB
Cached: 167848 kB
SwapCached: 84616 kB
Active: 175416 kB
Inact_dirty: 123512 kB
Inact_clean: 7728 kB
Inact_target: 61328 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 385300 kB
LowFree: 15536 kB
SwapTotal: 530104 kB
SwapFree: 265080 kB
Committed_AS: 365684 kB
And these are some of the apps running, basically those more
memory-hungry, at this point I mean:
UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
root 15312 15309 2 52651 50960 0 Nov29 ? 00:19:42 /usr/X11R6/bin/X
jmarcet 15409 15332 0 1223 4 0 Nov29 ? 00:00:00 /bin/sh --login /usr/kde/3.1/bin/startkde
jmarcet 15444 1 0 18564 1580 0 Nov29 ? 00:02:04 kdeinit: kded
jmarcet 15509 15436 0 2525 432 0 Nov29 ? 00:00:03 artsd
jmarcet 15553 1 0 19098 1060 0 Nov29 ? 00:00:00 kdeinit: knotify
jmarcet 15556 1 0 17672 832 0 Nov29 ? 00:00:00 kdeinit: ksmserver
jmarcet 15560 15436 0 17966 2684 0 Nov29 ? 00:00:08 kdeinit: kwin
jmarcet 15571 1 0 19961 2568 0 Nov29 ? 00:00:24 kdeinit: kdesktop
jmarcet 15591 1 0 19177 5488 0 Nov29 ? 00:00:32 kdeinit: kicker
jmarcet 15597 15436 0 5557 28 0 Nov29 ? 00:00:00 kdeinit: kio_file
jmarcet 15600 15571 0 3942 1672 0 Nov29 ? 00:03:15 /home/jmarcet/.kde/Autostart/gkrellm2
jmarcet 15637 15436 0 18397 2640 0 Nov29 ? 00:02:46 kdeinit: konsole
jmarcet 15659 15436 0 18191 3776 0 Nov29 ? 00:00:21 kdeinit: konsole
jmarcet 15682 15659 0 3598 6284 0 Nov29 pty/s1 00:00:17 /usr/bin/mutt
jmarcet 21557 15436 1 29878 23424 0 Nov29 ? 00:07:55 kdeinit: konqueror --silent
jmarcet 22528 1 0 17825 524 0 Nov29 ? 00:00:00 kdeinit: kio_uiserver
jmarcet 17165 15591 7 18864 5552 0 Nov29 ? 00:36:20 appletproxy
jmarcet 10990 21557 1 20661 1880 0 Nov29 ? 00:05:31 /usr/kde/3.1/bin/nspluginviewer
jmarcet 12338 15436 0 18646 5656 0 01:19 ? 00:00:28 kdeinit: konsole
This is after the system has been in use with a 512MB swap partition for
around 1 hour. I must say it is barely usable as a desktop, way _far_
from a responsive environment. looking at the memory numbers it's easy
to think I need more memory, but with other kernels, at the same load
-by load I mean of memory, not processor- or even much higher, swap
usage was maybe around 16MB, while there was still ~192MB if free cached
memory.
So yes, you are right that swap usage is not the problem. It seems more
like memory getting too dirty.
BTW, I was compiling something with make -j2, not very big, it was
htdig.
I must reinforce that switching virtual desktops was so slow that you
could see windows repainting slowly on the screen.
processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 6
model name : AMD Athlon(TM) XP 1800+
stepping : 2
cpu MHz : 1544.999
cache size : 256 KB
The system has both an IDE ATA-100 45GB an a SCSI U2W IBM 9GB drives,
swap partition is at the start of the IDE disk, since it is much faster,
throughput-wise, than the SCSI model.
--
Javier Marcet <[email protected]>
On Sat, 30 Nov 2002, Javier Marcet wrote:
> >First, lets get one thing straight: the problem is the slowness,
> >not necessarily the swap usage. It is very easy to jump to wrong
>
> OK, you might be right on this point.
> root # vmstat 1
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id wa
> 0 1 265048 5248 32248 119108 6 15 65 62 252 585 21 6 73 0
> 1 6 266648 4480 32316 120300 0 4656 2152 4652 1348 821 13 8 79 0
> 1 0 265052 4496 31036 120184 8 336 1668 340 1226 765 15 7 78 0
> 0 1 265052 4496 31112 121564 4 0 3152 0 1198 894 18 8 74 0
> 1 0 265052 4504 31076 123112 0 0 3024 8576 1229 857 17 7 76 0
^^^^^^^^^^^^^^^^^^^
Looks like my guess was right after all. The amount of swap
IO is maybe 10% of the amount of filesystem IO in your vmstat
snippet above.
> This is after the system has been in use with a 512MB swap partition for
> around 1 hour. I must say it is barely usable as a desktop, way _far_
> from a responsive environment. looking at the memory numbers it's easy
> to think I need more memory, but with other kernels,
> So yes, you are right that swap usage is not the problem. It seems more
> like memory getting too dirty.
Two things could be happening here:
1) the kernel decides to cache the wrong things in the
page cache
and/or
2) the IO scheduler is giving worse latencies now
If the problem is (1) it might get resolved by using the -rmap
or -aa kernels. If the problem is (2) you'll want Andrew Morton's
read_latency patch (which I'll port to 2.4.20 real soon now).
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://guru.conectiva.com/
Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
Hmm i'm using 2.4.19-rmap15 and i find my box's performance exceptional
both whilst sitting at the main console and remotely from a laptop.
Currently there is someone working in openoffice at it whilst i work
in X11 remotely. This same box is also my NFS server for 5 other boxes.
Committed_AS would probably be in the region of 1.2GB+ in normal usage,
rmap15 does make good decisions about things to swap out in my case and
manages to keep the working sets of the bulk of my apps in physical.
Dual PII-400, swap device is an IBM 9G UW2 (aic7891) with overflow swap on
a UDMA33 ATA disk (PIIX4)
21:34:26 up 6 days, 22:11, 34 users, load average: 6.40, 6.06, 6.83
Filename Type Size Used Priority
/dev/sda1 partition 1052216 805340 1
/dev/hdb1 partition 1052216 0 0
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
4 0 1 804176 4016 12416 233564 5 3 18 28 14 21 12 12 5
14 0 0 804176 6940 12352 233024 32 0 488 0 918 3741 22 21 56
12 2 1 803816 4872 12424 234048 0 0 176 3832 886 3745 44 16 40
19 1 0 803816 6072 12444 233972 0 0 636 4 844 3685 60 31 10
6 0 0 803840 4360 12476 234044 0 4 588 4 926 4123 68 23 10
13 1 0 803844 5908 12452 234096 0 0 668 0 836 3722 70 24 6
13 0 0 803844 4176 12456 234400 0 0 460 0 846 3740 64 25 11
24 2 0 803844 6688 12456 233716 0 0 376 5388 880 3893 24 18 58
8 0 2 804952 4236 12124 232612 0 2104 488 2104 859 3762 65 25 10
25 0 0 804952 5072 12104 233624 316 64 476 64 870 3888 41 21 38
total: used: free: shared: buffers: cached:
Mem: 525332480 522969088 2363392 0 10117120 365477888
Swap: 2154938368 873652224 1281286144
MemTotal: 513020 kB
MemFree: 2308 kB
MemShared: 0 kB
Buffers: 9880 kB
Cached: 282768 kB
SwapCached: 74144 kB
Active: 273452 kB
Inactive: 194068 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 513020 kB
LowFree: 2308 kB
SwapTotal: 2104432 kB
SwapFree: 1251256 kB
Probably around 300 processes
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1336 436 ? S Nov22 0:04 init
root 2 0.0 0.0 0 0 ? SW Nov22 0:00 [migration_CPU0]
root 3 0.0 0.0 0 0 ? SW Nov22 0:00 [migration_CPU1]
root 4 0.0 0.0 0 0 ? SW Nov22 0:30 [keventd]
root 5 0.0 0.0 0 0 ? SWN Nov22 0:01 [ksoftirqd_CPU0]
root 6 0.0 0.0 0 0 ? SWN Nov22 0:01 [ksoftirqd_CPU1]
root 7 0.0 0.0 0 0 ? SW Nov22 5:46 [kswapd]
root 8 0.0 0.0 0 0 ? SW Nov22 0:00 [bdflush]
root 9 0.0 0.0 0 0 ? SW Nov22 0:11 [kupdated]
root 10 0.0 0.0 0 0 ? SW Nov22 0:00 [scsi_eh_0]
root 11 0.0 0.0 0 0 ? SW Nov22 0:00 [scsi_eh_1]
root 12 0.0 0.0 0 0 ? SW Nov22 0:00 [khubd]
root 13 0.0 0.0 0 0 ? SW Nov22 0:00 [mdrecoveryd]
root 14 0.0 0.0 0 0 ? SW Nov22 0:14 [kjournald]
root 117 0.0 0.0 0 0 ? SW Nov22 0:07 [kjournald]
root 118 0.0 0.0 0 0 ? SW Nov22 0:02 [kjournald]
root 119 0.0 0.0 0 0 ? SW Nov22 0:00 [kjournald]
root 120 0.0 0.0 0 0 ? SW Nov22 0:03 [kjournald]
root 408 0.0 0.0 1400 488 ? S Nov22 0:05 syslogd -m 0
root 412 0.0 0.0 1336 372 ? S Nov22 0:00 klogd -x
rpc 423 0.0 0.0 1488 408 ? S Nov22 0:00 portmap
rpcuser 442 0.0 0.0 1528 464 ? S Nov22 0:00 rpc.statd
root 545 0.0 0.0 1440 456 ? S Nov22 0:00 /usr/sbin/automou
named 558 0.0 0.1 14024 772 ? S Nov22 0:06 named -u named
root 575 0.0 0.0 3276 472 ? S Nov22 0:01 /usr/sbin/sshd
root 585 0.0 0.0 2092 456 ? S Nov22 0:00 xinetd -stayalive
lp 598 0.0 0.1 5192 536 ? S Nov22 0:00 lpd Waiting
root 613 0.0 0.0 3716 324 ? S Nov22 0:00 rpc.rquotad
root 617 0.0 0.0 0 0 ? SW Nov22 1:58 [nfsd]
root 618 0.0 0.0 0 0 ? SW Nov22 1:56 [nfsd]
root 619 0.0 0.0 0 0 ? SW Nov22 1:58 [nfsd]
root 620 0.0 0.0 0 0 ? SW Nov22 1:59 [nfsd]
root 621 0.0 0.0 0 0 ? SW Nov22 1:59 [nfsd]
root 622 0.0 0.0 0 0 ? SW Nov22 1:57 [nfsd]
root 623 0.0 0.0 0 0 ? SW Nov22 1:58 [nfsd]
root 624 0.0 0.0 0 0 ? SW Nov22 1:55 [nfsd]
root 625 0.0 0.0 0 0 ? SW Nov22 0:00 [lockd]
root 626 0.0 0.0 0 0 ? SW Nov22 0:00 [rpciod]
root 632 0.0 0.0 1556 488 ? S Nov22 0:00 rpc.mountd
root 641 0.0 0.1 2524 656 ? S Nov22 0:01 /usr/sbin/dhcpd
root 657 0.0 0.1 5040 756 ? S Nov22 0:04 sendmail: accepti
smmsp 666 0.0 0.0 4856 488 ? S Nov22 0:00 sendmail: Queue r
root 676 0.0 0.0 17792 508 ? S Nov22 0:01 /usr/bin/perl /us
root 697 0.0 0.1 13276 644 ? S Nov22 0:01 /usr/sbin/httpd
root 706 0.0 0.0 1516 480 ? S Nov22 0:00 crond
root 741 0.0 0.0 1312 224 ? S Nov22 0:00 /usr/bin/vmnet-br
root 759 0.0 0.0 1520 404 ? S Nov22 0:00 /usr/bin/vmnet-na
xfs 1049 0.0 0.0 4548 496 ? S Nov22 0:00 xfs -droppriv -da
root 1058 0.0 0.0 4936 452 ? S Nov22 0:00 smbd -D
root 1062 0.0 0.1 3792 800 ? S Nov22 0:05 nmbd -D
daemon 1080 0.0 0.0 1368 440 ? S Nov22 0:00 /usr/sbin/atd
root 1090 0.0 0.0 3552 448 ? S Nov22 0:00 rhnsd --interval
ident 1098 0.0 0.1 6268 948 ? S Nov22 0:00 /usr/bin/perl /us
root 1101 0.0 0.0 1316 332 tty2 S Nov22 0:00 /sbin/mingetty tt
root 1102 0.0 0.0 1316 332 tty3 S Nov22 0:00 /sbin/mingetty tt
root 1103 0.0 0.0 1316 332 tty4 S Nov22 0:00 /sbin/mingetty tt
root 1104 0.0 0.0 1316 332 tty5 S Nov22 0:00 /sbin/mingetty tt
root 1105 0.0 0.0 1316 332 tty6 S Nov22 0:00 /sbin/mingetty tt
root 1106 0.0 0.1 13220 704 ? S Nov22 0:00 /usr/bin/gdm-bina
root 1139 0.0 0.1 13980 556 ? S Nov22 0:00 /usr/bin/gdm-bina
root 1140 8.0 6.4 337168 32948 ? S< Nov22 804:26 /usr/X11R6/bin/X
root 1143 0.0 0.0 1312 224 ? S Nov22 0:00 /usr/bin/vmnet-ne
root 1153 0.0 0.0 1684 452 ? S Nov22 0:00 /usr/bin/vmnet-dh
zwane 1163 0.0 0.0 4532 312 ? S Nov22 0:00 -/bin/tcsh -c /us
zwane 1225 0.0 0.1 4300 800 ? S Nov22 0:00 /bin/sh /usr/bin/
zwane 1226 0.0 0.0 2916 360 ? S Nov22 0:00 /usr/bin/ssh-agen
zwane 1269 0.0 0.2 20340 1048 ? S Nov22 0:02 kdeinit: Running.
zwane 1272 0.0 0.3 22344 1588 ? S Nov22 0:14 kdeinit: dcopserv
zwane 1275 0.0 0.3 23384 2008 ? S Nov22 0:01 kdeinit: klaunche
zwane 1277 0.0 0.5 23776 2616 ? S Nov22 2:17 kdeinit: kded
zwane 1288 0.0 0.4 26560 2288 ? S Nov22 0:13 kdeinit: knotify
zwane 1289 0.0 0.0 1392 264 ? S Nov22 0:01 kwrapper ksmserve
zwane 1291 0.0 0.4 23352 2104 ? S Nov22 0:11 kdeinit: ksmserve
zwane 1292 0.2 1.0 33140 5200 ? S Nov22 28:12 kdeinit: kwin -se
zwane 1293 0.0 0.0 4532 300 ? S Nov22 0:00 /bin/tcsh -c gkre
zwane 1295 0.0 0.0 4532 300 ? S Nov22 0:00 /bin/tcsh -c /usr
zwane 1301 0.0 0.0 4532 300 ? S Nov22 0:00 /bin/tcsh -c xter
zwane 1320 0.2 0.3 41448 1872 ? S Nov22 19:59 /opt/acrobat4/Rea
zwane 1383 0.1 0.3 9860 2004 ? S Nov22 18:02 gkrellm
zwane 1392 0.0 0.2 8416 1028 ? S Nov22 0:55 xterm -bg black -
zwane 1405 0.0 0.8 26408 4508 ? S Nov22 9:07 kdeinit: kdesktop
zwane 1407 0.0 0.0 4964 404 pts/0 S Nov22 0:00 -csh
zwane 1438 0.0 0.5 28740 2808 ? S Nov22 5:46 kdeinit: klipper
zwane 1442 0.0 0.2 24068 1312 ? S Nov22 0:07 kdeinit: kwrited
zwane 1444 0.0 0.2 11148 1220 ? S Nov22 0:27 /usr/bin/pam-pane
zwane 1445 0.7 1.8 42296 9572 ? S Nov22 77:21 kdeinit: konquero
root 1446 0.0 0.0 1364 408 ? S Nov22 0:01 /sbin/pam_timesta
zwane 1447 0.2 0.8 28600 4176 ? S Nov22 25:05 kdeinit: konsole
zwane 1450 0.1 0.5 24252 2856 ? S Nov22 16:07 kdeinit: kmix -se
zwane 1451 0.2 1.5 29368 7816 ? S Nov22 26:38 kdeinit: konsole
zwane 1453 0.2 0.9 16360 4644 ? S Nov22 24:41 xchat --sm-config
zwane 1455 0.0 0.5 25216 2592 ? S Nov22 1:48 kworldclock -sess
zwane 1464 0.0 0.0 5064 408 pts/3 S Nov22 0:00 -bin/tcsh
zwane 1466 0.0 0.2 5500 1028 pts/4 S Nov22 0:03 -bin/tcsh
zwane 1473 0.0 0.2 5292 1196 pts/5 S Nov22 0:03 -bin/tcsh
zwane 1476 0.0 0.0 5300 468 pts/6 S Nov22 0:03 -bin/tcsh
zwane 1481 0.0 0.0 5272 472 pts/7 S Nov22 0:01 -bin/tcsh
zwane 1487 0.0 0.1 5288 716 pts/8 S Nov22 0:00 -bin/tcsh
zwane 1509 0.0 0.1 5292 740 pts/9 S Nov22 0:01 -bin/tcsh
zwane 1527 0.0 0.1 5300 712 pts/10 S Nov22 0:04 -bin/tcsh
zwane 1648 0.0 0.1 6120 888 ? S Nov22 0:25 fetchmail -d300
zwane 1682 0.0 0.2 9876 1280 ? S Nov22 7:24 /usr/bin/wish /us
zwane 1731 15.1 2.1 168200 11096 ? R Nov22 1507:36 /usr/bin/vmware
zwane 1732 0.0 5.4 121100 28008 ? S Nov22 1:29 vmware-ui -A 14 -
zwane 1733 0.1 0.1 24340 1024 ? S Nov22 17:44 vmware-mks -A 15
zwane 4489 0.0 0.0 5268 508 pts/11 S Nov22 0:00 -bin/tcsh
zwane 4547 0.0 0.1 5252 732 pts/12 S Nov22 0:00 -bin/tcsh
zwane 5334 0.0 0.4 24056 2344 ? S Nov22 0:19 kdeinit: kcalc -c
zwane 12186 0.0 0.2 11272 1240 ? S Nov23 1:00 gvim arch/i386/ke
zwane 24288 0.0 0.4 40960 2204 ? S Nov23 0:23 kdict
zwane 24289 0.0 0.1 4284 776 ? S Nov23 0:00 /bin/bash /opt/bi
zwane 24310 0.0 0.1 75208 836 ? S Nov23 0:08 /opt/kylix3/bin/b
zwane 26794 0.0 0.5 23940 2876 ? S Nov23 0:07 kdeinit: kio_uise
zwane 26903 0.0 0.0 0 0 ? Z Nov23 0:01 [drkonqi <defunct
zwane 1456 0.0 0.3 28912 1576 ? T Nov22 0:02 knotes -session 1
zwane 26939 0.0 0.4 28740 2232 ? S Nov23 0:13 knotes
zwane 2378 0.8 0.4 103300 2368 ? S Nov23 80:11 /usr/bin/vmware
zwane 2379 0.0 0.2 13616 1188 ? S Nov23 0:49 vmware-ui -A 14 -
zwane 2380 0.0 0.1 23916 864 ? S Nov23 6:19 vmware-mks -A 15
zwane 3466 0.8 1.0 103928 5196 ? S Nov23 75:30 /usr/bin/vmware
zwane 3470 0.8 0.1 162296 1000 ? R Nov23 75:03 /usr/bin/vmware
zwane 3472 0.0 0.2 13484 1176 ? S Nov23 0:47 vmware-ui -A 14 -
zwane 3473 0.0 0.1 23908 768 ? S Nov23 6:04 vmware-mks -A 15
zwane 3474 0.0 0.1 13492 968 ? S Nov23 0:47 vmware-ui -A 14 -
zwane 3475 0.0 0.1 23912 796 ? S Nov23 6:03 vmware-mks -A 15
zwane 3522 0.3 0.7 102792 4008 ? S Nov23 33:05 /usr/bin/vmware
zwane 3523 0.0 0.2 13484 1172 ? S Nov23 0:47 vmware-ui -A 14 -
zwane 3546 0.0 0.1 23904 752 ? R Nov23 5:59 vmware-mks -A 15
zwane 3896 0.0 0.1 137900 708 ? S< Nov23 0:00 vmware [ide0:0]
zwane 3902 0.0 0.1 137832 676 ? S< Nov23 0:00 vmware [ide1:0]
zwane 3994 0.0 0.1 72372 904 ? S< Nov23 0:10 vmware [ide0:0]
zwane 3995 0.0 0.1 72360 644 ? S< Nov23 0:00 vmware [ide1:0]
zwane 4047 0.0 0.1 72368 968 ? S< Nov23 0:27 vmware [ide0:0]
zwane 4113 0.0 0.1 72244 688 ? S< Nov23 0:00 vmware [ide1:0]
zwane 20494 0.1 1.0 151472 5480 ? R Nov23 20:11 /usr/lib/openoffi
apache 30995 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 30996 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 30997 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 30998 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 30999 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 31000 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 31001 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
apache 31002 0.0 0.1 13276 620 ? S Nov24 0:00 /usr/sbin/httpd
zwane 10640 0.0 0.1 4292 788 ? S Nov24 0:00 /bin/sh /usr/bin/
zwane 10641 0.1 1.3 35804 6720 ? S Nov24 14:17 /usr/lib/opera/6.
zwane 10656 0.0 0.0 2192 256 ? S Nov24 0:00 /usr/lib/opera/pl
zwane 10773 0.0 0.2 11176 1196 ? S Nov24 0:51 gvim init/do_moun
zwane 11851 0.0 0.1 5276 768 pts/13 S Nov24 0:01 -bin/tcsh
zwane 11893 0.0 0.4 23392 2232 ? S Nov24 0:12 kcharselect -icon
zwane 12453 0.0 0.1 5264 744 pts/14 S Nov24 0:00 -bin/tcsh
zwane 12694 0.0 0.3 23512 1928 ? S Nov24 0:05 kdeinit: kcookiej
zwane 12700 0.0 0.1 18912 992 ? S Nov24 0:00 /usr/bin/kdesud
zwane 13758 0.0 0.1 4644 612 pts/0 S Nov24 0:10 minicom -o ttyS0
zwane 15739 0.0 0.1 5248 740 pts/15 S Nov24 0:00 -bin/tcsh
zwane 15741 0.0 0.0 5248 512 pts/16 S Nov24 0:00 -bin/tcsh
zwane 15765 0.0 0.0 5068 424 pts/17 S Nov24 0:00 -bin/tcsh
zwane 15791 0.0 0.0 5248 512 pts/18 S Nov24 0:00 -bin/tcsh
zwane 16129 0.0 0.1 72576 820 ? S< Nov24 0:52 vmware [ide0:0]
zwane 16130 0.0 0.1 72580 796 ? S< Nov24 0:00 vmware [ide1:0]
zwane 16131 0.0 0.2 72644 1124 ? S< Nov24 0:09 /usr/bin/vmware
zwane 17331 0.0 0.2 10916 1036 ? S Nov25 0:02 /usr/bin/gvim
zwane 24524 0.0 0.1 7944 824 ? S Nov25 0:00 xterm -bg black -
zwane 24532 0.0 0.0 5012 408 pts/19 S Nov25 0:00 -csh
zwane 24567 0.0 0.0 35732 12 pts/19 S Nov25 0:00 xski
zwane 26695 0.0 0.2 10856 1180 ? S Nov25 0:03 gvim iodev/ioapic
zwane 27457 0.0 0.1 3432 564 pts/18 S Nov25 0:00 ssh freebsd
zwane 27458 0.0 0.0 2100 512 pts/17 S Nov25 0:00 telnet netbsd
zwane 27473 0.0 0.0 2100 512 pts/16 S Nov25 0:00 telnet qnx
zwane 31803 0.0 0.2 10924 1180 ? S Nov26 0:04 gvim irq.h
zwane 2373 0.0 0.0 3332 280 pts/4 S Nov26 2:13 esd -as 60
zwane 2984 0.2 1.3 33788 6848 ? S Nov26 12:06 kdeinit: kicker
zwane 3147 0.0 0.5 27700 2776 ? S Nov26 1:49 appletproxy --con
zwane 5902 0.0 0.1 4332 824 ? S Nov27 0:00 /bin/sh /usr/lib/
zwane 5919 3.5 8.0 100180 41228 ? S Nov27 142:58 /usr/lib/mozilla/
zwane 5943 0.0 0.0 0 0 ? Z Nov27 0:00 [netstat <defunct
zwane 9883 0.7 0.1 46412 1012 ? S Nov27 31:03 /usr/bin/vmware
zwane 9884 0.0 0.2 13548 1036 ? S Nov27 0:22 vmware-ui -A 15 -
zwane 9885 0.0 0.1 23908 856 ? R Nov27 2:55 vmware-mks -A 16
zwane 10235 0.0 0.1 23100 768 ? S< Nov27 0:00 vmware [ide0:0]
zwane 10236 0.0 0.1 23092 772 ? S< Nov27 0:00 vmware [ide1:0]
zwane 14410 0.2 0.2 25532 1324 pts/11 S Nov27 6:39 pine
zwane 28578 0.0 0.1 7872 868 ? S Nov27 0:00 xterm -bg black -
zwane 28694 0.0 0.0 5188 512 pts/2 S Nov27 0:00 -csh
zwane 7709 0.0 0.1 138800 948 ? S< Nov27 0:11 vmware [ide1:0]
zwane 7710 0.0 0.3 138860 2008 ? S< Nov27 0:08 /usr/bin/vmware
root 7743 0.0 0.2 5468 1252 ? S Nov27 0:02 smbd -D
zwane 9439 0.0 0.2 75352 1144 ? S< Nov27 0:36 xine /raid/store/
zwane 12910 0.0 0.4 26716 2504 ? S Nov27 0:03 /usr/bin/kdesktop
zwane 17151 0.0 0.2 11212 1256 ? S Nov27 0:58 gvim debug
root 29455 0.0 0.1 4276 632 pts/3 S Nov28 0:00 su -
root 29458 0.0 0.1 5300 880 pts/3 S Nov28 0:01 -tcsh
zwane 406 0.0 0.0 3944 372 pts/6 S Nov28 0:00 less the-bad-patc
zwane 2662 0.0 0.1 5072 664 pts/20 S Nov28 0:00 -bin/tcsh
zwane 4289 0.0 0.0 1824 480 ? S Nov28 0:00 /usr/bin/lamd -H
zwane 6229 0.0 0.1 3612 968 pts/8 S Nov28 0:12 xscreensaver
zwane 17473 0.0 0.1 8964 860 pts/7 S 08:37 0:00 vi netlink_dev.c
zwane 23334 0.0 0.1 3432 740 pts/20 S 10:48 0:00 ssh root@mondecin
zwane 23349 0.0 0.1 3540 740 pts/14 S 10:50 0:00 ssh root@mondecin
zwane 23385 0.0 0.1 3432 744 pts/15 S 10:54 0:00 ssh root@linux
zwane 24798 0.1 0.9 25948 4728 ? S 12:33 0:40 kdeinit: konsole
zwane 24800 0.0 0.1 5424 812 pts/22 S 12:34 0:00 -bin/tcsh
zwane 24824 0.0 0.1 5164 832 pts/23 S 12:34 0:00 -bin/tcsh
zwane 24845 0.0 0.1 4984 720 pts/24 S 12:34 0:00 -bin/tcsh
zwane 24866 0.0 0.1 4740 648 pts/25 S 12:34 0:00 -bin/tcsh
zwane 24887 0.0 0.1 4740 648 pts/26 S 12:34 0:00 -bin/tcsh
zwane 24908 0.0 0.1 4740 648 pts/27 S 12:34 0:00 -bin/tcsh
zwane 25117 0.0 0.1 4300 900 pts/23 S 12:58 0:00 /bin/sh /opt/bin/
zwane 25140 0.0 0.0 3580 396 pts/23 S 12:58 0:00 tee /tmp/wine.log
zwane 25141 1.4 0.3 41024 1964 pts/23 S 12:58 7:28 /opt/bin/../wine/
zwane 25143 1.8 0.1 2796 652 ? S 12:58 9:20 /opt/bin/../wine/
zwane 25157 0.0 0.1 4300 900 pts/24 S 13:00 0:00 /bin/sh /opt/bin/
zwane 25180 0.0 0.0 3580 396 pts/24 S 13:00 0:00 tee /tmp/wine.log
zwane 25181 0.6 0.4 24812 2080 pts/24 S 13:00 3:06 /opt/bin/../wine/
root 25248 0.0 0.0 1316 352 tty1 S 13:23 0:00 /sbin/mingetty tt
zwane 25589 0.0 0.3 10932 1988 ? S 16:09 0:01 gvim debug
root 25706 0.0 0.2 13988 1196 ? S 16:52 0:00 /usr/bin/gdm-bina
zwane 25713 0.0 0.1 4548 552 ? S 16:53 0:00 -/bin/tcsh -c /us
zwane 25776 0.1 0.3 6996 1600 ? S 16:53 0:19 /usr/bin/wmaker
zwane 25777 0.0 0.1 2916 552 ? S 16:53 0:00 /usr/bin/ssh-agen
zwane 25780 0.1 0.4 9080 2328 ? S 16:53 0:29 gkrellm
zwane 25781 0.0 0.2 5148 1080 ? S 16:53 0:00 rxvt -bg black -f
zwane 25782 0.1 0.3 8236 1844 ? S 16:53 0:30 wmxmms
zwane 25783 0.0 0.1 2668 672 ? S 16:53 0:01 wmCalClock -24
zwane 25786 0.0 0.1 4772 664 pts/21 S 16:54 0:00 -csh
zwane 25859 0.1 1.2 29380 6296 ? S 16:55 0:29 konsole
zwane 25861 0.0 0.4 19948 2156 ? S 16:55 0:00 kdeinit: Running.
zwane 25864 0.0 0.5 22260 2580 ? S 16:55 0:00 kdeinit: dcopserv
zwane 25867 0.0 0.4 22320 2508 ? S 16:55 0:00 kdeinit: klaunche
zwane 25869 0.0 0.7 23116 3700 ? S 16:55 0:03 kdeinit: kded
zwane 25872 0.0 0.1 5012 724 pts/28 S 16:55 0:00 -bin/tcsh
zwane 25896 0.4 1.0 16320 5564 ? S 16:56 1:09 xchat
zwane 25905 0.0 0.2 5192 1088 pts/29 S 16:58 0:00 -bin/tcsh
zwane 25926 0.0 0.2 5200 1304 pts/30 S 16:58 0:00 -bin/tcsh
zwane 25947 0.0 0.2 5016 1332 pts/31 S 16:58 0:00 -bin/tcsh
zwane 25968 0.0 0.1 5016 696 pts/32 S 16:58 0:00 -bin/tcsh
zwane 26000 0.0 0.1 3432 756 pts/28 S 16:59 0:00 ssh presario
zwane 26008 2.8 1.0 63768 5376 pts/29 R 17:00 3:49 xmms
zwane 26030 0.0 0.2 5160 1172 ? S 17:06 0:11 rxvt -bg black -f
zwane 26031 0.0 0.1 5016 696 pts/33 S 17:06 0:00 -csh
zwane 26051 0.2 0.7 20280 4104 pts/33 S 17:06 0:43 pine
zwane 26389 0.0 0.1 4332 904 ? S 19:39 0:00 /bin/sh /usr/lib/
zwane 26406 1.6 0.6 52960 3500 ? S 19:39 1:54 /usr/lib/mozilla/
zwane 29318 0.2 0.2 4132 1116 pts/5 S 21:32 0:00 make -j2 bzImage
zwane 29615 0.0 0.1 3812 808 pts/5 S 21:33 0:00 make -f scripts/M
zwane 31738 0.5 0.1 3808 804 pts/5 S 21:35 0:00 make -f scripts/M
zwane 31771 1.6 0.1 3844 840 pts/5 S 21:35 0:00 make -f scripts/M
zwane 31916 6.0 0.1 3812 808 pts/5 S 21:35 0:00 make -f scripts/M
zwane 31951 0.0 0.2 4308 1036 pts/5 S 21:35 0:00 /bin/sh -c set -e
zwane 31953 0.0 0.0 1340 324 pts/5 S 21:35 0:00 ccache gcc -Wp,-M
zwane 31954 0.0 0.0 3652 476 pts/5 S 21:35 0:00 /usr/bin/gcc -Wp,
zwane 31955 0.0 0.3 4908 1768 pts/5 R 21:35 0:00 /usr/lib/gcc-lib/
zwane 31956 0.0 0.2 3044 1132 pts/30 R 21:35 0:00 ps aux
zwane 31960 0.0 0.2 4308 1036 pts/5 S 21:35 0:00 /bin/sh -c set -e
zwane 31961 0.0 0.0 1340 324 pts/5 S 21:35 0:00 ccache gcc -Wp,-M
zwane 31962 0.0 0.0 3652 476 pts/5 S 21:35 0:00 /usr/bin/gcc -Wp,
zwane 31963 0.0 0.1 3920 732 pts/5 R 21:35 0:00 /usr/lib/gcc-lib/
I also tend to notice general kernel crappiness (ask Con Kolivas ;)
Cheers,
Zwane
On Fri, 29 Nov 2002, Rik van Riel wrote:
> If the problem is (1) it might get resolved by using the -rmap
> or -aa kernels. If the problem is (2) you'll want Andrew Morton's
> read_latency patch (which I'll port to 2.4.20 real soon now).
The read_latency makes an immense difference, i've CC'd Con because
there was one other bit which i needed to get IO from stalling the box
(5-10s) which i can't recall right now.
Cheers,
Zwane
--
function.linuxpower.ca
Zwane Mwaikambo wrote:
>
> Hmm i'm using 2.4.19-rmap15
Don't think so.
>
> total: used: free: shared: buffers: cached:
> Mem: 525332480 522969088 2363392 0 10117120 365477888
> Swap: 2154938368 873652224 1281286144
> MemTotal: 513020 kB
> MemFree: 2308 kB
> MemShared: 0 kB
> Buffers: 9880 kB
> Cached: 282768 kB
> SwapCached: 74144 kB
> Active: 273452 kB
> Inactive: 194068 kB
> HighTotal: 0 kB
> HighFree: 0 kB
> LowTotal: 513020 kB
> LowFree: 2308 kB
> SwapTotal: 2104432 kB
> SwapFree: 1251256 kB
That's stock 2.4 meminfo output.
- "Inactive: %8u kB\n"
+ "Inact_dirty: %8u kB\n"
+ "Inact_laundry:%8u kB\n"
+ "Inact_clean: %8u kB\n"
+ "Inact_target: %8lu kB\n"
You're using an extraordinary amount of swap. Is something leaking?
On Fri, 29 Nov 2002, Andrew Morton wrote:
> Zwane Mwaikambo wrote:
> >
> > Hmm i'm using 2.4.19-rmap15
>
> Don't think so.
hmm
> > Inactive: 194068 kB
> > HighTotal: 0 kB
> > HighFree: 0 kB
> > LowTotal: 513020 kB
> > LowFree: 2308 kB
> > SwapTotal: 2104432 kB
> > SwapFree: 1251256 kB
>
> That's stock 2.4 meminfo output.
Looks like the little patch monkey (namely me) which puts my kernels
together forgot to apply a patch. You know you need a break when you don't
even know which kernel you're running! All rather embarassing.
> - "Inactive: %8u kB\n"
> + "Inact_dirty: %8u kB\n"
> + "Inact_laundry:%8u kB\n"
> + "Inact_clean: %8u kB\n"
> + "Inact_target: %8lu kB\n"
>
>
> You're using an extraordinary amount of swap. Is something leaking?
I wouldn't think so, some vmware sessions with 128M RAM which seems to
really use a lot of shmem(?)
nodev 1.0G 361M 663M 36% /tmp
9.2M /tmp
9.2M total
Then there is all the other crap running.
Cheers,
Zwane (Who is officially taking an extended break)
--
function.linuxpower.ca
* Rik van Riel <[email protected]> [021130 02:57]:
>> root # vmstat 1
>> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
>> r b swpd free buff cache si so bi bo in cs us sy id wa
>> 0 1 265048 5248 32248 119108 6 15 65 62 252 585 21 6 73 0
>> 1 6 266648 4480 32316 120300 0 4656 2152 4652 1348 821 13 8 79 0
>> 1 0 265052 4496 31036 120184 8 336 1668 340 1226 765 15 7 78 0
>> 0 1 265052 4496 31112 121564 4 0 3152 0 1198 894 18 8 74 0
>> 1 0 265052 4504 31076 123112 0 0 3024 8576 1229 857 17 7 76 0
> ^^^^^^^^^^^^^^^^^^^
>Looks like my guess was right after all. The amount of swap
>IO is maybe 10% of the amount of filesystem IO in your vmstat
>snippet above.
>Two things could be happening here:
>1) the kernel decides to cache the wrong things in the
> page cache
>and/or
>2) the IO scheduler is giving worse latencies now
>If the problem is (1) it might get resolved by using the -rmap
>or -aa kernels. If the problem is (2) you'll want Andrew Morton's
>read_latency patch (which I'll port to 2.4.20 real soon now).
All right. I might be wrong, but this was with kernel 2.4.20-rc4-ac1
Doesn't it include rmap?
Also it was patched with Robert Love's preempt-kernel. Of course I've
tried without it as with various other kernel tidbits.
I always tend to use ac kernels + some other patches which aside from
preempt have nothing to do with system's basic behavior.
Yet the only time I get a good system response is with 2.5.47-ac6
(2.5.48+ drive me nuts with module loading) and with Marc-Christian
Petersen's 2.4.18-wolk which includes nearly everything out there.
--
Javier Marcet <[email protected]>
BTW, I'm running 2.4.20-rc4-ac1+preempt and it seems to run good but
whenever I leave for a few hours or wake up in the morning mozilla is
swaped out.. Any idea when/how this might be fixed?
--
L1: khromy ;khromy(at)lnuxlab.ath.cx
* khromy <[email protected]> [021130 07:33]:
>BTW, I'm running 2.4.20-rc4-ac1+preempt and it seems to run good but
>whenever I leave for a few hours or wake up in the morning mozilla is
>swaped out.. Any idea when/how this might be fixed?
I have the problem without leaving it a few hours, but when I do it gets
definitely worse. Last vmstat output I quoted here showed around 256MB
swapped. A few hours later - the computer had been sitting idle, only
the mail server for three users was running which poses no overhead at
all -, the entire 512MB SWAP space was used. Why, I don't know.
I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
kernel later.
--
Javier Marcet <[email protected]>
On Fri, 2002-11-29 at 22:49, Javier Marcet wrote:
> * khromy <[email protected]> [021130 07:33]:
>
> >BTW, I'm running 2.4.20-rc4-ac1+preempt and it seems to run good but
> >whenever I leave for a few hours or wake up in the morning mozilla is
> >swaped out.. Any idea when/how this might be fixed?
>
> I have the problem without leaving it a few hours, but when I do it gets
> definitely worse. Last vmstat output I quoted here showed around 256MB
> swapped. A few hours later - the computer had been sitting idle, only
> the mail server for three users was running which poses no overhead at
> all -, the entire 512MB SWAP space was used. Why, I don't know.
As I saw the thread today, I started looking at it myself, on RH's
2.4.18-18.8.0. By now I have:
1 0 0 432892 23828 44648 407208 0 0 0 24 780 879 95 5 0
======
However, top says:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1450 dmitri 15 0 546M 178M 16212 S 1.1 23.6 21:50 pan
I close pan, and what a change!
3 0 0 432664 20720 44748 410260 88 0 392 20 710 925 95 4 1
1 0 0 46316 197172 44760 409684 40 0 40 160 782 1177 88 12 0
So you probably still have some process which is eating your memory. And
once you find it, restart it:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
3994 dmitri 15 0 15044 14M 6152 S 0.9 1.9 0:00 pan
Dmitri
* Dmitri <[email protected]> [021130 08:11]:
>> I have the problem without leaving it a few hours, but when I do it gets
>> definitely worse. Last vmstat output I quoted here showed around 256MB
>> swapped. A few hours later - the computer had been sitting idle, only
>> the mail server for three users was running which poses no overhead at
>> all -, the entire 512MB SWAP space was used. Why, I don't know.
>As I saw the thread today, I started looking at it myself, on RH's
> 2.4.18-18.8.0. By now I have:
> 1 0 0 432892 23828 44648 407208 0 0 0 24 780 879 95 5 0
> ======
>However, top says:
> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
> 1450 dmitri 15 0 546M 178M 16212 S 1.1 23.6 21:50 pan
>I close pan, and what a change!
> 3 0 0 432664 20720 44748 410260 88 0 392 20 710 925 95 4 1
> 1 0 0 46316 197172 44760 409684 40 0 40 160 782 1177 88 12 0
>So you probably still have some process which is eating your memory. And
>once you find it, restart it:
> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
> 3994 dmitri 15 0 15044 14M 6152 S 0.9 1.9 0:00 pan
This is not the problem in my case. It happens without any app getting
out of control in memory consumption, although with some kernel - don't
remember which one it was right now - X tend to show this, even killing
all the apps running. The only way to get back to normal memory usage
was re-starting the X server.
--
Javier Marcet <[email protected]>
* On Fri, Nov 29, 2002 at 02:31 PM (-0200), Rik van Riel wrote:
> On Fri, 29 Nov 2002, Javier Marcet wrote:
>
> > In recent 2.4.20 pre and rc kernels ( I tend to use the ac branch ), I
> > had notice my system, when using X mainly, got terribly slow after some
> > use.
>
> First, lets get one thing straight: the problem is the slowness,
> not necessarily the swap usage.
I've experienced a similar problem with "linux-2.4.20-rc2-ac3",
"linux-2.4.20-rc4-ac1" and "linux-2.4.20-ac1". At first I also
thought it's a swap problem, but this seems to be a wrong con-
clusion, too.
The problem occurs, for example, when Mozilla and RealPlayer are
running and I start to compile something or copy large files. The
RealPlayer interrupts, the mouse sometimes doesn't move smoothly
any more, and so on.
Using "linux-2.4.20" I don't see this behaviour.
I've collected some information when running "linux-2.4.20-ac1"
and "linux-2.4.20". I've uploaded it to:
http://www.uni-ulm.de/~s_smoser/lkml/
The "vmstat"- and "ps"-files were created during the "cp" of about
2,5 GB of data from one hard disk to the other one.
Bye,
Steffen
* Rik van Riel <[email protected]> [021130 02:57]:
>> root # vmstat 1
>> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
>> r b swpd free buff cache si so bi bo in cs us sy id wa
>> 0 1 265048 5248 32248 119108 6 15 65 62 252 585 21 6 73 0
>> 1 6 266648 4480 32316 120300 0 4656 2152 4652 1348 821 13 8 79 0
>> 1 0 265052 4496 31036 120184 8 336 1668 340 1226 765 15 7 78 0
>> 0 1 265052 4496 31112 121564 4 0 3152 0 1198 894 18 8 74 0
>> 1 0 265052 4504 31076 123112 0 0 3024 8576 1229 857 17 7 76 0
> ^^^^^^^^^^^^^^^^^^^
>Looks like my guess was right after all. The amount of swap
>IO is maybe 10% of the amount of filesystem IO in your vmstat
>snippet above.
[...]
>Two things could be happening here:
>1) the kernel decides to cache the wrong things in the
> page cache
>and/or
>2) the IO scheduler is giving worse latencies now
>If the problem is (1) it might get resolved by using the -rmap
>or -aa kernels. If the problem is (2) you'll want Andrew Morton's
>read_latency patch (which I'll port to 2.4.20 real soon now).
I tend to believe is (1). I was using 2.4.20-jam0 (-aa derived) all this
morning. It seemed to behave better than 2.4.20-rc4-ac1 on this point. I
compiled a couple apps without problems, had different apps and servers
running while both si & so and bi & bo were low. However, when I was
uncompressing mozilla-1.2.tar.gz the system locked up, just as
2.4.20-rc2aa1 did days before. ALT-SYSRQ allowed me to reboot, but I did
not take note of any kernel dump...
I am now running 2.5.47-ac6 and things are _way_ better in any aspect.
I'll post some vmstat output later when the system has been running for
a while. So far it has already uncompressed mozilla's source with no
slowdown or lockup.
--
Javier Marcet <[email protected]>
On Sat, Nov 30, 2002 at 07:49:10AM +0100, Javier Marcet wrote:
> * khromy <[email protected]> [021130 07:33]:
>
> >BTW, I'm running 2.4.20-rc4-ac1+preempt and it seems to run good but
> >whenever I leave for a few hours or wake up in the morning mozilla is
> >swaped out.. Any idea when/how this might be fixed?
>
> I have the problem without leaving it a few hours, but when I do it gets
> definitely worse. Last vmstat output I quoted here showed around 256MB
> swapped. A few hours later - the computer had been sitting idle, only
> the mail server for three users was running which poses no overhead at
> all -, the entire 512MB SWAP space was used. Why, I don't know.
>
> I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
> kernel later.
aa runs beautifully but it locked up once on me..
--
L1: khromy ;khromy(at)lnuxlab.ath.cx
On Sat, 30 Nov 2002, Steffen Moser wrote:
> I've experienced a similar problem with "linux-2.4.20-rc2-ac3",
> "linux-2.4.20-rc4-ac1" and "linux-2.4.20-ac1". At first I also
> thought it's a swap problem, but this seems to be a wrong con-
> clusion, too.
Known problem, rmap14 doesn't do pageout IO soon enough. This
is good if the inactive pages are clean (cache) but stalls the
system if the data needs to be written to disk.
This should be fixed in rmap15.
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://guru.conectiva.com/
Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
On Sat, Nov 30, 2002 at 01:23:45PM -0500, khromy wrote:
> On Sat, Nov 30, 2002 at 07:49:10AM +0100, Javier Marcet wrote:
> > * khromy <[email protected]> [021130 07:33]:
> >
> > >BTW, I'm running 2.4.20-rc4-ac1+preempt and it seems to run good but
> > >whenever I leave for a few hours or wake up in the morning mozilla is
> > >swaped out.. Any idea when/how this might be fixed?
> >
> > I have the problem without leaving it a few hours, but when I do it gets
> > definitely worse. Last vmstat output I quoted here showed around 256MB
> > swapped. A few hours later - the computer had been sitting idle, only
> > the mail server for three users was running which poses no overhead at
> > all -, the entire 512MB SWAP space was used. Why, I don't know.
> >
> > I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
> > kernel later.
>
> aa runs beautifully but it locked up once on me..
send me SYSRQ+T SYSRQ+P and everything else you know about it. if you
have AGP enabled try to reproduce with 10_x86-fast-pte-2 backed out.
thanks,
Andrea
* Andrea Arcangeli <[email protected]> [021130 19:47]:
>> > I have the problem without leaving it a few hours, but when I do it gets
>> > definitely worse. Last vmstat output I quoted here showed around 256MB
>> > swapped. A few hours later - the computer had been sitting idle, only
>> > the mail server for three users was running which poses no overhead at
>> > all -, the entire 512MB SWAP space was used. Why, I don't know.
>> > I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
>> > kernel later.
>> aa runs beautifully but it locked up once on me..
>send me SYSRQ+T SYSRQ+P and everything else you know about it. if you
>have AGP enabled try to reproduce with 10_x86-fast-pte-2 backed out.
>thanks,
It had locked here before. Right now I'm running 2.4.20-jam0 although
not complete, I didn't apply:
02-revert-fast-pte-2.bz2 -> since it seems this is a trouble spot
50-ide-10-partial.bz2 -> because it locked when uncompressing
51-severworks-ide.bz2 mozilla's sources
61-proconfig-0.9.5.bz2 -> it did not compile, besides I prefer
config.gz I get with other patch
80-bproc-3.2.3.bz2 -> didn't need that at all on my desktop
81-export-task_nice.bz2 -> idem
I have also applied acpi-20021126-2.4.20-rc3.diff.gz
So far I've uncompressed mozilla's sources a couple times while
compiling another thing with -j2 and there's been no problem at all.
Also, -aa kernels are the only ones which behave better in vm management
on my system, besides 2.5.x of course.
I'll put the system under some pressure along the day and see if no lock
ups show up.
Anything else you'd like me to try out? Be it io tests or whatever. At
the moment I can say I do not miss preempt-kernel patches nor have I
noticed any slowdown appreciable.
PS This is a UP system, Athlon-XP 1800+ and VIA KT133A based. Both IDE
ATA5 HD connected to VIA's controller and U2W drive connected to Adaptec
2940-U2W. Only 384MB of memory which had led me to believe all my
problems were due to lack of memory, not so I'm seeing with -aa and 2.5
kernels.
--
Javier Marcet <[email protected]>
* Andrea Arcangeli <[email protected]> [021130 19:47]:
>> > I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
>> > kernel later.
>> aa runs beautifully but it locked up once on me..
>send me SYSRQ+T SYSRQ+P and everything else you know about it. if you
>have AGP enabled try to reproduce with 10_x86-fast-pte-2 backed out.
I spoke too fast. Just after sending the e-mail, I kept reading the
mailing-list within mutt while compiling sane-backends and it locked up.
It's the first time I try kmsgdump, which comes included in -aa, I
dumped the info to a FAT12 disk. Yet the messages.txt I found on the
disk shows nothing.
I attach it anyway, just in case, since it is 16384 bytes in size but
doubt it'll be of any use.
How else can I take down the kernel dump without any additional computer
at hand? I'd prefer not having to write it down, ... but I'll do it if
there's no other way.
--
Javier Marcet <[email protected]>
* Andrea Arcangeli <[email protected]> [021130 19:47]:
Hmm, the attachment...
--
Javier Marcet <[email protected]>
* Rik van Riel <[email protected]> [021130 19:25]:
>> I've experienced a similar problem with "linux-2.4.20-rc2-ac3",
>> "linux-2.4.20-rc4-ac1" and "linux-2.4.20-ac1". At first I also
>> thought it's a swap problem, but this seems to be a wrong con-
>> clusion, too.
>Known problem, rmap14 doesn't do pageout IO soon enough. This
>is good if the inactive pages are clean (cache) but stalls the
>system if the data needs to be written to disk.
>This should be fixed in rmap15.
Is rmap15 included in 2.4.20-rc4-ac1?
That's the last ac I used and was paging out too much, so that the
system became slow unnecessarily.
--
Javier Marcet <[email protected]>
On Sun, Dec 01, 2002 at 08:59:21AM +0100, Javier Marcet wrote:
> * Andrea Arcangeli <[email protected]> [021130 19:47]:
>
> Hmm, the attachment...
the attachment doesn't show anything interesting. Next time you can
press SYSRQ+T and SYSRQ+P before dumping to floppy. BTW, you had
kmsgdump probably because it was in the jam patches, it's not in my tree
normally.
Andrea
On Sun, Dec 01, 2002 at 08:53:45AM +0100, Javier Marcet wrote:
> * Andrea Arcangeli <[email protected]> [021130 19:47]:
>
> >>> I'm about to try 2.4.20-jam0, -aa derived. I'll post results from that
> >>> kernel later.
>
> >>aa runs beautifully but it locked up once on me..
>
> >send me SYSRQ+T SYSRQ+P and everything else you know about it. if you
> >have AGP enabled try to reproduce with 10_x86-fast-pte-2 backed out.
>
> I spoke too fast. Just after sending the e-mail, I kept reading the
> mailing-list within mutt while compiling sane-backends and it locked up.
> It's the first time I try kmsgdump, which comes included in -aa, I
> dumped the info to a FAT12 disk. Yet the messages.txt I found on the
> disk shows nothing.
> I attach it anyway, just in case, since it is 16384 bytes in size but
> doubt it'll be of any use.
> How else can I take down the kernel dump without any additional computer
> at hand? I'd prefer not having to write it down, ... but I'll do it if
> there's no other way.
you can use kmsgdump (you should find it on sourceforge or similar). It
allows you to dump the oops to a floppy.
you can also try to apply 02-revert-fast-pte-2.bz2 and see if the
problem goes away. It must be some driver or something that deadlocks
the machine for you, I can't reproduce it here. Maybe you're using AGP
or some binary only module?
If the problem goes away by applying 02-revert-fast-pte-2.bz2 you can
back it out again and apply as well the debugging patch I posted
yesterday to l-k, so we'll get a stack trace in the right place.
Andrea
On 01.12.02 10:36 Andrea Arcangeli wrote:
> On Sun, Dec 01, 2002 at 08:59:21AM +0100, Javier Marcet wrote:
>> * Andrea Arcangeli <[email protected]> [021130 19:47]:
>>
>> Hmm, the attachment...
>
> the attachment doesn't show anything interesting. Next time you can
> press SYSRQ+T and SYSRQ+P before dumping to floppy. BTW, you had
> kmsgdump probably because it was in the jam patches, it's not in my tree
> normally.
>
Hi,
I too experienced the lockup others described in this thread so I just
built a kernel with kmsgdump and I got a trace for you. This is
2.4.20-rc4 with -rc2aa1 applied plus kmsgdump. I tried with J.A.
Magallon's 02-revert-fast-pte-2 but it still locks up (and this trace
_does not_ include that patch) -- besides, Im not even using AGP, as all
these tests are conducted on an old P200 test box. Its very easy to
reproduce the lock here, I just fire up X (with gnome 1.4), open a couple
aterms, xmms playing an audio cd and open a web brower -- at this point
the system is still responding nicely, but as soon as I fire up xchat, it
locks up. It still responds to pings though, but I cant ssh in. At the
console, SysRQ-S and SysRQ-U dont seem to work, but all others do. Ah,
yes, the CD playing never stops, too. The ksymoopsified SysRQ-T & SysRQ-P
trace is attached, I hope it can help somehow.
Nuno
On Sun, 1 Dec 2002, Javier Marcet wrote:
> >This should be fixed in rmap15.
>
> Is rmap15 included in 2.4.20-rc4-ac1?
No, -ac is still on rmap14. Alan may be adventurous with his
own code, but he certainly doesn't fool around with the VM.
I usually only send him -rmap code that's been tested for a
number of weeks...
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://guru.conectiva.com/
Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
On Sun, Dec 01, 2002 at 02:37:13PM +0000, Nuno Monteiro wrote:
> Pid: 2, comm: keventd
> EIP: 0010:[<c0142ae3>] CPU: 0 EFLAGS: 00000206 Not tainted
> EAX: c10ec45c EBX: 00000033 ECX: c11f8838 EDX: 40000000
> ESI: c2fed680 EDI: c10ec400 EBP: 00000033 DS: 0018 ES: 0018
[..]
> >>EIP; c0142ae3 <try_to_sync_unused_inodes+1f/1f8> <=====
>
> >>EAX; c10ec45c <_end+e521e4/13c9de8>
> >>ECX; c11f8838 <_end+f5e5c0/13c9de8>
> >>ESI; c2fed680 <[sb].data.end+a7ffd/25a9dd>
> >>EDI; c10ec400 <_end+e52188/13c9de8>
>
> Trace; c0117114 <__run_task_queue+4c/60>
> Trace; c011e0e9 <context_thread+11d/19c>
> Trace; c010588c <kernel_thread+28/38>
ok, now it's clear what the problem is. there are inuse-dirty inodes
that triggers a deadlock in the schedule-capable
try_to_sync_unused_inodes of 2.4.20rc2aa1 (that avoided me to backout an
otherwise corrupt lowlatency fix). It can trigger only in UP,
in SMP the other cpu can always run kupdate that will flush all dirty
inodes, so it would lockup one cpu as worse for 2.5 sec, this is
probably why I couldn't reproduce it, I assume all of you reproducing
the deadlock were running on an UP machine (doesn't matter if the kernel
was compiled for SMP or not).
Can you give a spin to this untested incremental fix?
--- 2.4.20rc2aa1/fs/inode.c.~1~ 2002-11-27 10:04:43.000000000 +0100
+++ 2.4.20rc2aa1/fs/inode.c 2002-12-02 01:09:05.000000000 +0100
@@ -459,13 +459,16 @@ static void try_to_sync_unused_inodes(vo
{
struct super_block * sb;
int nr_inodes = inodes_stat.nr_unused;
+ int global_pass = 0, local_pass;
restart:
spin_lock(&sb_lock);
+ local_pass = 0;
sb = sb_entry(super_blocks.next);
while (nr_inodes && sb != sb_entry(&super_blocks)) {
- if (list_empty(&sb->s_dirty)) {
+ if (local_pass < global_pass || list_empty(&sb->s_dirty)) {
sb = sb_entry(sb->s_list.next);
+ local_pass++;
continue;
}
sb->s_count++;
@@ -474,6 +477,7 @@ static void try_to_sync_unused_inodes(vo
if (sb->s_root)
nr_inodes = try_to_sync_unused_list(&sb->s_dirty, nr_inodes);
drop_super(sb);
+ global_pass = local_pass + 1;
goto restart;
}
spin_unlock(&sb_lock);
thanks,
Andrea
On 02.12.02 00:21 Andrea Arcangeli wrote:
>
> ok, now it's clear what the problem is. there are inuse-dirty inodes
> that triggers a deadlock in the schedule-capable
> try_to_sync_unused_inodes of 2.4.20rc2aa1 (that avoided me to backout an
> otherwise corrupt lowlatency fix). It can trigger only in UP,
> in SMP the other cpu can always run kupdate that will flush all dirty
> inodes, so it would lockup one cpu as worse for 2.5 sec, this is
> probably why I couldn't reproduce it, I assume all of you reproducing
> the deadlock were running on an UP machine (doesn't matter if the kernel
> was compiled for SMP or not).
>
> Can you give a spin to this untested incremental fix?
[snip snip]
Yes, this does the trick for me. With this fix it survived the last 30m
of torture (consisting of make -j4 bzImage, a gcc build plus 2 mozillas
and 1 OpenOffice.org word processor, bear in mind it is only a P200 box)
blissfully -- previously it took only 15 seconds with a 1/3 that load to
lock up. So, this patch definitely cures it.
Nuno
* Andrea Arcangeli <[email protected]> [021202 01:24]:
>> Pid: 2, comm: keventd
>> EIP: 0010:[<c0142ae3>] CPU: 0 EFLAGS: 00000206 Not tainted
>> EAX: c10ec45c EBX: 00000033 ECX: c11f8838 EDX: 40000000
>> ESI: c2fed680 EDI: c10ec400 EBP: 00000033 DS: 0018 ES: 0018
>[..]
>> >>EIP; c0142ae3 <try_to_sync_unused_inodes+1f/1f8> <=====
>> >>EAX; c10ec45c <_end+e521e4/13c9de8>
>> >>ECX; c11f8838 <_end+f5e5c0/13c9de8>
>> >>ESI; c2fed680 <[sb].data.end+a7ffd/25a9dd>
>> >>EDI; c10ec400 <_end+e52188/13c9de8>
>> Trace; c0117114 <__run_task_queue+4c/60>
>> Trace; c011e0e9 <context_thread+11d/19c>
>> Trace; c010588c <kernel_thread+28/38>
>ok, now it's clear what the problem is. there are inuse-dirty inodes
>that triggers a deadlock in the schedule-capable
>Can you give a spin to this untested incremental fix?
This appears to do the trick. I've tried some tests which always locked
it up before and now it's still rock stable.
It's also the 2.4 kernel which works more smoothly here, under heavy io
et all. Not yet 100% as well as 2.5 but quite near.
Very good work :)
Thanks for fixing the damn bug.
--
Javier Marcet <[email protected]>
On Mon, 2002-12-02 at 00:21, Andrea Arcangeli wrote:
> On Sun, Dec 01, 2002 at 02:37:13PM +0000, Nuno Monteiro wrote:
> >
> > Trace; c0117114 <__run_task_queue+4c/60>
> > Trace; c011e0e9 <context_thread+11d/19c>
> > Trace; c010588c <kernel_thread+28/38>
>
> ok, now it's clear what the problem is. there are inuse-dirty inodes
> that triggers a deadlock in the schedule-capable
> try_to_sync_unused_inodes of 2.4.20rc2aa1 (that avoided me to backout an
> otherwise corrupt lowlatency fix). It can trigger only in UP,
> in SMP the other cpu can always run kupdate that will flush all dirty
> inodes, so it would lockup one cpu as worse for 2.5 sec, this is
> probably why I couldn't reproduce it, I assume all of you reproducing
> the deadlock were running on an UP machine (doesn't matter if the kernel
Correct (for me anyways).
> was compiled for SMP or not).
>
> Can you give a spin to this untested incremental fix?
>
Yep, this also works for me.
> thanks,
>
> Andrea
Cheers, (excellent work)
Andrew Clayton
Hi Andrea,
> > ok, now it's clear what the problem is. there are inuse-dirty inodes
> > that triggers a deadlock in the schedule-capable
> > try_to_sync_unused_inodes of 2.4.20rc2aa1 (that avoided me to backout an
> > otherwise corrupt lowlatency fix). It can trigger only in UP,
> > in SMP the other cpu can always run kupdate that will flush all dirty
> > inodes, so it would lockup one cpu as worse for 2.5 sec, this is
> > probably why I couldn't reproduce it, I assume all of you reproducing
> > the deadlock were running on an UP machine (doesn't matter if the kernel
> Correct (for me anyways).
ok, deadlock is gone, instead I have EXT3-fs corruption (non data=journal
mode, just ordered) like this:
Dec 3 00:25:39 codeman kernel: EXT3-fs error (device ide0(3,9)):
ext3_free_blocks: Freeing blocks not in datazone - block = 1530182
, count = 1
Dec 3 00:25:39 codeman kernel: Aborting journal on device ide0(3,9).
Dec 3 00:25:39 codeman kernel: ext3_free_blocks: aborting transaction:
Journal has aborted in __ext3_journal_get_undo_access<2>EXT3
-fs error (device ide0(3,9)) in ext3_free_blocks: Journal has aborted
Dec 3 00:25:39 codeman kernel: ext3_reserve_inode_write: aborting
transaction: Journal has aborted in __ext3_journal_get_write_acce
ss<2>EXT3-fs error (device ide0(3,9)) in ext3_reserve_inode_write: Journal has
aborted
Dec 3 00:25:39 codeman kernel: EXT3-fs error (device ide0(3,9)) in
ext3_truncate: Journal has aborted
Dec 3 00:25:39 codeman kernel: ext3_reserve_inode_write: aborting
transaction: Journal has aborted in __ext3_journal_get_write_acce
ss<2>EXT3-fs error (device ide0(3,9)) in ext3_reserve_inode_write: Journal has
aborted
Dec 3 00:25:39 codeman kernel: EXT3-fs error (device ide0(3,9)) in
ext3_orphan_del: Journal has aborted
Dec 3 00:25:39 codeman kernel: ext3_reserve_inode_write: aborting
transaction: Journal has aborted in __ext3_journal_get_write_acce
ss<2>EXT3-fs error (device ide0(3,9)) in ext3_reserve_inode_write: Journal has
aborted
Dec 3 00:25:39 codeman kernel: EXT3-fs error (device ide0(3,9)) in
ext3_delete_inode: Journal has aborted
Dec 3 00:25:39 codeman kernel: ext3_abort called.
Dec 3 00:25:39 codeman kernel: EXT3-fs abort (device ide0(3,9)):
ext3_journal_start: Detected aborted journal
Dec 3 00:25:39 codeman kernel: Remounting filesystem read-only
Dec 3 00:25:39 codeman kernel: EXT3-fs error (device ide0(3,9)) in
start_transaction: Journal has aborted
BTW: UP, non-SMP.
ciao, Marc
On Tue, Dec 03, 2002 at 12:59:32AM +0100, Marc-Christian Petersen wrote:
> ext3_free_blocks: Freeing blocks not in datazone - block = 1530182
this is the interesting one. Did you run any unstable kernel/driver
software combination recently or maybe you got oopsed or crashes?
journaling sometime gives a false sense of reliability, you've to keep
in mind that unless you know why you had to reboot w/o a clean unmount
you should always force an e2fsck -f/reiserfsck in single user mode at
the next boot, no matter of journaling. If the machine crashed because
of a kernel oops or similar skipping the filesystemcheck at the very
next boot could left the fs corrupted for a long time until you notice
it possibly while running an unrelated kernel. So if you crashed
recently and you didn't run any e2fsck -f that could explain it. I doubt
there are corruption issues in my current tree (and even the UP deadlock
that I fixed couldn't explain the corruption, in this case [the UP
deadlock] I'm sure because I know what kind of side effect _this_ bug could
generate, for any other crash you cannot tell if e2fsck -f is needed
until/unless you know the details of the bug, and most of the time you
don't know the details of the bug at the time of the next reboot so
normally an e2fsck -f is always required after a kernel crash, this
can't be automated simply because if the kernel is crashed we can't
write to the superblock to notify e2fsck about it, so at the next boot
e2fsck will always think replying the log was enough).
Of course your problem could be explained by a bad cable or whatever
else hardware failure too. At the moment I doubt it's a problem in the
common code of my tree or mainline.
Andrea
On Tuesday 03 December 2002 01:59, you wrote:
Hi Andrea,
> this is the interesting one. Did you run any unstable kernel/driver
> software combination recently or maybe you got oopsed or crashes?
nope, no oops, no crash, afaik no unstable kernel/drivers. Kernel is yours ;)
and drivers, hmm, just intel i815, eepro100. That happend after some hours of
uptime and just doing "rm -rf linux-old"
> journaling sometime gives a false sense of reliability, you've to keep
> in mind that unless you know why you had to reboot w/o a clean unmount
> you should always force an e2fsck -f/reiserfsck in single user mode at
> the next boot, no matter of journaling. If the machine crashed because
Yep, I always do a forced fsck in case of that.
> of a kernel oops or similar skipping the filesystemcheck at the very
> next boot could left the fs corrupted for a long time until you notice
> it possibly while running an unrelated kernel. So if you crashed
> recently and you didn't run any e2fsck -f that could explain it. I doubt
I run e2fsck -fy every time after a crash. Fortunately it doesn't happen so
often :-)
> ...
> don't know the details of the bug at the time of the next reboot so
> normally an e2fsck -f is always required after a kernel crash, this
> can't be automated simply because if the kernel is crashed we can't
> write to the superblock to notify e2fsck about it, so at the next boot
> e2fsck will always think replying the log was enough).
yep. I tried to remove that 00_umount-against-unused-dirty-inodes-race fix and
after that (now 5 hours uptime) doing only copying and deleting, that ext3fs
error is away.
> Of course your problem could be explained by a bad cable or whatever
> else hardware failure too. At the moment I doubt it's a problem in the
> common code of my tree or mainline.
seems it's a problem in the umount-against-unused-dirty-inodes-race fix or if
the fix "is the right way" the problem is located somewhere else what
triggers the problem of your patch.
ciao, Marc
On Tue, Dec 03, 2002 at 11:13:22AM +0100, Marc-Christian Petersen wrote:
> On Tuesday 03 December 2002 01:59, you wrote:
>
> Hi Andrea,
>
> > this is the interesting one. Did you run any unstable kernel/driver
> > software combination recently or maybe you got oopsed or crashes?
> nope, no oops, no crash, afaik no unstable kernel/drivers. Kernel is yours ;)
> and drivers, hmm, just intel i815, eepro100. That happend after some hours of
> uptime and just doing "rm -rf linux-old"
>
> > journaling sometime gives a false sense of reliability, you've to keep
> > in mind that unless you know why you had to reboot w/o a clean unmount
> > you should always force an e2fsck -f/reiserfsck in single user mode at
> > the next boot, no matter of journaling. If the machine crashed because
> Yep, I always do a forced fsck in case of that.
>
> > of a kernel oops or similar skipping the filesystemcheck at the very
> > next boot could left the fs corrupted for a long time until you notice
> > it possibly while running an unrelated kernel. So if you crashed
> > recently and you didn't run any e2fsck -f that could explain it. I doubt
> I run e2fsck -fy every time after a crash. Fortunately it doesn't happen so
> often :-)
ok ;) I asked just in case.
>
> > ...
> > don't know the details of the bug at the time of the next reboot so
> > normally an e2fsck -f is always required after a kernel crash, this
> > can't be automated simply because if the kernel is crashed we can't
> > write to the superblock to notify e2fsck about it, so at the next boot
> > e2fsck will always think replying the log was enough).
> yep. I tried to remove that 00_umount-against-unused-dirty-inodes-race fix and
> after that (now 5 hours uptime) doing only copying and deleting, that ext3fs
> error is away.
>
> > Of course your problem could be explained by a bad cable or whatever
> > else hardware failure too. At the moment I doubt it's a problem in the
> > common code of my tree or mainline.
> seems it's a problem in the umount-against-unused-dirty-inodes-race fix or if
> the fix "is the right way" the problem is located somewhere else what
> triggers the problem of your patch.
can you reproduce in 2.4.20aa1 too?
Andrea
On Tuesday 03 December 2002 14:59, Andrea Arcangeli wrote:
Hi Andrea,
> > I run e2fsck -fy every time after a crash. Fortunately it doesn't happen
> > so often :-)
> ok ;) I asked just in case.
:-) np.
> > seems it's a problem in the umount-against-unused-dirty-inodes-race fix
> > or if the fix "is the right way" the problem is located somewhere else
> > what triggers the problem of your patch.
> can you reproduce in 2.4.20aa1 too?
I'll give it a try later this evening.
ciao, Marc
On Tuesday 03 December 2002 19:43, Marc-Christian Petersen wrote:
Hi Andrea,
> > can you reproduce in 2.4.20aa1 too?
> I'll give it a try later this evening.
nope, I cannot reproduce it. Seems there is another fix in 2.4.20aa1 that
fixes it.
ciao, Marc