2001-11-29 20:31:00

by space-00002

[permalink] [raw]
Subject: buffer/memory strangeness in 2.4.16

Hi,

I am experiencing a bit of strange system behaviour in a vanilla 2.4.16
kernel (2.95.3, very stable machine etc.)

I noticed, that after running for a while (day) I had significantly less
memory available for my simulation program than right after booting. Looking
at the problem using 'xosview' (or 'free'), I noticed that there was a large
number of MBs filled with 'buffers' that did not get wiped when other
programs need the memory. The system seems to rather kill an 'offender' than
clean out buffers.

Right after booting, I can allocate about 650MBs memory using the little
program attached below. After a day (or after running updatedb), under the
same conditions, even in single user mode with only a shell running (!) this
is not possible anymore and the program (below), trying to allocate only
300-400MBs, gets killed by the system after making it unresponsive for many
seconds.

Apparently this problem occurs after running 'updatedb', which fills 'free
memory' and generates lots of filled cache and buffers on my system.

This sort of behaviour must have been introduced after 2.4.13, which does not
show these problems.

Please tell me if somebody needs more information to debug this, or if this
behaviour is normal or expected. Please cc: me as I am only on lkml-digest.

Cheers
Jan


P.S. All RAM slots are full, so please don't suggest buying more memory as a
solution :^)

-------------------%<-----------------------

#include <stdio.h>
#define ONE_MEG 1024 * 1024

main ()
{
long mem_avail = ONE_MEG;
char *buf;
char userchar = '@';
int howmany;

while (1)
{
printf ("Number of MBs to allocate? ");
scanf ("%d", &howmany);
printf ("Trying to allocate %ld bytes: ", mem_avail*howmany);

getchar ();
if ((buf = (char *) malloc ((size_t) mem_avail*howmany))){
printf (" success!\n");
printf ("Now filling it up...\n");
memset (buf, userchar, mem_avail * howmany);
printf ("Hit ENTER to free the memory.\n");
getchar ();
free (buf);
} else {
printf (" failed :(\n");
}
}
}

-------------------%<-----------------------


2001-11-30 09:30:49

by Andrew Morton

[permalink] [raw]
Subject: Re: buffer/memory strangeness in 2.4.16

[email protected] wrote:
>
> Hi,
>
> I am experiencing a bit of strange system behaviour in a vanilla 2.4.16
> kernel (2.95.3, very stable machine etc.)
>
> I noticed, that after running for a while (day) I had significantly less
> memory available for my simulation program than right after booting. Looking
> at the problem using 'xosview' (or 'free'), I noticed that there was a large
> number of MBs filled with 'buffers' that did not get wiped when other
> programs need the memory. The system seems to rather kill an 'offender' than
> clean out buffers.
>
>

Seconded. After an updatedb run, my 768 megabyte 2.5.1-pre4 machine
shows:

total used free shared buffers cached
Mem: 770668 384460 386208 0 138548 17744
-/+ buffers/cache: 228168 542500

and, after malloc/memset of 700 megs:

total used free shared buffers cached
Mem: 770668 73340 697328 0 41160 5960
-/+ buffers/cache: 26220 744448
Swap: 499928 18628 481300

I repeated the malloc/memset a few times, wrote a gigabyte file
and was unable to make the 40 megabytes of buffermem go away.

MemTotal: 770668 kB
MemFree: 698008 kB
MemShared: 0 kB
Buffers: 42092 kB
Cached: 6088 kB
SwapCached: 9808 kB
Active: 48064 kB
Inactive: 10112 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 770668 kB
LowFree: 698008 kB
SwapTotal: 499928 kB
SwapFree: 484512 kB

After running an extremely memory-intensive test program for
two minutes, buffermem fell to 38 megabytes.

Seems broken to me.

-