Hello,
I am running 2.4.18-3 version of the kernel on smp dual
processor and 1GB of RAM. My memory usage is increasing and
I can't find what exactly is eating memory. Top and proc
are reporting increases, but I would like to know of a
better way of tracing usage of memory and possible leak in
application(s).
Please reply to [email protected]
thanks Sasha
=====
------
* "Last night I dreamed I ate a ten-pound marshmallow, and when I woke up the pillow was gone."
__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com
On Mon, 26 Aug 2002, Aleksandar Kacanski wrote:
> Hello,
> I am running 2.4.18-3 version of the kernel on smp dual
> processor and 1GB of RAM. My memory usage is increasing and
> I can't find what exactly is eating memory. Top and proc
> are reporting increases, but I would like to know of a
> better way of tracing usage of memory and possible leak in
> application(s).
>
> Please reply to [email protected]
> thanks Sasha
>
>
Applications that use malloc() and friends, get more memory from
the kernel by resetting the break address. It's called "morecore()".
You can put a procedure, perhaps off SIGALRM, that periodically
checks the break address and writes it to a log. Applications
that end up with an ever-increasing break address have memory
leaks. Note that the break address is almost never set back.
This is not an error; malloc() assumes that if you used a lot
of memory once, you'll probably use it again. Check out sbrk()
and brk() in the man pages.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
The US military has given us many words, FUBAR, SNAFU, now ENRON.
Yes, top management were graduates of West Point and Annapolis.
"Richard B. Johnson" wrote:
>
> On Mon, 26 Aug 2002, Aleksandar Kacanski wrote:
>
> > Hello,
> > I am running 2.4.18-3 version of the kernel on smp dual
> > processor and 1GB of RAM. My memory usage is increasing and
> > I can't find what exactly is eating memory. Top and proc
> > are reporting increases, but I would like to know of a
> > better way of tracing usage of memory and possible leak in
> > application(s).
> >
> > Please reply to [email protected]
> > thanks Sasha
> >
> >
>
> Applications that use malloc() and friends, get more memory from
> the kernel by resetting the break address. It's called "morecore()".
> You can put a procedure, perhaps off SIGALRM, that periodically
> checks the break address and writes it to a log. Applications
> that end up with an ever-increasing break address have memory
> leaks. Note that the break address is almost never set back.
> This is not an error; malloc() assumes that if you used a lot
> of memory once, you'll probably use it again. Check out sbrk()
> and brk() in the man pages.
But this all comes back when the application ends. You
should be able to see the memory reappear when the app
terminates.
--
George Anzinger [email protected]
High-res-timers:
http://sourceforge.net/projects/high-res-timers/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml
On Mon, 26 Aug 2002, george anzinger wrote:
> "Richard B. Johnson" wrote:
> >
> > On Mon, 26 Aug 2002, Aleksandar Kacanski wrote:
> >
> > > Hello,
> > > I am running 2.4.18-3 version of the kernel on smp dual
> > > processor and 1GB of RAM. My memory usage is increasing and
> > > I can't find what exactly is eating memory. Top and proc
> > > are reporting increases, but I would like to know of a
> > > better way of tracing usage of memory and possible leak in
> > > application(s).
> > >
> > > Please reply to [email protected]
> > > thanks Sasha
> > >
> > >
> >
> > Applications that use malloc() and friends, get more memory from
> > the kernel by resetting the break address. It's called "morecore()".
> > You can put a procedure, perhaps off SIGALRM, that periodically
> > checks the break address and writes it to a log. Applications
> > that end up with an ever-increasing break address have memory
> > leaks. Note that the break address is almost never set back.
> > This is not an error; malloc() assumes that if you used a lot
> > of memory once, you'll probably use it again. Check out sbrk()
> > and brk() in the man pages.
>
> But this all comes back when the application ends. You
> should be able to see the memory reappear when the app
> terminates.
>
Not by looking at /proc/meminfo. The fact that Linux redistributes
buffers, etc., during normal operation has often been misconceived
as a memory leak. Linux will try to use all available memory. This
is because unused memory is wasted memory. When Linux first starts
up, there is little to do, and very little history, so it hasn't
gotten a chance to use much memory. If you run a few apps, compile
the kernel, etc., you get to use all the memory. Then when the
system is quiescent, often persons look at /proc/meminfo, note that
a lot of memory is being used, and declare that there must be some
kind of memory leak.
There just might be some kind of memory leak, but, you can't tell
it from looking at /proc/meminfo. You would need to instrument each
procedure that allocates memory and verify that all execution paths,
including exceptions, result in that memory being freed. This would
be a good project for one of the college students that is looking
to get his/her hands dirty. It's a lot more useful than 'inventing'
some new file-system that never gets used, etc.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
The US military has given us many words, FUBAR, SNAFU, now ENRON.
Yes, top management were graduates of West Point and Annapolis.
On Mon, Aug 26, 2002 at 03:29:17PM -0400, Richard B. Johnson wrote:
> On Mon, 26 Aug 2002, Aleksandar Kacanski wrote:
>
> > Hello,
> > I am running 2.4.18-3 version of the kernel on smp dual
> > processor and 1GB of RAM. My memory usage is increasing and
> > I can't find what exactly is eating memory. Top and proc
> > are reporting increases, but I would like to know of a
> > better way of tracing usage of memory and possible leak in
> > application(s).
> >
> > Please reply to [email protected]
> > thanks Sasha
> >
> >
>
> Applications that use malloc() and friends, get more memory from
> the kernel by resetting the break address. It's called "morecore()".
> You can put a procedure, perhaps off SIGALRM, that periodically
> checks the break address and writes it to a log. Applications
> that end up with an ever-increasing break address have memory
> leaks. Note that the break address is almost never set back.
> This is not an error; malloc() assumes that if you used a lot
> of memory once, you'll probably use it again. Check out sbrk()
> and brk() in the man pages.
brk() isn't the only way applications get memory. A
commonly used method of getting process memory is to mmap()
anonymous regions. Some implementations of malloc() will
use mmap() for larger allocations. If you use realloc() a
lot this can be particularly useful.
Releasing memory that has been allocated via brk() is
dependent on allocation order so it is seldom done.
Realloc() and co. used poorly can produce lots of
fragmentation exacerbating things. Allocations done via
mmap() tend to be independent of one another so they can
released more readily.
If an application or library is using mmap() just watching
brk won't do much good. Actually detecting leakage is very
difficult especially from outside the code. It requires
tracking the modifications of all the pointers to allocated
memory.
The thing to keep in mind for the common case is that
applications that leak memory will release it on exit.
Most of the time any sizable amount of application memory is
in disuse it will be in sizable chunks so under memory
pressure will be paged out until exit so will have minimal
impact on the system.
Having said all that i'll now echo Linus et al and tell you
that you want the system to use all of your memory. Unused
memory is wasted memory and wasted money. That is why
otherwise free memory is used as cache and disused memory
will be paged out so that it too can be used as cache to
speed up the system as a whole.
And much thanks to all the good, and heartless bastard,
developers who have and continue to work on improving the
paging algorithms and VM system as a whole.
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [email protected]
Remember Cernan and Schmitt
At 04:05 PM 8/26/2002 -0400, Richard B. Johnson wrote:
>There just might be some kind of memory leak, but, you can't tell
>it from looking at /proc/meminfo. You would need to instrument each
>procedure that allocates memory [...]
Ingo did that a long time ago in his memleak patch. Unfortunately, it's
laying around unmaintained, as I ran out of time. If someone wants a
(kernel) memory leak detector, all they have to is rip it out of the IKD
patch and update it (trivial).
-Mike