Hello,
I have a memory-related question regarding a Slackware system I'm running
(kernel 2.4.22, compiled for i386, on a VIA Eden 600MHz processor, with
256MB RAM).
First, the background info:
This system is primarily an mp3 player. It boots directly to /bin/bash
bypassing everything under /etc/rc.d, and just runs a small script that does
modprobing and a few other miscellaneous things, then starts the main
control script (which is Perl).
I have about 250 CDs encoded as mp3s on the 40gig drive, laid out in
directories as /music/band/album/songs.mp3. I also have a text file which
contains one line for the full path to each album (so ~250 lines total).
This file is used by my play-random-album function, which plays a random
album each time I press the tick key. When I do that, the songs from the
randomly-chosen album become the current playlist, and the first song starts
playing.
Now, the problem/question:
Each time I press the tick key, a new directory is read from the disk (and a
song is played, and the directory contents are enqueued). This causes the
free memory (as reported by free) to drop by 1%, each time I press tick. I
often press it 5 or 10 or 20 times in a row until I find something I feel
like listening to, and the free memory will drop by almost exactly 5 or 10
or 20 percent by the time I settle on an album. If the free memory was
already very low (for example, when I do this after having already listened
to an album or two), then it will often get down to 2% or 3% at this point,
and stay there until I reboot.
This system has one of those awful on-board "ac'97" sound chips, which
(AIUI) uses the system CPU to do much of the audio work instead of using
dedicated audio chips for it. When the free memory is less than about 10%,
each song will "skip" when it first starts playing. (It sounds like 1/2 or
1 or 2 seconds of audio was deleted from the song; you don't hear silence,
you hear the music at seconds 1, 2, and then 4, or just 2, 3, 4, ...
skipping the very start, etc. But it's always some portion of the first ~5
seconds getting skipped.)
When the system is first booted, the free mem is around 90%, and this
skipping doesn't happen. As time goes on, the free mem drops, and skipping
starts happening somewhere along the line; by the time it's down to ~10%
free, every song skips when it starts.
The disk is a modern one using DMA. And I even set up a ramdisk to try
playing the songs from there (copying the "next" track into ramdisk while
the current track is playing), but it still skips at the beginning of each
track when the free mem is low enough.
My understanding/analysis/proposed solution:
The kernel is buffering the contents of each directory (album) that it reads
(and also, mpg321 copies each mp3 file into RAM before playing it?). I
understand that the idea is to stuff as much into RAM as possible to reduce
pagefile usage, and that the kernel will reclaim memory utilized by buffers
if/when it needs to. But apparently that isn't happening fast enough to
allow a realtime process like music-playing to work skip-free on this system
with this soundcard. I think that if I could regularly forcibly dump the
buffered stuff out of the RAM (dropping the used-RAM percentage down to,
say, 10%, like at boot time) then this would make the skipping stop.
So... do I have a correct understanding of the problem, and a correct
analysis of the kernel/mem issues that are related to it? Is it possible to
clear some of the RAM; if so, would that help?
Thanks,
Anthony
PS - you can see the system at:
http://nodivisions.com/tech/systems/musicbox/
Hello !
On Sat, Nov 01, 2003 at 12:31:15AM -0500, Anthony DiSante wrote:
> The kernel is buffering the contents of each directory (album) that it
> reads (and also, mpg321 copies each mp3 file into RAM before playing it?).
> I understand that the idea is to stuff as much into RAM as possible to
> reduce pagefile usage, and that the kernel will reclaim memory utilized by
> buffers if/when it needs to. But apparently that isn't happening fast
> enough to allow a realtime process like music-playing to work skip-free on
> this system with this soundcard. I think that if I could regularly
> forcibly dump the buffered stuff out of the RAM (dropping the used-RAM
> percentage down to, say, 10%, like at boot time) then this would make the
> skipping stop.
1) are you certain that none of your programs (including mpg321) leaks
memory ? As I understand it, it's really the cache which fills memory.
2) you can try to preload your file into the cache just before playing it :
cp $file /dev/null ; mpg321 $file
> So... do I have a correct understanding of the problem, and a correct
> analysis of the kernel/mem issues that are related to it? Is it possible
> to clear some of the RAM; if so, would that help?
Unless a leak happens somewhere in the kernel, sound driver, etc..., memory
consumed by programs is restored when your programs exit. The only part which
is not restored immediately is the cache.
BTW, there may be one other reason for your problem. Considering that you
scan your disk to find random albums, I think that the system updates all
directories access time after 5s, thus preventing your player from reading
an uncached file fast enough. You might be interested in mounting it with
the 'noatime,nodiratime' options in /etc/fstab.
You might also want to try 2.4.23pre9 which includes VM changes compared to
2.4.22, and seems quite stable to me.
Regards,
Willy
Willy Tarreau wrote:
> Hello !
>
> On Sat, Nov 01, 2003 at 12:31:15AM -0500, Anthony DiSante wrote:
>
>>The kernel is buffering the contents of each directory (album) that it
>>reads (and also, mpg321 copies each mp3 file into RAM before playing it?).
>>I understand that the idea is to stuff as much into RAM as possible to
>>reduce pagefile usage, and that the kernel will reclaim memory utilized by
>>buffers if/when it needs to. But apparently that isn't happening fast
>>enough to allow a realtime process like music-playing to work skip-free on
>>this system with this soundcard. I think that if I could regularly
>>forcibly dump the buffered stuff out of the RAM (dropping the used-RAM
>>percentage down to, say, 10%, like at boot time) then this would make the
>>skipping stop.
>
>
> 1) are you certain that none of your programs (including mpg321) leaks
> memory ? As I understand it, it's really the cache which fills memory.
No, I'm not certain that mpg321 (or any other app, though there's
next-to-nothing else running) is leak-free.
> 2) you can try to preload your file into the cache just before playing it :
> cp $file /dev/null ; mpg321 $file
But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file" accomplish
the same thing? (I've done that.)
>>So... do I have a correct understanding of the problem, and a correct
>>analysis of the kernel/mem issues that are related to it? Is it possible
>>to clear some of the RAM; if so, would that help?
>
>
> Unless a leak happens somewhere in the kernel, sound driver, etc..., memory
> consumed by programs is restored when your programs exit. The only part which
> is not restored immediately is the cache.
>
> BTW, there may be one other reason for your problem. Considering that you
> scan your disk to find random albums, I think that the system updates all
> directories access time after 5s, thus preventing your player from reading
> an uncached file fast enough. You might be interested in mounting it with
> the 'noatime,nodiratime' options in /etc/fstab.
Actually, I don't scan the disk to find random albums; I have a text file
that contains a list of every album's full path, and I pick a random line
from that file. So only the selected album's directory gets scanned. And
the mp3 partition is mounted read-only (I should have mentioned that
before), so the atimes shouldn't be getting written as it is.
> You might also want to try 2.4.23pre9 which includes VM changes compared to
> 2.4.22, and seems quite stable to me.
Maybe I will give that a try.
So I'm guessing that there isn't actually a way to manually move buffer-data
out of RAM?
-Anthony
http://nodivisions.com/
On Sat, Nov 01, 2003 at 01:34:12AM -0500, Anthony DiSante wrote:
> >2) you can try to preload your file into the cache just before playing it :
> > cp $file /dev/null ; mpg321 $file
>
> But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file"
> accomplish the same thing? (I've done that.)
It's different because you need to reserve a fixed amount of ram for the
ramdisk while I was only using the caching features of the kernel as a
fast read-ahead mechanism. But I agree with you that the result should be
the same.
> Actually, I don't scan the disk to find random albums; I have a text file
> that contains a list of every album's full path, and I pick a random line
> from that file. So only the selected album's directory gets scanned. And
> the mp3 partition is mounted read-only (I should have mentioned that
> before), so the atimes shouldn't be getting written as it is.
OK, so it's not a disk IO problem at all. It's really related to the sound
driver it seems. Now that you say it, I remember having noticed skips on
my laptop with a via 82cxxx chip after tens of minutes playing. At first
I thought it was related to other activity on the system, but it did exactly
what you describe, play seconds 1, 2, then 5 without a hole between them.
There may be a problem with the way the audio buffer gets allocated or freed.
> So I'm guessing that there isn't actually a way to manually move
> buffer-data out of RAM?
Yes, there is. I have a quick'n'dirty program which does exactly that.
Basically, you tell it how many kB you want to free, then it allocates
and uses that amount of memory, frees it and exits. Buffered data gets
flushed very quickly. I sometimes give it a try before starting to work
on large kernel trees, because it helps the entire directories to fit in
cache.
Here it is if you're interested. Don't start it without an argument, it
will try to allocate 4G !
Regards,
Willy
---
#include <stdio.h>
main(int argc, char **argv) {
unsigned long int i,k=0, max;
char *p;
if (argc>1)
max=atol(argv[1]);
else
max=0xffffffff;
printf("Freeing %lu kB ram\n",max);
while (((p=(char *)malloc(1048576))!=NULL) && (k+1024<=max)) {
for (i=0;i<256;p[4096*i++]=0);
k+=1024;
fprintf(stderr,"\r%d kB allocated",k);
}
while (((p=(char *)malloc(65536))!=NULL) && (k+64<=max)) {
for (i=0;i<16;p[4096*i++]=0);
k+=64;
fprintf(stderr,"\r%d kB allocated",k);
}
while (((p=(char *)malloc(1024))!=NULL) && (k+1<=max)) {
for (i=0;i<16;p[64*i++]=0);
k+=1;
fprintf(stderr,"\r%d kB allocated",k);
}
fprintf(stderr,"\nMemory freed.\n");
exit(0);
}
> So I'm guessing that there isn't actually a way to manually move buffer-data
> out of RAM?
Just the naive allocate X MB, set the first byte of every 4KB (PageSize)
block to 1 and then free the memory - presto X MB free RAM.
Cheers,
MaZe.
Willy Tarreau wrote:
>>Actually, I don't scan the disk to find random albums; I have a text file
>>that contains a list of every album's full path, and I pick a random line
>>from that file. So only the selected album's directory gets scanned. And
>>the mp3 partition is mounted read-only (I should have mentioned that
>>before), so the atimes shouldn't be getting written as it is.
>
>
> OK, so it's not a disk IO problem at all. It's really related to the sound
> driver it seems. Now that you say it, I remember having noticed skips on
> my laptop with a via 82cxxx chip after tens of minutes playing. At first
> I thought it was related to other activity on the system, but it did exactly
> what you describe, play seconds 1, 2, then 5 without a hole between them.
> There may be a problem with the way the audio buffer gets allocated or freed.
Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one
thing, with both ALSA and OSS drivers, output is distorted unless I keep the
volume below ~70%...). But as I said, the skipping only happens when the
memory is full, or getting close to full. If I select specific albums to
play instead of using my random function, I can usually play a few albums
and there aren't any skips, not until the memory gets pretty full.
>>So I'm guessing that there isn't actually a way to manually move
>>buffer-data out of RAM?
>
>
> Yes, there is. I have a quick'n'dirty program which does exactly that.
> Basically, you tell it how many kB you want to free, then it allocates
> and uses that amount of memory, frees it and exits. Buffered data gets
> flushed very quickly. I sometimes give it a try before starting to work
> on large kernel trees, because it helps the entire directories to fit in
> cache.
>
> Here it is if you're interested. Don't start it without an argument, it
> will try to allocate 4G !
4G!! (Note to self: s/ffffffff/0000ffff/...) Thanks a lot, I'll give this a
try. It certainly works here on my home system.
-Anthony
http://nodivisions.com/
On Sat, 1 Nov 2003, Anthony DiSante wrote:
> Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one
> thing, with both ALSA and OSS drivers, output is distorted unless I keep the
> volume below ~70%...).
It's ok. The AC97 codecs can apply gain (not only attenuation). Thus you
can overdrive your analog output and hear distortion. It's analog
behaviour not a bug.
Jaroslav
-----
Jaroslav Kysela <[email protected]>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs
Hi!
Check your sound driver's sources . Perhaps it allocates GFP_ATOMIC
memory nd drops data on the floor if it is not available.
--
Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...