Hi all,
My PC (x86_64) used to have 4GB RAM and 4GB swap. I found that if I was doing
something very memory-intensive it would thrash, in that the entire PC would
grind to a halt while data was moved in and out of swap. In some cases the PC
would be unresponsive for over 12 hours when I would just give up and reboot.
To avoid this I decided to upgrade the system to 8GB of RAM and disable swap.
I figured the system should work fine without swap given that the total
available memory hasn't changed, it's just that half of it used to be swap and
now it's actual RAM. It's been fine for about a year now.
I was just compiling KDE and had a couple of VirtualBoxes running, which
appeared to max out my memory - it reached 6.9GB in use. I say "appeared" to
max out my memory because the entire system froze again and the disk was
thrashing, just like it used to when I had swap enabled.
I eventually managed to suspend the compile job which stopped all disk
activity, and told VirtualBox to grab a snapshot of the running VMs which took
about 10 minutes (compared to the usual 10 seconds) with the disk being
hammered the whole time and the system unusable (couldn't move the mouse
cursor more than a few pixels a minute.)
Does anyone know what could have caused this 'thrash' to occur? I have
checked and definitely have swap disabled (/proc/swaps is empty) and I have no
swap partitions on my disks anyway. Could it be that the disk cache was
forced to shrink and suddenly all accesses to the filesystem became unbuffered
and incredibly slow? Why would this stop the X11 mouse cursor from moving?
Thanks,
Adam.
On Sun, 2009-11-29 at 14:59 +1000, Adam Nielsen wrote:
> Could it be that the disk cache was
> forced to shrink and suddenly all accesses to the filesystem became unbuffered
> and incredibly slow? Why would this stop the X11 mouse cursor from moving?
Pretty much, yeah. Well, not so much that the filesystem became
unbuffered, the new files you're accessing would have still been pulled
in to RAM, but rather all the old files you were no longer accessing but
were still in memory had to hit the disk before new allocations could
succeed.
Given the modern programmer's love of OOP, pretty much every action you
can think of causes a massive number of (generally small) allocations
and even though each one would likely free shortly after, could still
potentially trigger a dirty page writeback before the allocation would
succeed.
This is one of my pet annoyances - most OOP programmers forget that
object creation is potentially IO-bound in low memory situations (either
by swap or by pagecache writeback) and wonder why seemingly innocuous
things like "new Integer()" can take many seconds to complete!
--Ben.