It happened twice recently(within one week), therefore I'm posting here.
32bit Fedora7, 2.6.23.17-88.fc7PAE, 16G ram, 12G swap, 2*dualXeon
My machine went sluggish. Tried to debug it, and observed, that free
ram(the unused one) was increasing while swap usage was increasing as
well. It stopped after whole swap partition was used. Then it came back
normal. Required processes were brought back to ram and server become
responsible again.
Now the most strange thing is that there was 5G free(unused) ram, during
those few minutes it growed to 12G! In the very same time swap usage
increased from ~4G to 12G.
This is irrational to me. Can anybody explain? How to prevent?
vm.swappiness was decreased to 30, to avoid excessive swap usage, but no
avail.
It's application and nfs server/client if it matters. During that
"memory sweep" I observed many nfsd processes(kernel threads?) in D state.
On Wed, 2008-07-23 at 18:53 +0200, Rafal Wijata wrote:
> It happened twice recently(within one week), therefore I'm posting here.
>
> 32bit Fedora7, 2.6.23.17-88.fc7PAE, 16G ram, 12G swap, 2*dualXeon
16g ram on a 32bit kernel is stretching it... may I suggest running a
64bit kernel?
Peter Zijlstra wrote:
> 16g ram on a 32bit kernel is stretching it... may I suggest running a
> 64bit kernel?
>
Is it? Do we expect that behaviour with so much RAM?
David Newall wrote:
> Peter Zijlstra wrote:
>> 16g ram on a 32bit kernel is stretching it... may I suggest running a
>> 64bit kernel?
>>
>
> Is it? Do we expect that behaviour with so much RAM?
The memory must be mapped into the 4GB (32-bit) virtual address space to
be used. Having this much memory on a system with 32 bit pointers leads
to a lot of VM overhead and strange behavior.
Cheers,
Dan
--
/--------------- - - - - - -
| Dan No?
| http://isomerica.net/~dpn/
On Thu, 2008-07-24 at 06:47 +0930, David Newall wrote:
> Peter Zijlstra wrote:
> > 16g ram on a 32bit kernel is stretching it... may I suggest running a
> > 64bit kernel?
> >
>
> Is it? Do we expect that behaviour with so much RAM?
I expect your 32bit machine to go wonky after say 8gb,.. maybe even
earlier for some workloads.
The thing is, 32bit machines have less than 1gb of kernel memory - the
more memory you have the less of that remains for system duties since
such things as the page frame array grow with the number of physical
pages present.
Also, all such things as buffer cache reside in that tiny space. At some
point the kernel itself is just going to run out of memory trying to
manage your machine.
Hi Rafal
> vm.swappiness was decreased to 30, to avoid excessive swap usage, but no
> avail.
> It's application and nfs server/client if it matters. During that
> "memory sweep" I observed many nfsd processes(kernel threads?) in D state.
I think no related.
nfsd always remain D state.
because if nfsd waiting to new packet, ps (or other tool) display "D".
OK, things are little more clear now I suppose.
I tried (just for fun) echo 8 > /proc/sys/vm/nr_hugepages
and it happened again, and no hugepages were allocated of course.
But this brought the following to my mind.
Since it's nfs server/client, and the nfs is rather heavily utilized(at
least sometimes), maybe nfsd tasks were unable to find long enough lowmem
chunk. How can we get such ram? By sweeping pages out to swap, isn't it?
Now the question is, if Linux actually has such feature like blind swapping
out if lowmem is too fragmented to find requested size?
Can anybody confirm or negate?
Next, I was thinking then I could run 64bit system, but I don't want to
reinstall it. Anybody knows if it's possible to have installed 32bit OS, but
run 64bit kernel alone(no 64bit glibc or anything else)?
Thanx in advance.
On Thu, 2008-07-24 at 15:45 +0200, Rafal Wijata (NULL) wrote:
> OK, things are little more clear now I suppose.
>
> I tried (just for fun) echo 8 > /proc/sys/vm/nr_hugepages
> and it happened again, and no hugepages were allocated of course.
>
> But this brought the following to my mind.
>
> Since it's nfs server/client, and the nfs is rather heavily utilized(at
> least sometimes), maybe nfsd tasks were unable to find long enough lowmem
> chunk. How can we get such ram? By sweeping pages out to swap, isn't it?
>
> Now the question is, if Linux actually has such feature like blind swapping
> out if lowmem is too fragmented to find requested size?
> Can anybody confirm or negate?
Well, not blind - we have lumpy reclaim that tries to focus on the right
spot for large order allocations. However there are a myrad of
conditions that can make it impossible to reclaim certain pages,
therefore defeating this strategy.
> Next, I was thinking then I could run 64bit system, but I don't want to
> reinstall it. Anybody knows if it's possible to have installed 32bit OS, but
> run 64bit kernel alone(no 64bit glibc or anything else)?
Yes running as I type ;-)