On a 2.4.21 kernel (kernel.org + KDB patch + MITRE security patches),
I've seen bizarre (I think) behavior from kswapd.
My question boils down to this: given the (simple) scenario below,
am I missing critical VM/kswapd patches, or is this behavior
expected and OK, or is it wrong and possibly fixed in the 2.6 kernel?
Or is the kswapd behavior somehow tunable to avoid the apparent
thrashing that I saw?
I had a large source tree on an ext3 filesystem on machine A, and
did a 'make' from machine B via NFS. Then I did the equivalent of
'make clean' on machine A itself, and instead of taking maybe 30
seconds, the clean took 10 minutes. During all of that time, kswapd
ran at around 50% CPU, and kjournald was also extremely busy:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
49161 root 12 0 616 616 488 D 46.5 0.0 3:41.43 find
5 root 19 0 0 0 0 R 40.9 0.0 503:03.26 kswapd
261 root 19 0 0 0 0 R 33.2 0.0 54:22.41 kjournald
51794 root 12 0 948 948 720 R 1.7 0.0 0:02.23 top
Meanwhile, memory was pretty constant. Here's a brief snapshot:
# vmstat 2 100000
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us
sy id wa
3 0 30876 245536 630012 3865628 0 0 21 34 27 2 3
14 83 0
3 0 30876 245308 629968 3865628 0 0 50 0 119 77 0
98 2 0
4 1 30876 245284 629996 3865628 0 0 134 0 140 114 0
98 1 0
Out of 5 GB of memory, only 1/4 GB was free, and 3.8 GB was used
by the page cache. I'm wondering if possibly the 'find/rm' that the
clean was doing freed pages in the cache and that the 5% of memory
that was free is somehow a significant number that caused kswapd
to work like crazy while the find/rm freed cached file data up.
I should say that some KDB backtrace samples from kswapd
showed that it was busy doing kswapd_balance_pgdat().
(I assume the kjournald activity is expected due to the amount of
metadata
updates that the clean would be generating?)
Thanks,
Chris Johns
On Tue, Jun 08, 2004 at 10:41:32AM -0500, Chris Johns wrote:
> On a 2.4.21 kernel (kernel.org + KDB patch + MITRE security patches),
> I've seen bizarre (I think) behavior from kswapd.
>
> My question boils down to this: given the (simple) scenario below,
> am I missing critical VM/kswapd patches, or is this behavior
> expected and OK, or is it wrong and possibly fixed in the 2.6 kernel?
> Or is the kswapd behavior somehow tunable to avoid the apparent
> thrashing that I saw?
Recent 2.4 VM should fix this, but you better of use 2.6.
> Recent 2.4 VM should fix this, but you better of use 2.6.
>
Thanks Marcelo. Do you know of specific patches that have
gone into 2.4 that might fix this? I may be able to just apply them
rather than try a whole new kernel release.
Thanks,
Chris
> >
> > My question boils down to this: given the (simple) scenario below,
> > am I missing critical VM/kswapd patches, or is this behavior
> > expected and OK, or is it wrong and possibly fixed in the 2.6
> kernel?> Or is the kswapd behavior somehow tunable to avoid the
> apparent> thrashing that I saw?
>
On Wed, Jun 09, 2004 at 02:23:58PM -0500, [email protected] wrote:
> > Recent 2.4 VM should fix this, but you better of use 2.6.
> >
>
> Thanks Marcelo. Do you know of specific patches that have
> gone into 2.4 that might fix this? I may be able to just apply them
> rather than try a whole new kernel release.
The -aa VM merge during 2.4.23, most notable 2.4.23-pre4:
o aa VM merge: Per-zone watermark changes, add lower_zone_reserve_ratio
o aa VM merge: page reclaiming logic changes: Kills oom killer
o aa VM merge: Page accounting helpers changes
o aa VM merge: tunables
o aa VM merge: Kill PF_MEMDIE
o aa VM merge: Fixup page reclaiming changes patch
That plus a few merge mistakes fixed during later 2.4.23-pre.