2007-12-30 00:14:11

by Kovacs Gabor

[permalink] [raw]
Subject: NFS client freezes under heavy disk IO

Hello All,

We have some NFS problem on a big partition. NFS (v3) server runs kernel
version 2.6.21.6, client runs kernel version 2.6.21.7. Exported file system
is on XFS file system, 8.9T, typical file size is 1-10M. NFS tools are from
Fedora 7: nfs-utils-1.1.0-3.fc7

On the client side, during heavy read-IO (a program walks on tens of thousands
of files by opening them for reading and collects some data) after a couple
of time (1-5 days) the user process freezes. Top command shows a "D" state
for it, but many other commands trying to "touch" it, freezes, too. (ps,
fuser /nfs_dir, w) making its own virtual console unusable. None of the
frozen processes can be killed. Actually, the computer cannot be shut down
cleanly any more, only the reset button helps.

There was a kernel error:
http://www.mediafire.com/imageview.php?quickkey=0to7xa0eowd

I guess it is some kind of out of memory condition. Is this possible? How much
memory required by kernel nfs (client) modules? Any idea to try?
(System has 16G memory, swap is 32G, user program consumed approx. 12G.)

A very similar deadlock was produced using NFSv4.

Thanks
Gabor Kovacs