2002-04-02 08:09:01

by Andrew Ryan

[permalink] [raw]
Subject: v3/tcp mounts occasionally hang for 10+ minutes

Linux client running 2.4.17+NFS_ALL patch set mounting a netapp server (via
switched 100FDx ethernet) with the following mount opts:
rw,tcp,nfsvers=3,rsize=32768,wsize=32768,intr,hard

Works great, but occasionally (every few days) the mount hangs, for about
10 minutes each time. I am not exactly sure what causes the hung mounts but
at times it has seemed to correlate with the machine running very low on
virtual memory and the kernel interceding to kill processes. Other times
this has not seemed to be the case. Either way, it doesn't seem like
something which should be happening.

Eventually, after ~10 minutes, the mount does come back, but in the
meantime this causes some havoc. Several of the following messages are
written to console:
nfs_statfs: statfs error = 512

Interestingly enough, during the time when the hang occurs, other NFS
mounts on this client from the same server work just fine. And
rpcinfo/showmount/etc against the server from the client all work just fine
during the time when the mount is hung.

My questions:
1) Anyone know what the hung mount behavior could be triggered and/or how
to prevent it?
2) If the hung mount behavior couldn't be prevented, could it at least be
mitigated somehow by lowering some timeout to a time less than 10 minutes?
Clients can forgive 30 seconds. But a hang of 10 minutes is sure to
generate help desk calls.


andrew


_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2002-04-02 08:36:14

by Trond Myklebust

[permalink] [raw]
Subject: Re: v3/tcp mounts occasionally hang for 10+ minutes

>>>>> " " == Andrew Ryan <[email protected]> writes:

> Linux client running 2.4.17+NFS_ALL patch set mounting a netapp
> server (via switched 100FDx ethernet) with the following mount
> opts:

> rw,tcp,nfsvers=3,rsize=32768,wsize=32768,intr,hard

> Works great, but occasionally (every few days) the mount hangs,
> for about 10 minutes each time. I am not exactly sure what
> causes the hung mounts but at times it has seemed to correlate
> with the machine running very low on virtual memory and the
> kernel interceding to kill processes. Other times this has not
> seemed to be the case. Either way, it doesn't seem like
> something which should be happening.

If the OOM killer is signalling the rpciod process, then that could
indeed cause the sort of problem you are describing.
If so, please could you look into the circumstances that are causing
the OOM killer to do this. Normally rpciod should never be much of a
memory hog.

Cheers,
Trond

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2002-04-02 15:44:03

by Andrew Ryan

[permalink] [raw]
Subject: Re: v3/tcp mounts occasionally hang for 10+ minutes

At 10:36 AM 4/2/02 +0200, Trond Myklebust wrote:

>If the OOM killer is signalling the rpciod process, then that could
>indeed cause the sort of problem you are describing.
>If so, please could you look into the circumstances that are causing
>the OOM killer to do this. Normally rpciod should never be much of a
>memory hog.


No, rpciod has never been killed by the OOM killer, afaict, the other
processes OOM killer has killed have all been logged and rpciod has never
been in that list. rpciod is always using a small amount of RAM.

Also, I can confirm that this problem has happened when the machine was
fairly low on memory but not low enough to invoke the OOM killer. And as I
noted before, access to other mounts - mounted from the same NFS server -
on the client system is not interrupted when one of these hangs occur.

I realize part of the problem is I haven't been able to reliably identify
exactly what is causing the problem, but I was hoping someone with more
knowledge of the client code or someone who had seen this problem before
could point me in the right direction.


Thanks,
andrew


_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs