2009-06-09 00:20:35

by Simon Kirby

[permalink] [raw]
Subject: RPC: multiple fragments per record not supported

Hello!

With 2.6.28.10 and 2.6.29.4 knfsd after a few hours of heavy concurrent
NFSv3 write loads (migration of Maildirs with rsync), we are seeing this
error:

kernel: RPC: multiple fragments per record not supported

...followed by knfsd immediately ignoring all further requests.
Restarting knfsd (/etc/init.d/nfs-kernel-server restart) seems to make
it start responding and work as normal again.

"ps -eo pid,stat,args,wchan" during the event shows all nfsd processes
sleeping in "svc_recv":

22714 S< [nfsd] svc_recv

This client with the high write load is an old 2.6.17 kernel, mounting
the server with rw,nolock,hard,intr,timeo=9,retrans=3,rsize=8192,wsize=8192
(jumbo frames are enabled).

Is this any sort of known issue? I'm trying to reproduce it in a testing
environment.

Cheers,

Simon-


2009-06-09 02:03:50

by NeilBrown

[permalink] [raw]
Subject: Re: RPC: multiple fragments per record not supported

On Tue, June 9, 2009 10:20 am, Simon Kirby wrote:
> Hello!
>
> With 2.6.28.10 and 2.6.29.4 knfsd after a few hours of heavy concurrent
> NFSv3 write loads (migration of Maildirs with rsync), we are seeing this
> error:
>
> kernel: RPC: multiple fragments per record not supported
>
> ...followed by knfsd immediately ignoring all further requests.
> Restarting knfsd (/etc/init.d/nfs-kernel-server restart) seems to make
> it start responding and work as normal again.
>
> "ps -eo pid,stat,args,wchan" during the event shows all nfsd processes
> sleeping in "svc_recv":
>
> 22714 S< [nfsd] svc_recv
>
> This client with the high write load is an old 2.6.17 kernel, mounting
> the server with
> rw,nolock,hard,intr,timeo=9,retrans=3,rsize=8192,wsize=8192
> (jumbo frames are enabled).
>
> Is this any sort of known issue? I'm trying to reproduce it in a testing
> environment.

Yes. Known issue. The data stream from the client gets corrupted.
Fix is

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=e0ab53deaa91293a7958d63d5a2cf4c5645ad6f0

which is in 2.6.18

NeilBrown