I tried mounting an NFS server running FC8 (2.6.26.8-57.fc8 kernel)
using a Fedora 11 system running an un-patched 2.6.31-rc3 64-bit kernel.
I am not sure at all that the FC8 system is set up to handle NFSv4 properly, but
I was expecting some sort of useful error if that was the case.
Instead, I get this continually spewing to /var/log/messages:
Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
...
On the file-server, I see this:
Jul 19 04:47:59 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
Jul 19 04:49:29 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
Jul 19 04:49:29 fs2 ntpd[2585]: kernel time sync status change 0001
Jul 19 04:50:59 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
I added some debug patches (on top of my other patches, including those to nfs)
and got some debug info:
It seems that nfs4_init_clientid is returning -2, and establish_clid is returning
-2. This causes reclaim lease logic to fail, and that causes state manager to print
out the error repeatedly. -2 means ENOENT.
As far as I can tell, the mount never completes, staying in D state and filling up
logs (I deleted a 16GB /var/log/messages file a few minutes ago!)
The mount command I'm trying is:
mount -t nfs4 192.168.100.6:/export/tmp /mnt/lf/nfs4-0
My kernel config is found here:
http://www.candelatech.com/oss/i7_config.txt
Thanks,
Ben
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Mon, 2009-07-20 at 15:09 -0700, Ben Greear wrote:
> I tried mounting an NFS server running FC8 (2.6.26.8-57.fc8 kernel)
> using a Fedora 11 system running an un-patched 2.6.31-rc3 64-bit kernel.
>
> I am not sure at all that the FC8 system is set up to handle NFSv4 properly, but
> I was expecting some sort of useful error if that was the case.
>
> Instead, I get this continually spewing to /var/log/messages:
>
> Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
> Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
> Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
> Error: state manager failed on NFSv4 server 192.168.100.6 with error 2
> ...
>
> On the file-server, I see this:
> Jul 19 04:47:59 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
> Jul 19 04:49:29 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
> Jul 19 04:49:29 fs2 ntpd[2585]: kernel time sync status change 0001
> Jul 19 04:50:59 fs2 kernel: nfs4_cb: server 192.168.100.196 not responding, timed out
>
>
> I added some debug patches (on top of my other patches, including those to nfs)
> and got some debug info:
>
> It seems that nfs4_init_clientid is returning -2, and establish_clid is returning
> -2. This causes reclaim lease logic to fail, and that causes state manager to print
> out the error repeatedly. -2 means ENOENT.
>
>
> As far as I can tell, the mount never completes, staying in D state and filling up
> logs (I deleted a 16GB /var/log/messages file a few minutes ago!)
>
> The mount command I'm trying is:
> mount -t nfs4 192.168.100.6:/export/tmp /mnt/lf/nfs4-0
>
>
> My kernel config is found here:
> http://www.candelatech.com/oss/i7_config.txt
>
What does /var/lib/nfs/v4recovery look like on the server?
Trond
On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>
> What does /var/lib/nfs/v4recovery look like on the server?
The server was misconfigured, but I still think the client should
behave better in this case. If you cannot reproduce it, let me know
and I can try to be more specific. If you still want the v4recovery
information, let me know and I'll send it.
I had my /etc/exports looking like this on the file-server:
/export/tmp 192.168.100.0/24(rw)
When I changed it to be:
/export/tmp 192.168.100.0/24(rw,fsid=0)
And mounted 192.168.100.6:/ instead of 192.168.100.6:/export/tmp
then it mounts properly and appears to work just fine.
If I leave file-server configured properly with fsid=0, but try
to mount 192.168.100.6:/export/tmp then I get an error about
no such file or directory, which also appears to be correct
behaviour.
Thanks,
Ben
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>
> >
> > What does /var/lib/nfs/v4recovery look like on the server?
>
> The server was misconfigured, but I still think the client should
> behave better in this case. If you cannot reproduce it, let me know
> and I can try to be more specific. If you still want the v4recovery
> information, let me know and I'll send it.
So how should the client behave, when a screwed up server allows it to
mount but starts returning illegal values for setclientid? The only
thing I can see we could do is to tell the user EINSANESERVER...
Now, we _should_ fix the wretched NFS server so that it doesn't do NFSv4
mounts when there is no configured root partition. We _should_ also fix
the damned thing so that it doesn't return illegal values.
Trond
On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>>
>>> What does /var/lib/nfs/v4recovery look like on the server?
>> The server was misconfigured, but I still think the client should
>> behave better in this case. If you cannot reproduce it, let me know
>> and I can try to be more specific. If you still want the v4recovery
>> information, let me know and I'll send it.
>
> So how should the client behave, when a screwed up server allows it to
> mount but starts returning illegal values for setclientid? The only
> thing I can see we could do is to tell the user EINSANESERVER...
Well, it could just fail the mount and give up and not overly spam
/var/log/messages in a tight loop perhaps?
> Now, we _should_ fix the wretched NFS server so that it doesn't do NFSv4
> mounts when there is no configured root partition. We _should_ also fix
> the damned thing so that it doesn't return illegal values.
Sounds fine to me.
Thanks,
Ben
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
Ben Greear wrote:
> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>> What does /var/lib/nfs/v4recovery look like on the server?
>
> The server was misconfigured, but I still think the client should
> behave better in this case. If you cannot reproduce it, let me know
> and I can try to be more specific. If you still want the v4recovery
> information, let me know and I'll send it.
>
> I had my /etc/exports looking like this on the file-server:
>
> /export/tmp 192.168.100.0/24(rw)
>
> When I changed it to be:
> /export/tmp 192.168.100.0/24(rw,fsid=0)
>
> And mounted 192.168.100.6:/ instead of 192.168.100.6:/export/tmp
> then it mounts properly and appears to work just fine.
>
> If I leave file-server configured properly with fsid=0, but try
> to mount 192.168.100.6:/export/tmp then I get an error about
> no such file or directory, which also appears to be correct
> behaviour.
Duh! I had exactly the same problem this week when I "quickly" set up an
NFS server on one of my boxes: forgot to add fsid=0 for the NFS4 root.
Result was the mount hanging on the client and the same errors as Ben
reported.
It would be great if this configuration problem was detected better.
P.S. /var/lib/nfs/v4recovery on the server was empty in my case.
On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> > On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> >> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
> >>
> >>> What does /var/lib/nfs/v4recovery look like on the server?
> >> The server was misconfigured, but I still think the client should
> >> behave better in this case. If you cannot reproduce it, let me know
> >> and I can try to be more specific. If you still want the v4recovery
> >> information, let me know and I'll send it.
> >
> > So how should the client behave, when a screwed up server allows it to
> > mount but starts returning illegal values for setclientid? The only
> > thing I can see we could do is to tell the user EINSANESERVER...
>
> Well, it could just fail the mount and give up and not overly spam
> /var/log/messages in a tight loop perhaps?
This doesn't happen at mount time. It happens when you open a file.
Trond
On 07/21/2009 10:59 AM, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>>>>
>>>>> What does /var/lib/nfs/v4recovery look like on the server?
>>>> The server was misconfigured, but I still think the client should
>>>> behave better in this case. If you cannot reproduce it, let me know
>>>> and I can try to be more specific. If you still want the v4recovery
>>>> information, let me know and I'll send it.
>>> So how should the client behave, when a screwed up server allows it to
>>> mount but starts returning illegal values for setclientid? The only
>>> thing I can see we could do is to tell the user EINSANESERVER...
>> Well, it could just fail the mount and give up and not overly spam
>> /var/log/messages in a tight loop perhaps?
>
> This doesn't happen at mount time. It happens when you open a file.
Not for me, and evidently not for the other person that reported
similar results. All I had to do was attempt the mount (which never
completed).
Thanks,
Ben
>
> Trond
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
> > On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
> >> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> >>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> >>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
> >>>>
> >>>>> What does /var/lib/nfs/v4recovery look like on the server?
> >>>> The server was misconfigured, but I still think the client should
> >>>> behave better in this case. If you cannot reproduce it, let me know
> >>>> and I can try to be more specific. If you still want the v4recovery
> >>>> information, let me know and I'll send it.
> >>> So how should the client behave, when a screwed up server allows it to
> >>> mount but starts returning illegal values for setclientid? The only
> >>> thing I can see we could do is to tell the user EINSANESERVER...
> >> Well, it could just fail the mount and give up and not overly spam
> >> /var/log/messages in a tight loop perhaps?
> >
> > This doesn't happen at mount time. It happens when you open a file.
>
> Not for me, and evidently not for the other person that reported
> similar results. All I had to do was attempt the mount (which never
> completed).
>
> Thanks,
> Ben
Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
bug occur when you turn this off too?
Trond
On 07/21/2009 11:28 AM, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
>> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
>>> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
>>>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
>>>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
>>>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>>>>>>
>>>>>>> What does /var/lib/nfs/v4recovery look like on the server?
>>>>>> The server was misconfigured, but I still think the client should
>>>>>> behave better in this case. If you cannot reproduce it, let me know
>>>>>> and I can try to be more specific. If you still want the v4recovery
>>>>>> information, let me know and I'll send it.
>>>>> So how should the client behave, when a screwed up server allows it to
>>>>> mount but starts returning illegal values for setclientid? The only
>>>>> thing I can see we could do is to tell the user EINSANESERVER...
>>>> Well, it could just fail the mount and give up and not overly spam
>>>> /var/log/messages in a tight loop perhaps?
>>> This doesn't happen at mount time. It happens when you open a file.
>> Not for me, and evidently not for the other person that reported
>> similar results. All I had to do was attempt the mount (which never
>> completed).
>>
>> Thanks,
>> Ben
>
> Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> bug occur when you turn this off too?
Yes, it did.
Thanks,
Ben
>
> Trond
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Tue, 2009-07-21 at 11:32 -0700, Ben Greear wrote:
> On 07/21/2009 11:28 AM, Trond Myklebust wrote:
> > On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
> >> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
> >>> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
> >>>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> >>>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> >>>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
> >>>>>>
> >>>>>>> What does /var/lib/nfs/v4recovery look like on the server?
> >>>>>> The server was misconfigured, but I still think the client should
> >>>>>> behave better in this case. If you cannot reproduce it, let me know
> >>>>>> and I can try to be more specific. If you still want the v4recovery
> >>>>>> information, let me know and I'll send it.
> >>>>> So how should the client behave, when a screwed up server allows it to
> >>>>> mount but starts returning illegal values for setclientid? The only
> >>>>> thing I can see we could do is to tell the user EINSANESERVER...
> >>>> Well, it could just fail the mount and give up and not overly spam
> >>>> /var/log/messages in a tight loop perhaps?
> >>> This doesn't happen at mount time. It happens when you open a file.
> >> Not for me, and evidently not for the other person that reported
> >> similar results. All I had to do was attempt the mount (which never
> >> completed).
> >>
> >> Thanks,
> >> Ben
> >
> > Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> > bug occur when you turn this off too?
>
> Yes, it did.
OK. Then the NFSv4.1 merge has screwed up the state management. We do
NOT want to hold client id state on the server when there is no file
state...
Trond
Trond Myklebust wrote:
> Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> bug occur when you turn this off too?
Eh, no. In my case both client and server are 2.6.30 which did not yet
have NFS_V4_1 (and in all my .31 configs I did disable it).
On Tue, 2009-07-21 at 21:23 +0200, Frans Pop wrote:
> Trond Myklebust wrote:
> > Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> > bug occur when you turn this off too?
>
> Eh, no. In my case both client and server are 2.6.30 which did not yet
> have NFS_V4_1 (and in all my .31 configs I did disable it).
...and you are seeing the state manager running when you try to mount an
NFSv4 partition using the 2.6.30 client?
Trond
Trond Myklebust wrote:
> OK. Then the NFSv4.1 merge has screwed up the state management. We do
> NOT want to hold client id state on the server when there is no file
> state...
Hmm. I doubt that as I can reproduce the same problem (at least the mount
hanging, not necessarily the errors) with .30, which I presume is before
the merge.
I have seen the errors too, but that may have been with 2.6.31 (though
definitely without NFS_V4_1). I seem unable to reproduce those with .30.
On the server (.30) in /etc/exportfs:
/srv/exports 10.19.66.0/24(rw,subtree_check,sync,mp,nohide)
/srv/exports/data 10.19.66.0/24(rw,subtree_check,sync,mp,nohide)
# exportfs
/srv/exports 10.19.66.0/24
/srv/exports/data 10.19.66.0/24
On the client (.30.2):
# mount thorin:/data /mnt -t nfs4
<hangs>
Cheers,
FJP
On Tuesday 21 July 2009, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 21:23 +0200, Frans Pop wrote:
> > Trond Myklebust wrote:
> > > Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does
> > > the bug occur when you turn this off too?
> >
> > Eh, no. In my case both client and server are 2.6.30 which did not
> > yet have NFS_V4_1 (and in all my .31 configs I did disable it).
>
> ...and you are seeing the state manager running when you try to mount
> an NFSv4 partition using the 2.6.30 client?
Per my previous message, I think there may be two related issues:
- mount hanging when no NFS4 root is defined, reproducible on .30
- state errors occurring in that situation, which may have been
introduced with .31
I've just checked and when I saw the state errors I was running .31-rc3 on
the client, and partially also on the server. I saw error 2, 5 and 10022.
See log below.
I first switched back to .30 on the server to see if that fixed the mount
problem (it didn't) and later also switched back to .30 on the client.
(Sorry for the kernel version confusion, but I've been working on
different issues requiring frequent reboots; but I hope the info helps.)
Cheers,
FJP
Log from the client:
Jul 19 20:10:16 aragorn kernel: Linux version 2.6.31-rc3
[...]
Jul 20 00:44:45 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 2
Jul 20 00:49:27 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 00:52:30 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 00:55:30 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 00:59:39 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:01:15 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
[... server rebooted from 2.6.31-rc3 to 2.6.30 ...]
Jul 20 01:06:37 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:09:37 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:12:37 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:15:37 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:18:37 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 10022
Jul 20 01:23:52 aragorn kernel: Error: state manager failed on NFSv4 server thorin with error 5
Unrelated, but I just noticed the following on the server (with 2.6.30);
not sure what exactly triggered that, maybe a stop/start of the NFS services:
Jul 20 01:23:22 thorin kernel: RPC: Unregistered udp transport module.
Jul 20 01:23:22 thorin kernel: RPC: Unregistered tcp transport module.
Jul 20 01:23:22 thorin kernel: nfsd: failed to unregister auth.unix.gid cache
Jul 20 01:23:22 thorin kernel: ------------[ cut here ]------------
Jul 20 01:23:22 thorin kernel: WARNING: at /home/fjp/projects/kernel/linux-2.6/fs/proc/generic.c:847
remove_proc_entry+0x164/0x1b0()
Jul 20 01:23:22 thorin kernel: remove_proc_entry: removing non-empty directory 'net/rpc', leaking at least 'auth.unix.gid'
Jul 20 01:23:22 thorin kernel: Modules linked in: ipv6 exportfs sunrpc(-) dm_mod loop evdev ehci_hcd usbcore mv643xx_eth libphy
gpio_keys [last unloaded: auth_rpcgss]
Jul 20 01:23:22 thorin kernel: [<c002ab6c>] (unwind_backtrace+0x0/0xd4) from [<c003fd78>] (warn_slowpath_common+0x4c/0x68)
Jul 20 01:23:22 thorin kernel: [<c003fd78>] (warn_slowpath_common+0x4c/0x68) from [<c003fdd0>] (warn_slowpath_fmt+0x28/0x38)
Jul 20 01:23:22 thorin kernel: [<c003fdd0>] (warn_slowpath_fmt+0x28/0x38) from [<c00e43c4>] (remove_proc_entry+0x164/0x1b0)
Jul 20 01:23:22 thorin kernel: [<c00e43c4>] (remove_proc_entry+0x164/0x1b0) from [<c00628b8>] (sys_delete_module+0x1d0/0x230)
Jul 20 01:23:22 thorin kernel: [<c00628b8>] (sys_delete_module+0x1d0/0x230) from [<c0024ce0>] (ret_fast_syscall+0x0/0x2c)
Jul 20 01:23:22 thorin kernel: ---[ end trace b5320efdfa0e8be3 ]---
On Tue, 2009-07-21 at 11:32 -0700, Ben Greear wrote:
> On 07/21/2009 11:28 AM, Trond Myklebust wrote:
> > On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
> >> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
> >>> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
> >>>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> >>>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> >>>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
> >>>>>>
> >>>>>>> What does /var/lib/nfs/v4recovery look like on the server?
> >>>>>> The server was misconfigured, but I still think the client should
> >>>>>> behave better in this case. If you cannot reproduce it, let me know
> >>>>>> and I can try to be more specific. If you still want the v4recovery
> >>>>>> information, let me know and I'll send it.
> >>>>> So how should the client behave, when a screwed up server allows it to
> >>>>> mount but starts returning illegal values for setclientid? The only
> >>>>> thing I can see we could do is to tell the user EINSANESERVER...
> >>>> Well, it could just fail the mount and give up and not overly spam
> >>>> /var/log/messages in a tight loop perhaps?
> >>> This doesn't happen at mount time. It happens when you open a file.
> >> Not for me, and evidently not for the other person that reported
> >> similar results. All I had to do was attempt the mount (which never
> >> completed).
> >>
> >> Thanks,
> >> Ben
> >
> > Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> > bug occur when you turn this off too?
>
> Yes, it did.
OK. The following patch should fix that particular regression.
Note that there is a bug remaining inside nfs4_init_session(): we
shouldn't be copying the rsize/wsize into the nfs_client if the latter
was already initialised. I'm going to leave that for the moment, though,
since that bug wasn't introduced by my patch, and it doesn't affect
NFSv4.0.
Cheers,
Trond
-----------------------------------------------------------------
From: Trond Myklebust <[email protected]>
NFSv4: Fix an NFSv4 mount regression
Commit 008f55d0e019943323c20a03493a2ba5672a4cc8 (nfs41: recover lease in
_nfs4_lookup_root) forces the state manager to always run on mount. This is
a bug in the case of NFSv4.0, which doesn't require us to send a
setclientid until we want to grab file state.
In any case, this is completely the wrong place to be doing state
management. Moving that code into nfs4_init_session...
Signed-off-by: Trond Myklebust <[email protected]>
---
fs/nfs/client.c | 18 +++---------------
fs/nfs/nfs4_fs.h | 6 ++++++
fs/nfs/nfs4proc.c | 24 +++++++++++++++++-------
3 files changed, 26 insertions(+), 22 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index c2d0616..8d25ccb 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1242,20 +1242,6 @@ error:
return error;
}
-/*
- * Initialize a session.
- * Note: save the mount rsize and wsize for create_server negotiation.
- */
-static void nfs4_init_session(struct nfs_client *clp,
- unsigned int wsize, unsigned int rsize)
-{
-#if defined(CONFIG_NFS_V4_1)
- if (nfs4_has_session(clp)) {
- clp->cl_session->fc_attrs.max_rqst_sz = wsize;
- clp->cl_session->fc_attrs.max_resp_sz = rsize;
- }
-#endif /* CONFIG_NFS_V4_1 */
-}
/*
* Session has been established, and the client marked ready.
@@ -1350,7 +1336,9 @@ struct nfs_server *nfs4_create_server(const struct nfs_parsed_mount_data *data,
BUG_ON(!server->nfs_client->rpc_ops);
BUG_ON(!server->nfs_client->rpc_ops->file_inode_ops);
- nfs4_init_session(server->nfs_client, server->wsize, server->rsize);
+ error = nfs4_init_session(server);
+ if (error < 0)
+ goto error;
/* Probe the root fh to retrieve its FSID */
error = nfs4_path_walk(server, mntfh, data->nfs_server.export_path);
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 61bc3a3..6ea07a3 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -220,6 +220,7 @@ extern void nfs4_destroy_session(struct nfs4_session *session);
extern struct nfs4_session *nfs4_alloc_session(struct nfs_client *clp);
extern int nfs4_proc_create_session(struct nfs_client *, int reset);
extern int nfs4_proc_destroy_session(struct nfs4_session *);
+extern int nfs4_init_session(struct nfs_server *server);
#else /* CONFIG_NFS_v4_1 */
static inline int nfs4_setup_sequence(struct nfs_client *clp,
struct nfs4_sequence_args *args, struct nfs4_sequence_res *res,
@@ -227,6 +228,11 @@ static inline int nfs4_setup_sequence(struct nfs_client *clp,
{
return 0;
}
+
+static inline int nfs4_init_session(struct nfs_server *server)
+{
+ return 0;
+}
#endif /* CONFIG_NFS_V4_1 */
extern struct nfs4_state_maintenance_ops *nfs4_state_renewal_ops[];
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ff0c080..df24f67 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2040,15 +2040,9 @@ static int _nfs4_lookup_root(struct nfs_server *server, struct nfs_fh *fhandle,
.rpc_argp = &args,
.rpc_resp = &res,
};
- int status;
nfs_fattr_init(info->fattr);
- status = nfs4_recover_expired_lease(server);
- if (!status)
- status = nfs4_check_client_ready(server->nfs_client);
- if (!status)
- status = nfs4_call_sync(server, &msg, &args, &res, 0);
- return status;
+ return nfs4_call_sync(server, &msg, &args, &res, 0);
}
static int nfs4_lookup_root(struct nfs_server *server, struct nfs_fh *fhandle,
@@ -4793,6 +4787,22 @@ int nfs4_proc_destroy_session(struct nfs4_session *session)
return status;
}
+int nfs4_init_session(struct nfs_server *server)
+{
+ struct nfs_client *clp = server->nfs_client;
+ int ret;
+
+ if (!nfs4_has_session(clp))
+ return 0;
+
+ clp->cl_session->fc_attrs.max_rqst_sz = server->wsize;
+ clp->cl_session->fc_attrs.max_resp_sz = server->rsize;
+ ret = nfs4_recover_expired_lease(server);
+ if (!ret)
+ ret = nfs4_check_client_ready(clp);
+ return ret;
+}
+
/*
* Renew the cl_session lease.
*/
On Tue, 2009-07-21 at 17:17 -0400, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 11:32 -0700, Ben Greear wrote:
> > On 07/21/2009 11:28 AM, Trond Myklebust wrote:
> > > On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
> > >> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
> > >>> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
> > >>>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
> > >>>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
> > >>>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
> > >>>>>>
> > >>>>>>> What does /var/lib/nfs/v4recovery look like on the server?
> > >>>>>> The server was misconfigured, but I still think the client should
> > >>>>>> behave better in this case. If you cannot reproduce it, let me know
> > >>>>>> and I can try to be more specific. If you still want the v4recovery
> > >>>>>> information, let me know and I'll send it.
> > >>>>> So how should the client behave, when a screwed up server allows it to
> > >>>>> mount but starts returning illegal values for setclientid? The only
> > >>>>> thing I can see we could do is to tell the user EINSANESERVER...
> > >>>> Well, it could just fail the mount and give up and not overly spam
> > >>>> /var/log/messages in a tight loop perhaps?
> > >>> This doesn't happen at mount time. It happens when you open a file.
> > >> Not for me, and evidently not for the other person that reported
> > >> similar results. All I had to do was attempt the mount (which never
> > >> completed).
> > >>
> > >> Thanks,
> > >> Ben
> > >
> > > Ah... You have NFS_V4_1 enabled despite the Kconfig warning... Does the
> > > bug occur when you turn this off too?
> >
> > Yes, it did.
>
> OK. The following patch should fix that particular regression.
>
> Note that there is a bug remaining inside nfs4_init_session(): we
> shouldn't be copying the rsize/wsize into the nfs_client if the latter
> was already initialised. I'm going to leave that for the moment, though,
> since that bug wasn't introduced by my patch, and it doesn't affect
> NFSv4.0.
With that patch applied, I now get the expected behaviour when mounting
a server with no NFSv4 root from a 2.6.31-rc3 based client:
root@heimdal:~# mount -t nfs4 munkholm.local:/ /mnt
mount.nfs4: mounting munkholm.local:/ failed, reason given by server:
No such file or directory
Trond
On 07/21/2009 02:48 PM, Trond Myklebust wrote:
>> Note that there is a bug remaining inside nfs4_init_session(): we
>> shouldn't be copying the rsize/wsize into the nfs_client if the latter
>> was already initialised. I'm going to leave that for the moment, though,
>> since that bug wasn't introduced by my patch, and it doesn't affect
>> NFSv4.0.
>
> With that patch applied, I now get the expected behaviour when mounting
> a server with no NFSv4 root from a 2.6.31-rc3 based client:
>
> root@heimdal:~# mount -t nfs4 munkholm.local:/ /mnt
> mount.nfs4: mounting munkholm.local:/ failed, reason given by server:
> No such file or directory
>
> Trond
I just successfully tested it on my systems as well, so I think this
does indeed fix the problem.
Thanks!
Ben
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Jul 21, 2009, at 5:17 PM, Trond Myklebust wrote:
> On Tue, 2009-07-21 at 11:32 -0700, Ben Greear wrote:
>> On 07/21/2009 11:28 AM, Trond Myklebust wrote:
>>> On Tue, 2009-07-21 at 11:01 -0700, Ben Greear wrote:
>>>> On 07/21/2009 10:59 AM, Trond Myklebust wrote:
>>>>> On Tue, 2009-07-21 at 10:36 -0700, Ben Greear wrote:
>>>>>> On 07/21/2009 10:12 AM, Trond Myklebust wrote:
>>>>>>> On Tue, 2009-07-21 at 09:49 -0700, Ben Greear wrote:
>>>>>>>> On 07/21/2009 05:15 AM, Trond Myklebust wrote:
>>>>>>>>
>>>>>>>>> What does /var/lib/nfs/v4recovery look like on the server?
>>>>>>>> The server was misconfigured, but I still think the client
>>>>>>>> should
>>>>>>>> behave better in this case. If you cannot reproduce it, let
>>>>>>>> me know
>>>>>>>> and I can try to be more specific. If you still want the
>>>>>>>> v4recovery
>>>>>>>> information, let me know and I'll send it.
>>>>>>> So how should the client behave, when a screwed up server
>>>>>>> allows it to
>>>>>>> mount but starts returning illegal values for setclientid? The
>>>>>>> only
>>>>>>> thing I can see we could do is to tell the user EINSANESERVER...
>>>>>> Well, it could just fail the mount and give up and not overly
>>>>>> spam
>>>>>> /var/log/messages in a tight loop perhaps?
>>>>> This doesn't happen at mount time. It happens when you open a
>>>>> file.
>>>> Not for me, and evidently not for the other person that reported
>>>> similar results. All I had to do was attempt the mount (which
>>>> never
>>>> completed).
>>>>
>>>> Thanks,
>>>> Ben
>>>
>>> Ah... You have NFS_V4_1 enabled despite the Kconfig warning...
>>> Does the
>>> bug occur when you turn this off too?
>>
>> Yes, it did.
>
> OK. The following patch should fix that particular regression.
>
> Note that there is a bug remaining inside nfs4_init_session(): we
> shouldn't be copying the rsize/wsize into the nfs_client if the latter
> was already initialised.
The rsize/wsize is copied into the session prior to the create_session
call (triggered by the state management code you moved), and is used
for session negotiation. At this point the nfs_client cl_cons_state is
set to NFS_CS_SESSION_INITING (see nfs4_alloc_session), so the
nfs_client is not initialized. The cl_cons_state is set to
NFS_CS_READY after a successful create_session call.
-->Andy
> I'm going to leave that for the moment, though,
> since that bug wasn't introduced by my patch, and it doesn't affect
> NFSv4.0.
>
> Cheers,
> Trond
> -----------------------------------------------------------------
> From: Trond Myklebust <[email protected]>
> NFSv4: Fix an NFSv4 mount regression
>
> Commit 008f55d0e019943323c20a03493a2ba5672a4cc8 (nfs41: recover
> lease in
> _nfs4_lookup_root) forces the state manager to always run on mount.
> This is
> a bug in the case of NFSv4.0, which doesn't require us to send a
> setclientid until we want to grab file state.
>
> In any case, this is completely the wrong place to be doing state
> management. Moving that code into nfs4_init_session...
>
> Signed-off-by: Trond Myklebust <[email protected]>
> ---
>
> fs/nfs/client.c | 18 +++---------------
> fs/nfs/nfs4_fs.h | 6 ++++++
> fs/nfs/nfs4proc.c | 24 +++++++++++++++++-------
> 3 files changed, 26 insertions(+), 22 deletions(-)
>
>
> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> index c2d0616..8d25ccb 100644
> --- a/fs/nfs/client.c
> +++ b/fs/nfs/client.c
> @@ -1242,20 +1242,6 @@ error:
> return error;
> }
>
> -/*
> - * Initialize a session.
> - * Note: save the mount rsize and wsize for create_server
> negotiation.
> - */
> -static void nfs4_init_session(struct nfs_client *clp,
> - unsigned int wsize, unsigned int rsize)
> -{
> -#if defined(CONFIG_NFS_V4_1)
> - if (nfs4_has_session(clp)) {
> - clp->cl_session->fc_attrs.max_rqst_sz = wsize;
> - clp->cl_session->fc_attrs.max_resp_sz = rsize;
> - }
> -#endif /* CONFIG_NFS_V4_1 */
> -}
>
> /*
> * Session has been established, and the client marked ready.
> @@ -1350,7 +1336,9 @@ struct nfs_server *nfs4_create_server(const
> struct nfs_parsed_mount_data *data,
> BUG_ON(!server->nfs_client->rpc_ops);
> BUG_ON(!server->nfs_client->rpc_ops->file_inode_ops);
>
> - nfs4_init_session(server->nfs_client, server->wsize, server->rsize);
> + error = nfs4_init_session(server);
> + if (error < 0)
> + goto error;
>
> /* Probe the root fh to retrieve its FSID */
> error = nfs4_path_walk(server, mntfh, data->nfs_server.export_path);
> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
> index 61bc3a3..6ea07a3 100644
> --- a/fs/nfs/nfs4_fs.h
> +++ b/fs/nfs/nfs4_fs.h
> @@ -220,6 +220,7 @@ extern void nfs4_destroy_session(struct
> nfs4_session *session);
> extern struct nfs4_session *nfs4_alloc_session(struct nfs_client
> *clp);
> extern int nfs4_proc_create_session(struct nfs_client *, int reset);
> extern int nfs4_proc_destroy_session(struct nfs4_session *);
> +extern int nfs4_init_session(struct nfs_server *server);
> #else /* CONFIG_NFS_v4_1 */
> static inline int nfs4_setup_sequence(struct nfs_client *clp,
> struct nfs4_sequence_args *args, struct nfs4_sequence_res *res,
> @@ -227,6 +228,11 @@ static inline int nfs4_setup_sequence(struct
> nfs_client *clp,
> {
> return 0;
> }
> +
> +static inline int nfs4_init_session(struct nfs_server *server)
> +{
> + return 0;
> +}
> #endif /* CONFIG_NFS_V4_1 */
>
> extern struct nfs4_state_maintenance_ops *nfs4_state_renewal_ops[];
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index ff0c080..df24f67 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -2040,15 +2040,9 @@ static int _nfs4_lookup_root(struct
> nfs_server *server, struct nfs_fh *fhandle,
> .rpc_argp = &args,
> .rpc_resp = &res,
> };
> - int status;
>
> nfs_fattr_init(info->fattr);
> - status = nfs4_recover_expired_lease(server);
> - if (!status)
> - status = nfs4_check_client_ready(server->nfs_client);
> - if (!status)
> - status = nfs4_call_sync(server, &msg, &args, &res, 0);
> - return status;
> + return nfs4_call_sync(server, &msg, &args, &res, 0);
> }
>
> static int nfs4_lookup_root(struct nfs_server *server, struct nfs_fh
> *fhandle,
> @@ -4793,6 +4787,22 @@ int nfs4_proc_destroy_session(struct
> nfs4_session *session)
> return status;
> }
>
> +int nfs4_init_session(struct nfs_server *server)
> +{
> + struct nfs_client *clp = server->nfs_client;
> + int ret;
> +
> + if (!nfs4_has_session(clp))
> + return 0;
> +
> + clp->cl_session->fc_attrs.max_rqst_sz = server->wsize;
> + clp->cl_session->fc_attrs.max_resp_sz = server->rsize;
> + ret = nfs4_recover_expired_lease(server);
> + if (!ret)
> + ret = nfs4_check_client_ready(clp);
> + return ret;
> +}
> +
> /*
> * Renew the cl_session lease.
> */
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 2009-07-22 at 15:49 -0400, Andy Adamson wrote:
> On Jul 21, 2009, at 5:17 PM, Trond Myklebust wrote:
> > Note that there is a bug remaining inside nfs4_init_session(): we
> > shouldn't be copying the rsize/wsize into the nfs_client if the latter
> > was already initialised.
>
> The rsize/wsize is copied into the session prior to the create_session
> call (triggered by the state management code you moved), and is used
> for session negotiation. At this point the nfs_client cl_cons_state is
> set to NFS_CS_SESSION_INITING (see nfs4_alloc_session), so the
> nfs_client is not initialized. The cl_cons_state is set to
> NFS_CS_READY after a successful create_session call.
The call to nfs4_init_session() is in nfs4_create_server(). It can be
called several times _after_ the nfs_client has been initialised when
you mount more than one partition from the same NFS server.
If that is the case, and if you use different rsize/wsize values on
those different mounts, then you will end up clobbering the values of
fc_attrs.max_rqst_sz, and fc_attrs.max_resp_sz, having set them to the
wsize/rsize that was set by the very last mount call.
AFAICS, what you _should_ be doing in nfs4_init_session, is something
like
if (clp->cl_session->fc_attrs.max_rqst_sz < server->wsize)
clp->cl_session->fc_attrs.max_rqst_sz = server->wsize;
Trond
On Jul 22, 2009, at 4:20 PM, Trond Myklebust wrote:
> On Wed, 2009-07-22 at 15:49 -0400, Andy Adamson wrote:
>> On Jul 21, 2009, at 5:17 PM, Trond Myklebust wrote:
>>> Note that there is a bug remaining inside nfs4_init_session(): we
>>> shouldn't be copying the rsize/wsize into the nfs_client if the
>>> latter
>>> was already initialised.
>>
>> The rsize/wsize is copied into the session prior to the
>> create_session
>> call (triggered by the state management code you moved), and is used
>> for session negotiation. At this point the nfs_client cl_cons_state
>> is
>> set to NFS_CS_SESSION_INITING (see nfs4_alloc_session), so the
>> nfs_client is not initialized. The cl_cons_state is set to
>> NFS_CS_READY after a successful create_session call.
>
> The call to nfs4_init_session() is in nfs4_create_server(). It can be
> called several times _after_ the nfs_client has been initialised when
> you mount more than one partition from the same NFS server.
>
> If that is the case, and if you use different rsize/wsize values on
> those different mounts, then you will end up clobbering the values of
> fc_attrs.max_rqst_sz, and fc_attrs.max_resp_sz, having set them to the
> wsize/rsize that was set by the very last mount call.
You are right, this is a bug.
>
>
> AFAICS, what you _should_ be doing in nfs4_init_session, is something
> like
>
> if (clp->cl_session->fc_attrs.max_rqst_sz < server->wsize)
> clp->cl_session->fc_attrs.max_rqst_sz = server->wsize;
Currently, we only support one session per nfs_client - one session
for all partition mounts to the same NFS server.
For NFSv4.1 the per partition struct nfs_server rsize/wsize is used to
construct rpc requests that are sent over the session so the rsize/
wsize must not be larger than the session max_resp_sz/max_rqst_sz.
nfs4_init_session should simply return if the nfs_client cl_cons_state
is not NFS_CS_SESSION_INITING.
I shouldn't be trying to set the session max_resp_sz/max_rqst_sz to
the rsize/wsize, but rather to the maximum rsize/wsize supported by
the client.
If the server accepts or increases the max_resp_sz/max_rqst_sz then
all is well.
If the server reduces the max_resp_sz/max_rqst_sz, the maximum rsize/
wsize available for NFSv4.1 partition mounts to the server needs to be
reduced accordingly. So the nfs_server rsize/wsize needs to be bound
by the session max_resp_sz/max_rqst_sz as well as by the maximum
supported size.
If you think this is correct, I'll send a patch.
-->Andy
>
>
> Trond
>
On Wed, 2009-07-22 at 17:32 -0400, Andy Adamson wrote:
> nfs4_init_session should simply return if the nfs_client cl_cons_state
> is not NFS_CS_SESSION_INITING.
> I shouldn't be trying to set the session max_resp_sz/max_rqst_sz to
> the rsize/wsize, but rather to the maximum rsize/wsize supported by
> the client.
> If the server accepts or increases the max_resp_sz/max_rqst_sz then
> all is well.
> If the server reduces the max_resp_sz/max_rqst_sz, the maximum rsize/
> wsize available for NFSv4.1 partition mounts to the server needs to be
> reduced accordingly. So the nfs_server rsize/wsize needs to be bound
> by the session max_resp_sz/max_rqst_sz as well as by the maximum
> supported size.
Well.... The rsize/wsize is one thing, but how about acls? We simply
don't know how big they may become.
An alternative would be to let the server choose the
max_rqst_sz/max_resp_sz by always requesting the maximum allowed value.
Trond
On Jul 22, 2009, at 5:47 PM, Trond Myklebust wrote:
> On Wed, 2009-07-22 at 17:32 -0400, Andy Adamson wrote:
>> nfs4_init_session should simply return if the nfs_client
>> cl_cons_state
>> is not NFS_CS_SESSION_INITING.
>> I shouldn't be trying to set the session max_resp_sz/max_rqst_sz to
>> the rsize/wsize, but rather to the maximum rsize/wsize supported by
>> the client.
>> If the server accepts or increases the max_resp_sz/max_rqst_sz then
>> all is well.
>> If the server reduces the max_resp_sz/max_rqst_sz, the maximum rsize/
>> wsize available for NFSv4.1 partition mounts to the server needs to
>> be
>> reduced accordingly. So the nfs_server rsize/wsize needs to be bound
>> by the session max_resp_sz/max_rqst_sz as well as by the maximum
>> supported size.
>
> Well.... The rsize/wsize is one thing, but how about acls? We simply
> don't know how big they may become.
Yeah - has anything been submitted for 4.2 to fix this? (a readdir
cookie - like thingie)
>
>
> An alternative would be to let the server choose the
> max_rqst_sz/max_resp_sz by always requesting the maximum allowed
> value.
OK - maximum allowed value instead of maximum client supported value...
>
> Trond
>
On Wed, 2009-07-22 at 18:03 -0400, Andy Adamson wrote:
> On Jul 22, 2009, at 5:47 PM, Trond Myklebust wrote:
> > Well.... The rsize/wsize is one thing, but how about acls? We simply
> > don't know how big they may become.
>
> Yeah - has anything been submitted for 4.2 to fix this? (a readdir
> cookie - like thingie)
Not really. You pretty much have to set ACLs atomically, or you may end
up with security issues.
The good news is, though, that we should be able to figure out an upper
limit by looking at the max_rqst_sz/max_resp_sz that the server wants to
set.
On Jul 22, 2009, at 6:17 PM, Trond Myklebust wrote:
> On Wed, 2009-07-22 at 18:03 -0400, Andy Adamson wrote:
>> On Jul 22, 2009, at 5:47 PM, Trond Myklebust wrote:
>>> Well.... The rsize/wsize is one thing, but how about acls? We simply
>>> don't know how big they may become.
>>
>> Yeah - has anything been submitted for 4.2 to fix this? (a readdir
>> cookie - like thingie)
>
> Not really. You pretty much have to set ACLs atomically, or you may
> end
> up with security issues.
Sure, but we could help get ACL.
>
>
> The good news is, though, that we should be able to figure out an
> upper
> limit by looking at the max_rqst_sz/max_resp_sz that the server
> wants to
> set.
Yep.
-->Andy
>
>
>