I've finished fiddling with statd, and am happy with mount.nfs now, so
it must be time for the first release candidate for nfs-utils-1.1.0.
You can find it at:
http://www.kernel.org/pub/linux/utils/nfs/
https://sourceforge.net/project/showfiles.php?group_id=14&package_id=174&release_id=497197
git://linux-nfs.org/nfs-utils/
Any testing and reports would be most appreciated.
The "NEWS" file currently contains the following.
Thnaks,
NeilBrown
Significant changes for nfs-utils 1.1.0 - March/April 2007
- rpc.lockd is gone. One 3 old kernel releases need it.
- /sbin/{u,}mount.nfs{,4} is now installed so 'mount' will
use these to mount nfs filesystems instead of internal code.
+ mount.nfs will check for 'statd' to be running when mounting
a filesystem which requires it. If it is not running it will
run "/usr/sbin/start-statd" to try to start it.
If statd is not running and cannot be started, mount.nfs will
refuse to mount the filesystem and will suggest the 'nolock'
option.
- Substantial changes to statd
+ The 'notify' process that must happen at boot has been split
into a separate program "sm-notify". It ensures that it
only runs once even if you restart statd. This is correct
behaviour.
+ statd stores state in the files in /var/lib/nfs/sm/ so that
if you kill and restart it, it will restore that state and
continue working correctly.
+ statd makes more use of DNS lookup and should handle
multi-homed peers better.
- If you export a directory as 'crossmnt', all filesystems
mounted beneath are automatically exported with the same
options (unless explicitly exported with different options).
- subtree_check is no-longer the default. The default is now
no_subtree_check.
- By default the system 'rpcgen' is used while building
nfs-utils rather than the internal one.
Further notes on statd:
statd should be installed in /usr/sbin, not /sbin.
If you need to mount /usr via nfs, use 'nolock'
At boot time, run "/usr/sbin/sm-notify".
Run "statd" only when starting the NFS server.
"statd" should be run before starting the NFS server.
You do not need to start statd at boot time incase an
NFS filesystem is mounted. mount.nfs will take care of that.
Make sure /usr/sbin/start-statd will run statd with required
arguments.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 29 Mar 2007, Neil Brown wrote:
> Run "statd" only when starting the NFS server.
> "statd" should be run before starting the NFS server.
While you're at it, can you tell me what should be the order of starting
all NFS/NFS4 daemons? So far I got to this:
1) statd
2) mountd
3) nfsd
4) idmapd
5) svcgssd
6) gssd
7) rquotad
8) any NFS mounts
Is it correct?
Jan
--
Jan Rekorajski | ALL SUSPECTS ARE GUILTY. PERIOD!
baggins<at>mimuw.edu.pl | OTHERWISE THEY WOULDN'T BE SUSPECTS, WOULD THEY?
BOFH, MANIAC | -- TROOPS by Kevin Rubio
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thursday March 29, [email protected] wrote:
> On Thu, 29 Mar 2007, Neil Brown wrote:
>
> > Run "statd" only when starting the NFS server.
> > "statd" should be run before starting the NFS server.
>
> While you're at it, can you tell me what should be the order of starting
> all NFS/NFS4 daemons? So far I got to this:
>
> 1) statd
> 2) mountd
> 3) nfsd
> 4) idmapd
> 5) svcgssd
> 6) gssd
> 7) rquotad
> 8) any NFS mounts
>
> Is it correct?
Hmm... good question.
I think:
- idmapd and gssd should be started before any nfs mount
- idmapd and svcgssd should be started before nfsd
- mountd should start before nfsd.
- sm-notify should start *after* nfsd.
- on reflection, statd can probably start slightly after nfsd,
and if statd is doing the notify, it should definitely be
after, so my above comment is wrong. I'll fix it in the
next RC.
Similarly it should start before, or atmost slightly after, any
NFS mount.
- rquotad should probably start before nfsd, but it probably isn't
very important.
Rationale:
sm-notify:
On an NFS server, this tells the client to reclaim locks. They
will try to do so straight away. If nfsd hasn't started lockd yet,
they will fail and give up. This is the most important
dependency.
On an NFS client, it simply tells the server to forget about old
locks, so when it starts isn't terribly important.
statd:
This provides service to nfsd/lockd, but also it responds to
SM_NOTIFY from peers and then needs to talk to the kernel.
So kernel needs to talk to statd, and statd need to talk to kernel.
Both sides will retry on failure, so order isn't too important,
but there should not be a long gap between startup time.
mountd:
provides "is this exported" service to nfsd. If it isn't running
when the first nfs request arrives, it might be rejected
incorrectly. This is probably the second most important
dependency.
rquotad:
provides service to clients completely independently of nfsd
so when it starts probably doesn't matter.
idmapd/svcgssd/gssd:
simply provide service to the kernel modules. So they should
be running before the kernel wants them.
I think that covers everything.
NeilBrown
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thursday 29 March 2007 06:12:45 Neil Brown wrote:
> I've finished fiddling with statd, and am happy with mount.nfs now, so
> it must be time for the first release candidate for nfs-utils-1.1.0.
>
While you are working on statd, maybe you can look into a problem I recently
had?
After the update from Debian Sarge to Etch, the update scripts dumped the
current portmapper status to read them in after the portmap restart. Well,
this was an update of a chroot and the portmapper never got restartet on the
server and so also the dump file wasn't deleted.
The diskless clients using this chroot read in the dump file, but also
couldn't delete it, since it was located in a read-only nfs-export (*).
The problem was statd, the dumpfile said that statd is running on a port <
1024, but since rpc.statd as user statd is not allowed to use ports <1024, it
refused to start up at all.
Well, while this clearly can and should be solved by removing the stale
portmapper dump file, maybe there's also a solution for rpc.statd?
Thanks,
Bernd
PS: (*) Somewhere in /var, which will become writable after a union mount,
which will be done after the start-up of portmap.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 29 Mar 2007, Neil Brown wrote:
> On Thursday March 29, [email protected] wrote:
> > On Thu, 29 Mar 2007, Neil Brown wrote:
> >
> > > Run "statd" only when starting the NFS server.
> > > "statd" should be run before starting the NFS server.
> >
> > While you're at it, can you tell me what should be the order of starting
> > all NFS/NFS4 daemons? So far I got to this:
> >
> > 1) statd
> > 2) mountd
> > 3) nfsd
> > 4) idmapd
> > 5) svcgssd
> > 6) gssd
> > 7) rquotad
> > 8) any NFS mounts
> >
> > Is it correct?
>
> Hmm... good question.
>
> I think:
>
> - idmapd and gssd should be started before any nfs mount
That's quite obvious.
> - idmapd and svcgssd should be started before nfsd
Here I must disagree, my experience shows that idmapd must be started (or
at least restarted) after mountd (or exportfs) is run.
I had no id mapping when I started idmapd before mountd/nfsd.
[...]
> idmapd/svcgssd/gssd:
> simply provide service to the kernel modules. So they should
> be running before the kernel wants them.
What's troubling me here is svcgssd - does it behave like idmapd and
requires mountd? My quick investigation in the source telss me it is so.
Jan
--
Jan Rekorajski | ALL SUSPECTS ARE GUILTY. PERIOD!
baggins<at>mimuw.edu.pl | OTHERWISE THEY WOULDN'T BE SUSPECTS, WOULD THEY?
BOFH, MANIAC | -- TROOPS by Kevin Rubio
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 2007-03-29 at 14:44 +0200, Jan Rekorajski wrote:
> > - idmapd and svcgssd should be started before nfsd
>
> Here I must disagree, my experience shows that idmapd must be started (or
> at least restarted) after mountd (or exportfs) is run.
> I had no id mapping when I started idmapd before mountd/nfsd.
The only reason I can think of why you might have to restart idmapd
after mountd is if your idmapd init script is failing to
mount /proc/fs/nfsd.
> > idmapd/svcgssd/gssd:
> > simply provide service to the kernel modules. So they should
> > be running before the kernel wants them.
>
> What's troubling me here is svcgssd - does it behave like idmapd and
> requires mountd? My quick investigation in the source telss me it is so.
Ditto. svcgssd also assumes that you have mounted /proc/fs/nfsd. Fix
your svcgssd init script if this is not already the case.
Trond
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 2007-03-29 at 19:31 +1000, Neil Brown wrote:
> - idmapd and gssd should be started before any nfs mount
> - idmapd and svcgssd should be started before nfsd
> - mountd should start before nfsd.
> - sm-notify should start *after* nfsd.
...and _after_ statd. If the server is not ready to accept lock reclaim
requests, then it is wrong to be sending out sm-notify requests.
Trond
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
At 05:31 AM 3/29/2007, Neil Brown wrote:
>On Thursday March 29, [email protected] wrote:
>> On Thu, 29 Mar 2007, Neil Brown wrote:
>>
>> > Run "statd" only when starting the NFS server.
>> > "statd" should be run before starting the NFS server.
>>
>> While you're at it, can you tell me what should be the order of starting
>> all NFS/NFS4 daemons? So far I got to this:
>>
Don't forget...
0) portmapper
*This* is the most important one! ;-) Interesting things happen when
portmapper isn't present btw, some clients fail to retry if they get errors
on portmap lookups. They often fail to retry if the portmapper replies
with a zero port (service not found), which is the more common startup
ordering issue.
Really interesting things happen after the statmon notifications, if clients
start to reclaim and fail to find the portmapper or the lock manager. Solaris,
for example, starts killing client processes with SIGLOST. So I agree that
it is critical to ensure that portmap, statd and nfsd (which implies lockd
on Linux of course) are started before the server's notifications.
Tom.
>> 1) statd
>> 2) mountd
>> 3) nfsd
>> 4) idmapd
>> 5) svcgssd
>> 6) gssd
>> 7) rquotad
>> 8) any NFS mounts
>>
>> Is it correct?
>
>Hmm... good question.
>
>I think:
>
> - idmapd and gssd should be started before any nfs mount
> - idmapd and svcgssd should be started before nfsd
> - mountd should start before nfsd.
> - sm-notify should start *after* nfsd.
> - on reflection, statd can probably start slightly after nfsd,
> and if statd is doing the notify, it should definitely be
> after, so my above comment is wrong. I'll fix it in the
> next RC.
> Similarly it should start before, or atmost slightly after, any
> NFS mount.
> - rquotad should probably start before nfsd, but it probably isn't
> very important.
>
>Rationale:
>
> sm-notify:
> On an NFS server, this tells the client to reclaim locks. They
> will try to do so straight away. If nfsd hasn't started lockd yet,
> they will fail and give up. This is the most important
> dependency.
> On an NFS client, it simply tells the server to forget about old
> locks, so when it starts isn't terribly important.
> statd:
> This provides service to nfsd/lockd, but also it responds to
> SM_NOTIFY from peers and then needs to talk to the kernel.
> So kernel needs to talk to statd, and statd need to talk to kernel.
> Both sides will retry on failure, so order isn't too important,
> but there should not be a long gap between startup time.
> mountd:
> provides "is this exported" service to nfsd. If it isn't running
> when the first nfs request arrives, it might be rejected
> incorrectly. This is probably the second most important
> dependency.
> rquotad:
> provides service to clients completely independently of nfsd
> so when it starts probably doesn't matter.
> idmapd/svcgssd/gssd:
> simply provide service to the kernel modules. So they should
> be running before the kernel wants them.
>
>I think that covers everything.
>
>NeilBrown
>
>-------------------------------------------------------------------------
>Take Surveys. Earn Cash. Influence the Future of IT
>Join SourceForge.net's Techsay panel and you'll get the chance to share your
>opinions on IT & business topics through brief surveys-and earn cash
>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>_______________________________________________
>NFS maillist - [email protected]
>https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Mar 29, 2007 at 10:20:43AM -0400, Trond Myklebust wrote:
> On Thu, 2007-03-29 at 14:44 +0200, Jan Rekorajski wrote:
>
> > > - idmapd and svcgssd should be started before nfsd
> >
> > Here I must disagree, my experience shows that idmapd must be started (or
> > at least restarted) after mountd (or exportfs) is run.
> > I had no id mapping when I started idmapd before mountd/nfsd.
>
> The only reason I can think of why you might have to restart idmapd
> after mountd is if your idmapd init script is failing to
> mount /proc/fs/nfsd.
>
> > > idmapd/svcgssd/gssd:
> > > simply provide service to the kernel modules. So they should
> > > be running before the kernel wants them.
> >
> > What's troubling me here is svcgssd - does it behave like idmapd and
> > requires mountd? My quick investigation in the source telss me it is so.
>
> Ditto. svcgssd also assumes that you have mounted /proc/fs/nfsd. Fix
> your svcgssd init script if this is not already the case.
That's been a common source of errors in the past. Maybe svcgssd and
idmapd should be taught to watch for changes to the /proc/ directories
it cares about, maybe using dnotify or inotify.
--b.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
At 12:35 PM 3/29/2007, J. Bruce Fields wrote:
>> Ditto. svcgssd also assumes that you have mounted /proc/fs/nfsd. Fix
>> your svcgssd init script if this is not already the case.
>
>That's been a common source of errors in the past. Maybe svcgssd and
>idmapd should be taught to watch for changes to the /proc/ directories
>it cares about, maybe using dnotify or inotify.
Does inotify work on /proc? Cool!
Anyway, I totally agree on the /proc/fs/nfsd pipefs. I remember being
quite surprised when I first discovered that it usually is mounted by a
single-line shell script deeply buried in /etc/modprobe.d/modprobe.conf.dist.
Tom.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thursday March 29, [email protected] wrote:
> On Thursday 29 March 2007 06:12:45 Neil Brown wrote:
> > I've finished fiddling with statd, and am happy with mount.nfs now, so
> > it must be time for the first release candidate for nfs-utils-1.1.0.
> >
>
> While you are working on statd, maybe you can look into a problem I recently
> had?
> After the update from Debian Sarge to Etch, the update scripts dumped the
> current portmapper status to read them in after the portmap restart. Well,
> this was an update of a chroot and the portmapper never got restartet on the
> server and so also the dump file wasn't deleted.
> The diskless clients using this chroot read in the dump file, but also
> couldn't delete it, since it was located in a read-only nfs-export (*).
>
> The problem was statd, the dumpfile said that statd is running on a port <
> 1024, but since rpc.statd as user statd is not allowed to use ports <1024, it
> refused to start up at all.
>
> Well, while this clearly can and should be solved by removing the stale
> portmapper dump file, maybe there's also a solution for rpc.statd?
rpc.statd used to drop root privilege before binding the port to
listen on. This is fixed in 1.1.0, so that little part of your
problem should go away.
NeilBrown
>
>
> Thanks,
> Bernd
>
> PS: (*) Somewhere in /var, which will become writable after a union mount,
> which will be done after the start-up of portmap.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Friday 30 March 2007 04:19:49 Neil Brown wrote:
> rpc.statd used to drop root privilege before binding the port to
> listen on. This is fixed in 1.1.0, so that little part of your
> problem should go away.
Great, thanks a lot!
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Mar 29, 2007 at 10:44:33AM +0200, Jan Rekorajski wrote:
> On Thu, 29 Mar 2007, Neil Brown wrote:
>
> > Run "statd" only when starting the NFS server.
> > "statd" should be run before starting the NFS server.
>
> While you're at it, can you tell me what should be the order of starting
> all NFS/NFS4 daemons? So far I got to this:
>
> 1) statd
> 2) mountd
> 3) nfsd
> 4) idmapd
> 5) svcgssd
> 6) gssd
> 7) rquotad
> 8) any NFS mounts
>
It would be a good idea to put the final answer in a FAQ.
--
Frank
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Mar 29, 2007 at 07:31:25PM +1000, Neil Brown wrote:
> On Thursday March 29, [email protected] wrote:
> > On Thu, 29 Mar 2007, Neil Brown wrote:
> >
> > > Run "statd" only when starting the NFS server.
> > > "statd" should be run before starting the NFS server.
> >
> > While you're at it, can you tell me what should be the order of starting
> > all NFS/NFS4 daemons? So far I got to this:
> >
> > 1) statd
> > 2) mountd
> > 3) nfsd
> > 4) idmapd
> > 5) svcgssd
> > 6) gssd
> > 7) rquotad
> > 8) any NFS mounts
> >
> > Is it correct?
>
> Hmm... good question.
>
> I think:
>
> - idmapd and gssd should be started before any nfs mount
> - idmapd and svcgssd should be started before nfsd
> - mountd should start before nfsd.
At first that seems the logically correct order but suppose a client
succeeds in mounting something but issues NFS requests before nfsd got
started on the server?
where's the exportfs?
Consider a "sleep 30" at every point in the startup sequence and think
about what would go wrong. Also consider a dead-again server at every
point in the startup sequence.
--
Frank
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Sunday April 1, [email protected] wrote:
> On Thu, Mar 29, 2007 at 07:31:25PM +1000, Neil Brown wrote:
> >
> > Hmm... good question.
> >
> > I think:
> >
> > - idmapd and gssd should be started before any nfs mount
> > - idmapd and svcgssd should be started before nfsd
> > - mountd should start before nfsd.
>
> At first that seems the logically correct order but suppose a client
> succeeds in mounting something but issues NFS requests before nfsd got
> started on the server?
That shouldn't matter. The NFS server will not respond so the client
will retry.
Ofcourse if you have a soft mount it could get messy, but then soft
mounts are always messy.
>
> where's the exportfs?
Good point. exportfs should be run before mountd is started, and
after nfsd filesystem is mounted.
NeilBrown
>
> Consider a "sleep 30" at every point in the startup sequence and think
> about what would go wrong. Also consider a dead-again server at every
> point in the startup sequence.
>
> --
> Frank
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs