Hi all...
I'm beginning to move my boxen to use nfs4, and have some questions...
First of all, versions of my software: kernel 2.6.33.5, nfs-utils 1.2.2
(perhaps some of my doubts are solved in newer versions)
Questions:
- Is it mandatory to use a setup with a common root for exported filesystems ?
In my setup I first did not use it, no fsid=0 root, and everything
worked fine, just change the mount type in the clients. In fact, new
nfs-utils just try first version 4 and you dont have to do anything.
You loose the ability to use simple paths inside /export, for example,
and have to use full paths like in V3, but that is not so bad.
Am I loosing something else wrt functionality ?
- If you have a common root, is it mandatory to mount it in the client ?
- If I setup a common root (say /export), and aoutomount some folders:
/scratch/local
/scratch/node/00
/scratch/node/01
......
/export/scratch -> /scratch/local (bind mount).
/scratch/node/?? is automounted from node ??, in which it is its own
/scratch/local. When I acess scratch for local node, it backs to a local
mount and mounts the REAL /scratch, not /export/scratch.
Any ideas ?
- AFAIK, with NFS4 the only needed daemons are nfsd and idmapd. And the
only accesible port from the outside is 2049, for nfsd.
I have tried to strip down my nfs server (-N 2 -N 3 -U),
but rpcinfo still gives me:
annwn:~# rpcinfo -p localhost
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100024 1 udp 48461 status
100024 1 tcp 37515 status
100021 1 udp 38583 nlockmgr
100021 3 udp 38583 nlockmgr
100021 4 udp 38583 nlockmgr
100021 1 tcp 37873 nlockmgr
100021 3 tcp 37873 nlockmgr
100021 4 tcp 37873 nlockmgr
100003 4 tcp 2049 nfs
100005 1 udp 45341 mountd
100005 1 tcp 58639 mountd
disabling portampper and mountd is just a matter of initscripts
requirements, but how can I disable nlockmgr ? It isn't needed for
NFS4, isn't it ? Nor portmapper nor mountd...
TIA
--
J.A. Magallon <jamagallon()ono!com> \ Software is like sex:
\ It's better when it's free
> - If you have a common root, is it mandatory to mount it in the client ?
>
> - If I setup a common root (say /export), and aoutomount some folders:
>
> /scratch/local
> /scratch/node/00
> /scratch/node/01
> ......
> /export/scratch -> /scratch/local (bind mount).
>
> /scratch/node/?? is automounted from node ??, in which it is its own
> /scratch/local. When I acess scratch for local node, it backs to a local
> mount and mounts the REAL /scratch, not /export/scratch.
> Any ideas ?
>
I'm not sure i have understand your means, is it like this?
# cat /etc/exports
/nfsroot *(fsid=0,insecure,no_root_squash,rw)
# mkdir -p /nfsroot/nfsroot
# touch /nfsroot/tstfile
# service nfs restart
# mount -t nfs 127.0.0.1:/nfsroot /mnt
# ls /mnt <== Here, we access real /nfsroot/nfsroot, not /nfsroot,
so we can not find "tstfile", right?
Actually, when we use "-t nfs", nfs-utils try to mount NFSv4 first,
if you want use NFSv3, you should use "-t nfs -o vers=3" in the nfs-utils-1.2.2.
> - AFAIK, with NFS4 the only needed daemons are nfsd and idmapd. And the
> only accesible port from the outside is 2049, for nfsd.
> I have tried to strip down my nfs server (-N 2 -N 3 -U),
> but rpcinfo still gives me:
>
... snip ...
> disabling portampper and mountd is just a matter of initscripts
> requirements, but how can I disable nlockmgr ? It isn't needed for
> NFS4, isn't it ? Nor portmapper nor mountd...
>
As i see, there's no way to disabe nlockmgr, it's start by lockd,
and lockd will be up when we start nfsd. It's always work until the
last nfsd thread stop.
--
Best Regards
Bian Naimeng
On Wed, Jul 28, 2010 at 10:28:12AM +0800, Bian Naimeng wrote:
> As i see, there's no way to disabe nlockmgr, it's start by lockd,
> and lockd will be up when we start nfsd. It's always work until the
> last nfsd thread stop.
Some day we should probably modify the server to turn off lockd in the
v4-only case. For now we don't do that.
It should be fine to firewall off lockd if you only care about NFSv4.
--b.
On 07/27/10 06:46 PM, J.A. Magallón wrote:
> - AFAIK, with NFS4 the only needed daemons are nfsd and idmapd. And the
> only accesible port from the outside is 2049, for nfsd.
> I have tried to strip down my nfs server (-N 2 -N 3 -U),
> but rpcinfo still gives me:
>
> annwn:~# rpcinfo -p localhost
> program vers proto port service
> 100000 4 tcp 111 portmapper
> 100000 3 tcp 111 portmapper
> 100000 2 tcp 111 portmapper
> 100000 4 udp 111 portmapper
> 100000 3 udp 111 portmapper
> 100000 2 udp 111 portmapper
> 100024 1 udp 48461 status
> 100024 1 tcp 37515 status
> 100021 1 udp 38583 nlockmgr
> 100021 3 udp 38583 nlockmgr
> 100021 4 udp 38583 nlockmgr
> 100021 1 tcp 37873 nlockmgr
> 100021 3 tcp 37873 nlockmgr
> 100021 4 tcp 37873 nlockmgr
> 100003 4 tcp 2049 nfs
> 100005 1 udp 45341 mountd
> 100005 1 tcp 58639 mountd
>
> disabling portampper and mountd is just a matter of initscripts
> requirements, but how can I disable nlockmgr ? It isn't needed for
> NFS4, isn't it ? Nor portmapper nor mountd...
Strictly speaking, portmapper is not required for NFSv4 service.
However, the NFS infrastructure on Linux is still designed for NFSv2 and
v3. There remains some work needed to make portmapper optional for a
v4-only server. For now, continue to run it in order to handle kernel
upcalls.
rpc.mountd is, however, still required on Linux NFSv4 servers. Although
NFSv4 clients do not contact the server's mountd, the kernel's NFS
server performs upcalls to rpc.mountd to manage export information. You
can firewall off the mountd service on the server without affecting
NFSv4 clients. Recent versions of rpc.mountd accept command line
options that disable the mountd network service while still handling
kernel upcalls.
And, as long as lockd is running, you will need to keep rpc.statd
around. Again, you can firewall this service so that it is not exposed
on the network, but it must continue to be available to handle kernel
upcalls. This is something we hope to address eventually as part of the
lockd work Bruce mentioned.
On Wed, 28 Jul 2010 14:08:08 -0400, Chuck Lever <[email protected]> wrote:
> On 07/27/10 06:46 PM, J.A. Magall?n wrote:
> > - AFAIK, with NFS4 the only needed daemons are nfsd and idmapd. And the
> > only accesible port from the outside is 2049, for nfsd.
> > I have tried to strip down my nfs server (-N 2 -N 3 -U),
> > but rpcinfo still gives me:
> >
> > annwn:~# rpcinfo -p localhost
> > program vers proto port service
> > 100000 4 tcp 111 portmapper
> > 100000 3 tcp 111 portmapper
> > 100000 2 tcp 111 portmapper
> > 100000 4 udp 111 portmapper
> > 100000 3 udp 111 portmapper
> > 100000 2 udp 111 portmapper
> > 100024 1 udp 48461 status
> > 100024 1 tcp 37515 status
> > 100021 1 udp 38583 nlockmgr
> > 100021 3 udp 38583 nlockmgr
> > 100021 4 udp 38583 nlockmgr
> > 100021 1 tcp 37873 nlockmgr
> > 100021 3 tcp 37873 nlockmgr
> > 100021 4 tcp 37873 nlockmgr
> > 100003 4 tcp 2049 nfs
> > 100005 1 udp 45341 mountd
> > 100005 1 tcp 58639 mountd
> >
> > disabling portampper and mountd is just a matter of initscripts
> > requirements, but how can I disable nlockmgr ? It isn't needed for
> > NFS4, isn't it ? Nor portmapper nor mountd...
>
> Strictly speaking, portmapper is not required for NFSv4 service.
> However, the NFS infrastructure on Linux is still designed for NFSv2 and
> v3. There remains some work needed to make portmapper optional for a
> v4-only server. For now, continue to run it in order to handle kernel
> upcalls.
>
> rpc.mountd is, however, still required on Linux NFSv4 servers. Although
> NFSv4 clients do not contact the server's mountd, the kernel's NFS
> server performs upcalls to rpc.mountd to manage export information. You
> can firewall off the mountd service on the server without affecting
> NFSv4 clients. Recent versions of rpc.mountd accept command line
> options that disable the mountd network service while still handling
> kernel upcalls.
>
> And, as long as lockd is running, you will need to keep rpc.statd
> around. Again, you can firewall this service so that it is not exposed
> on the network, but it must continue to be available to handle kernel
> upcalls. This is something we hope to address eventually as part of the
> lockd work Bruce mentioned.
Err...
First of all, plz correct me if this is in some kind of wiki, web page or
the like, but I have not been able to find it. All this things should be
on a document somewhere, perhaps even in the kernel itself...
These are the things I have found out (thanks to your answers...).
I'm a long time unix admin, not new to NFS, but all this things were not
obvious to me, so perhaps they deserve to be in a document, something like
"NFS 3 to 4 migration for dummy admins":
- Everything just works if you mount shares as nfs4. Even, if you have a
bunch of linux boxen with recent kernel/nfs-utils, probably you are
already doing nfs4... modern mount tries nfs4 first.
- Using nfsroot with fsid=0 is not mandatory, nor bind-mounting everything
under some /export (like many documents say), that only forces you to
use the old way of specifying paths in the server (absolute, not
relative to /export).
- Even if you use nfsroot(fsid=0), and you mount it on the client
at /somepoint, you are not forced to mount everything else under
/somepoint (plz, correct me if I'm wrong).
- You can strip your server for NFS4, but not too much... Old daemons
are still needed locally. For example, you can get rid of NFS2 and
UDP for nfsd (-N 2 -U, -any os still uses NFS2??-), and old
mount protocols (-N 1 -N 2 for mountd)...
- ... but they can be firewalled, use is just local
- portmap/rpcbind is not needed, but still used because nfsd is not
yet proper clean for only-nfs4-behavior.
There are also some things I have not been able to discover, like
using the interesting things of NFS4:
- How do you activate delegations ? Is this an automatic thing, or
do I have to add any option somewhere ?
- How do you use cache ? Many docs talk about fsc option, but man
does not mention it (nfs-utils 1.2.2).
I have tried to use cachefiles module, and cachefilesd, but
when I try to run it, I get:
bran:~/c/cachefilesd-0.10# ./cachefilesd -dddns -f $PWD/cachefilesd.conf
About to bind cache
CacheFiles bind failed: errno 95 (Operation not supported)
In dmesg:
[cachef] ==> cachefiles_get_security_ID({system_u:system_r:cachefiles_kernel_t:s0})
CacheFiles: Security denies permission to nominate security context: error -95
I can not run cachefiles without SELinux or something like that ?
Thanks, perhaps this notes help someone in the future.
TIA
--
J.A. Magallon <jamagallon()ono!com> \ Software is like sex:
\ It's better when it's free
On Fri, Jul 30, 2010 at 02:18:37AM +0200, J.A. Magallón wrote:
> First of all, plz correct me if this is in some kind of wiki, web page or
> the like, but I have not been able to find it. All this things should be
> on a document somewhere, perhaps even in the kernel itself...
Agreed.
> These are the things I have found out (thanks to your answers...).
> I'm a long time unix admin, not new to NFS, but all this things were not
> obvious to me, so perhaps they deserve to be in a document, something like
> "NFS 3 to 4 migration for dummy admins":
>
> - Everything just works if you mount shares as nfs4. Even, if you have a
> bunch of linux boxen with recent kernel/nfs-utils, probably you are
> already doing nfs4... modern mount tries nfs4 first.
>
> - Using nfsroot with fsid=0 is not mandatory, nor bind-mounting everything
> under some /export (like many documents say), that only forces you to
We should update that documentation.
> use the old way of specifying paths in the server (absolute, not
> relative to /export).
>
> - Even if you use nfsroot(fsid=0), and you mount it on the client
> at /somepoint, you are not forced to mount everything else under
> /somepoint (plz, correct me if I'm wrong).
>
> - You can strip your server for NFS4, but not too much... Old daemons
> are still needed locally. For example, you can get rid of NFS2 and
> UDP for nfsd (-N 2 -U, -any os still uses NFS2??-), and old
> mount protocols (-N 1 -N 2 for mountd)...
>
> - ... but they can be firewalled, use is just local
>
> - portmap/rpcbind is not needed, but still used because nfsd is not
> yet proper clean for only-nfs4-behavior.
>
> There are also some things I have not been able to discover, like
> using the interesting things of NFS4:
>
> - How do you activate delegations ? Is this an automatic thing, or
> do I have to add any option somewhere ?
It's automatic. You can turn them off on the server side with
/proc/sys/fs/leases-enable. They'll be turned off automatically if the
server can't open a tcp connection back to the client (for example, if
the client is behind a NAT).
> - How do you use cache ? Many docs talk about fsc option, but man
> does not mention it (nfs-utils 1.2.2).
> I have tried to use cachefiles module, and cachefilesd, but
> when I try to run it, I get:
I haven't used that. I don't believe there's anything NFSv4-specific
about it.
--b.