Dear NFS experts, I have a few questions:
1) Concerning the NFSv4 clientaddr option, I'm curious about the
technical details of why the server needs a callback address, and what
to do if the client isn't directly routable? (eg: behind NAT) I am
thinking of the situation with *many* clients.
Also, what ports need to be open on the client? Does it need to respond
to "NEW" traffic, or only "ESTABLISHED" or ?
2) In /etc/exports, for an NFSv4 export, you often see docs suggesting:
sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you
instead do 'sec=krb5p' will this *force* clients to use full encryption
and authentication, and deny those who try to mount without sec=krb5p ?
In particular, if a client tries to mount with sec=krb5i, what should
happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I
wanted to know what was up.
Thank you in advance,
James
On Sun, 2013-06-23 at 11:35 -0400, James wrote:
> On Sat, 2013-06-22 at 18:42 +0000, Myklebust, Trond wrote:
> >
> > Kernel parameters cannot be per-namespace; containers don't boot a
> > separate kernel.
> >
> > Note that if you have compiled nfs as a module, you will want to do
> > something along the lines of:
> >
> > echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
> This is very useful to know.
> Is there any way to specify this as a mount option?
No. The NFSv4 callback channel port number is a global property. It does
not want to be a per-mount thing.
> Will NFSv4.1 silently ignore this and use the same RPC connection if I
> include it for both 4.0, and 4.1 machines?
Yes.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
> -----Original Message-----
> From: [email protected] [mailto:linux-nfs-
> [email protected]] On Behalf Of Jim Rees
> Sent: Sunday, June 23, 2013 3:43 PM
> To: Myklebust, Trond
> Cc: James; Chuck Lever; [email protected]
> Subject: Re: NFS clientaddr, kerberos
>
> Myklebust, Trond wrote:
>
> > > echo "options nfs callback_tcpport=<port>"
> >>/etc/modprobe.d/options-local.conf
> > This is very useful to know.
> > Is there any way to specify this as a mount option?
>
> No. The NFSv4 callback channel port number is a global property. It does
> not want to be a per-mount thing.
>
> Would it make sense as a sysctl? I can sort of imagine cases where this might
> be useful. Not that I'm volunteering to implement it.
Hi Jim,
With the sysfs interface to kernel parameters, there is little advantage in implementing a sysctl.
Since this is a writeable kernel parameter, it means that you can also change it via the pseudofile in /sys/module/nfs/parameters/callback_tcpport after the module has been loaded (but you still need to do so before mounting any NFSv4 volumes).
Cheers
Trond
On Sat, 2013-06-22 at 11:22 -0400, Chuck Lever wrote:
> On Jun 22, 2013, at 10:59 AM, James <[email protected]> wrote:
>
> > Dear NFS experts, I have a few questions:
> >
> > 1) Concerning the NFSv4 clientaddr option, I'm curious about the
> > technical details of why the server needs a callback address, and what
> > to do if the client isn't directly routable? (eg: behind NAT) I am
> > thinking of the situation with *many* clients.
>
> If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
>
> > Also, what ports need to be open on the client? Does it need to respond
> > to "NEW" traffic, or only "ESTABLISHED" or ?
>
> Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
>
> The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.
>
> You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
>
> nfs.callback_tcpport=
> [NFS] set the TCP port on which the NFSv4 callback
> channel should listen.
>
> Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
Kernel parameters cannot be per-namespace; containers don't boot a
separate kernel.
Note that if you have compiled nfs as a module, you will want to do
something along the lines of:
echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use
the same connection as the outgoing RPC calls, and so support callbacks
through NAT without requiring you to open for incoming connections.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
On Jun 22, 2013, at 10:59 AM, James <[email protected]> wrote:
> Dear NFS experts, I have a few questions:
>
> 1) Concerning the NFSv4 clientaddr option, I'm curious about the
> technical details of why the server needs a callback address, and what
> to do if the client isn't directly routable? (eg: behind NAT) I am
> thinking of the situation with *many* clients.
If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
> Also, what ports need to be open on the client? Does it need to respond
> to "NEW" traffic, or only "ESTABLISHED" or ?
Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.
You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback
channel should listen.
Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
> 2) In /etc/exports, for an NFSv4 export, you often see docs suggesting:
> sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you
> instead do 'sec=krb5p' will this *force* clients to use full encryption
> and authentication, and deny those who try to mount without sec=krb5p ?
> In particular, if a client tries to mount with sec=krb5i, what should
> happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I
> wanted to know what was up.
If you specify "sec=krb5p" for an export, clients can mount that export only using sec=krb5p. Thus a mount command that specifies sec=krb5i should fail with EACCES.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Myklebust, Trond wrote:
> > echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
> This is very useful to know.
> Is there any way to specify this as a mount option?
No. The NFSv4 callback channel port number is a global property. It does
not want to be a per-mount thing.
Would it make sense as a sysctl? I can sort of imagine cases where this
might be useful. Not that I'm volunteering to implement it.
On Sat, 2013-06-22 at 18:42 +0000, Myklebust, Trond wrote:
> On Sat, 2013-06-22 at 11:22 -0400, Chuck Lever wrote:
> > On Jun 22, 2013, at 10:59 AM, James <[email protected]> wrote:
> >
> > > Dear NFS experts, I have a few questions:
> > >
> > > 1) Concerning the NFSv4 clientaddr option, I'm curious about the
> > > technical details of why the server needs a callback address, and what
> > > to do if the client isn't directly routable? (eg: behind NAT) I am
> > > thinking of the situation with *many* clients.
> >
> > If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
> >
> > > Also, what ports need to be open on the client? Does it need to respond
> > > to "NEW" traffic, or only "ESTABLISHED" or ?
> >
> > Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
> >
> > The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.
> >
> > You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
> >
> > nfs.callback_tcpport=
> > [NFS] set the TCP port on which the NFSv4 callback
> > channel should listen.
> >
> > Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
Thank you Chuck, Trond for your prompt replies, it really helped me have
a productive weekend of hacking.
>
> Kernel parameters cannot be per-namespace; containers don't boot a
> separate kernel.
>
> Note that if you have compiled nfs as a module, you will want to do
> something along the lines of:
>
> echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
This is very useful to know.
Is there any way to specify this as a mount option?
Will NFSv4.1 silently ignore this and use the same RPC connection if I
include it for both 4.0, and 4.1 machines?
Cheers,
James
>
> Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use
> the same connection as the outgoing RPC calls, and so support callbacks
> through NAT without requiring you to open for incoming connections.
>