My University information security office does not like rpcbind and will
automatically quarantine any system for which they detect a portmapper
running on an exposed port.
Since I exclusively use NFSv4 I was happy to "learn" that NFSv4 doesn't
require rpcbind any more. For example, here's what it says in the
current RHEL documentation:
"NFS version 4 (NFSv4) works through firewalls and on the Internet, no
longer requires an rpcbind service, supports Access Control Lists
(ACLs), and utilizes stateful operations."
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
absolutely will not start if it can't launch rpcbind as a precursor:
-----------------------------
root@helios:~# systemctl stop rpcbind
Warning: Stopping rpcbind.service, but it can still be activated by:
rpcbind.socket
root@helios:~# systemctl mask rpcbind
Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
root@helios:~# systemctl restart nfs-server
Job for nfs-server.service canceled.
root@helios:~# systemctl status nfs-server
● nfs-server.service - NFS server and services
Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled;
vendor preset: enabled)
Drop-In: /run/systemd/generator/nfs-server.service.d
└─order-with-mounts.conf
Active: failed (Result: exit-code) since Fri 2020-09-25 14:21:46
UTC; 10s ago
Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
status=0/SUCCESS)
Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
(code=exited, status=1/FAILURE)
Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
status=0/SUCCESS)
Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
status=0/SUCCESS)
Main PID: 3925 (code=exited, status=1/FAILURE)
Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to kernel
failed: errno 111 (Connection refused)
Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
sockets for nfsd
Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
exited, code=exited, status=1/FAILURE
Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
result 'exit-code'.
Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
-----------------------------
So, now I'm confused. Does NFSv4 need rpcbind to be running, does it
just need it when it launches, or something else? I made a local copy
of the systemd service file and edited out the rpcbind dependency, so
it's not that.
On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
> My University information security office does not like rpcbind and
> will automatically quarantine any system for which they detect a
> portmapper running on an exposed port.
>
> Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
> doesn't require rpcbind any more. For example, here's what it says
> in the current RHEL documentation:
>
> "NFS version 4 (NFSv4) works through firewalls and on the Internet,
> no longer requires an rpcbind service, supports Access Control Lists
> (ACLs), and utilizes stateful operations."
>
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
>
> I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
> absolutely will not start if it can't launch rpcbind as a precursor:
>
> -----------------------------
> root@helios:~# systemctl stop rpcbind
> Warning: Stopping rpcbind.service, but it can still be activated by:
> rpcbind.socket
> root@helios:~# systemctl mask rpcbind
> Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
>
> root@helios:~# systemctl restart nfs-server
> Job for nfs-server.service canceled.
> root@helios:~# systemctl status nfs-server
> ● nfs-server.service - NFS server and services
> Loaded: loaded (/lib/systemd/system/nfs-server.service;
> enabled; vendor preset: enabled)
> Drop-In: /run/systemd/generator/nfs-server.service.d
> └─order-with-mounts.conf
> Active: failed (Result: exit-code) since Fri 2020-09-25
> 14:21:46 UTC; 10s ago
> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
> status=0/SUCCESS)
> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
> (code=exited, status=1/FAILURE)
> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
> status=0/SUCCESS)
> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
> status=0/SUCCESS)
> Main PID: 3925 (code=exited, status=1/FAILURE)
>
> Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
> kernel failed: errno 111 (Connection refused)
> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
> sockets for nfsd
> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
> exited, code=exited, status=1/FAILURE
> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
> result 'exit-code'.
> Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
> -----------------------------
>
> So, now I'm confused. Does NFSv4 need rpcbind to be running, does
> it just need it when it launches, or something else? I made a local
> copy of the systemd service file and edited out the rpcbind
> dependency, so it's not that.
Do you have v2 and v3 turned off in /etc/nfs.conf?
If yes, and nfsd is still refusing to start, that sounds like an nfsd
bug; with v4 only it should definitely be ignoring any failures to
contact rpcbind.
--b.
Hi Bruce,
Thanks for the reply. See below.
On 10/1/20 1:30 PM, J. Bruce Fields wrote:
> On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
>> My University information security office does not like rpcbind and
>> will automatically quarantine any system for which they detect a
>> portmapper running on an exposed port.
>>
>> Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
>> doesn't require rpcbind any more. For example, here's what it says
>> in the current RHEL documentation:
>>
>> "NFS version 4 (NFSv4) works through firewalls and on the Internet,
>> no longer requires an rpcbind service, supports Access Control Lists
>> (ACLs), and utilizes stateful operations."
>>
>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
>>
>> I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
>> absolutely will not start if it can't launch rpcbind as a precursor:
>>
>> -----------------------------
>> root@helios:~# systemctl stop rpcbind
>> Warning: Stopping rpcbind.service, but it can still be activated by:
>> rpcbind.socket
>> root@helios:~# systemctl mask rpcbind
>> Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
>>
>> root@helios:~# systemctl restart nfs-server
>> Job for nfs-server.service canceled.
>> root@helios:~# systemctl status nfs-server
>> ● nfs-server.service - NFS server and services
>> Loaded: loaded (/lib/systemd/system/nfs-server.service;
>> enabled; vendor preset: enabled)
>> Drop-In: /run/systemd/generator/nfs-server.service.d
>> └─order-with-mounts.conf
>> Active: failed (Result: exit-code) since Fri 2020-09-25
>> 14:21:46 UTC; 10s ago
>> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
>> status=0/SUCCESS)
>> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
>> (code=exited, status=1/FAILURE)
>> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
>> status=0/SUCCESS)
>> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
>> status=0/SUCCESS)
>> Main PID: 3925 (code=exited, status=1/FAILURE)
>>
>> Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
>> kernel failed: errno 111 (Connection refused)
>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
>> sockets for nfsd
>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
>> exited, code=exited, status=1/FAILURE
>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
>> result 'exit-code'.
>> Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
>> -----------------------------
>>
>> So, now I'm confused. Does NFSv4 need rpcbind to be running, does
>> it just need it when it launches, or something else? I made a local
>> copy of the systemd service file and edited out the rpcbind
>> dependency, so it's not that.
>
> Do you have v2 and v3 turned off in /etc/nfs.conf?
It's an Ubuntu system, hence doesn't use /etc/nfs.conf; however I do
have these variables set in /etc/default/nfs-kernel-server :
MOUNTD_NFS_V2="no"
MOUNTD_NFS_V3="no"
RPCMOUNTDOPTS="--manage-gids -N 2 -N 3"
maybe this isn't the correct way to disable NFSv2/3, but it's all I
could find documented.
The linux kernel version is 5.4.0, and the nfs-kernel-server package
version is 1:1.3.4-2.5ubuntu3.3 (so upstream 1.3.4), but I'm not sure
this is relevant.
>
> If yes, and nfsd is still refusing to start, that sounds like an nfsd
> bug; with v4 only it should definitely be ignoring any failures to
> contact rpcbind.
>
> --b.
>
On Thu, Oct 01, 2020 at 01:41:39PM -0500, Patrick Goetz wrote:
> Hi Bruce,
>
> Thanks for the reply. See below.
>
> On 10/1/20 1:30 PM, J. Bruce Fields wrote:
> >On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
> >>My University information security office does not like rpcbind and
> >>will automatically quarantine any system for which they detect a
> >>portmapper running on an exposed port.
> >>
> >>Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
> >>doesn't require rpcbind any more. For example, here's what it says
> >>in the current RHEL documentation:
> >>
> >>"NFS version 4 (NFSv4) works through firewalls and on the Internet,
> >>no longer requires an rpcbind service, supports Access Control Lists
> >>(ACLs), and utilizes stateful operations."
> >>
> >>https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
> >>
> >>I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
> >>absolutely will not start if it can't launch rpcbind as a precursor:
> >>
> >>-----------------------------
> >>root@helios:~# systemctl stop rpcbind
> >>Warning: Stopping rpcbind.service, but it can still be activated by:
> >> rpcbind.socket
> >>root@helios:~# systemctl mask rpcbind
> >>Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
> >>
> >>root@helios:~# systemctl restart nfs-server
> >>Job for nfs-server.service canceled.
> >>root@helios:~# systemctl status nfs-server
> >>● nfs-server.service - NFS server and services
> >> Loaded: loaded (/lib/systemd/system/nfs-server.service;
> >>enabled; vendor preset: enabled)
> >> Drop-In: /run/systemd/generator/nfs-server.service.d
> >> └─order-with-mounts.conf
> >> Active: failed (Result: exit-code) since Fri 2020-09-25
> >>14:21:46 UTC; 10s ago
> >> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
> >>status=0/SUCCESS)
> >> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
> >>(code=exited, status=1/FAILURE)
> >> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
> >>status=0/SUCCESS)
> >> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
> >>status=0/SUCCESS)
> >> Main PID: 3925 (code=exited, status=1/FAILURE)
> >>
> >>Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
> >>Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
> >>kernel failed: errno 111 (Connection refused)
> >>Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
> >>sockets for nfsd
> >>Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
> >>exited, code=exited, status=1/FAILURE
> >>Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
> >>result 'exit-code'.
> >>Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
> >>-----------------------------
> >>
> >>So, now I'm confused. Does NFSv4 need rpcbind to be running, does
> >>it just need it when it launches, or something else? I made a local
> >>copy of the systemd service file and edited out the rpcbind
> >>dependency, so it's not that.
> >
> >Do you have v2 and v3 turned off in /etc/nfs.conf?
>
> It's an Ubuntu system, hence doesn't use /etc/nfs.conf; however I do
> have these variables set in /etc/default/nfs-kernel-server :
>
> MOUNTD_NFS_V2="no"
> MOUNTD_NFS_V3="no"
> RPCMOUNTDOPTS="--manage-gids -N 2 -N 3"
>
> maybe this isn't the correct way to disable NFSv2/3, but it's all I
> could find documented.
That should do it, but if you want to verify that it worked, you can
read /proc/fs/nfsd/versions.
> The linux kernel version is 5.4.0, and the nfs-kernel-server package
> version is 1:1.3.4-2.5ubuntu3.3 (so upstream 1.3.4), but I'm not
> sure this is relevant.
I can't reproduce the problem on my 5.9-ish server, but I also can't
recall any relevant changes here.
Looking back through the history.... Kinglong Mee fixed the server to
ignore rpbind failures in the v4-only case about 7 years ago, back in
4.13.
--b.
On 10/1/20 3:06 PM, J. Bruce Fields wrote:
> On Thu, Oct 01, 2020 at 01:41:39PM -0500, Patrick Goetz wrote:
>> Hi Bruce,
>>
>> Thanks for the reply. See below.
>>
>> On 10/1/20 1:30 PM, J. Bruce Fields wrote:
>>> On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
>>>> My University information security office does not like rpcbind and
>>>> will automatically quarantine any system for which they detect a
>>>> portmapper running on an exposed port.
>>>>
>>>> Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
>>>> doesn't require rpcbind any more. For example, here's what it says
>>>> in the current RHEL documentation:
>>>>
>>>> "NFS version 4 (NFSv4) works through firewalls and on the Internet,
>>>> no longer requires an rpcbind service, supports Access Control Lists
>>>> (ACLs), and utilizes stateful operations."
>>>>
>>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
>>>>
>>>> I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
>>>> absolutely will not start if it can't launch rpcbind as a precursor:
>>>>
>>>> -----------------------------
>>>> root@helios:~# systemctl stop rpcbind
>>>> Warning: Stopping rpcbind.service, but it can still be activated by:
>>>> rpcbind.socket
>>>> root@helios:~# systemctl mask rpcbind
>>>> Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
>>>>
>>>> root@helios:~# systemctl restart nfs-server
>>>> Job for nfs-server.service canceled.
>>>> root@helios:~# systemctl status nfs-server
>>>> ● nfs-server.service - NFS server and services
>>>> Loaded: loaded (/lib/systemd/system/nfs-server.service;
>>>> enabled; vendor preset: enabled)
>>>> Drop-In: /run/systemd/generator/nfs-server.service.d
>>>> └─order-with-mounts.conf
>>>> Active: failed (Result: exit-code) since Fri 2020-09-25
>>>> 14:21:46 UTC; 10s ago
>>>> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
>>>> status=0/SUCCESS)
>>>> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
>>>> (code=exited, status=1/FAILURE)
>>>> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
>>>> status=0/SUCCESS)
>>>> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
>>>> status=0/SUCCESS)
>>>> Main PID: 3925 (code=exited, status=1/FAILURE)
>>>>
>>>> Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
>>>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
>>>> kernel failed: errno 111 (Connection refused)
>>>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
>>>> sockets for nfsd
>>>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
>>>> exited, code=exited, status=1/FAILURE
>>>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
>>>> result 'exit-code'.
>>>> Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
>>>> -----------------------------
>>>>
>>>> So, now I'm confused. Does NFSv4 need rpcbind to be running, does
>>>> it just need it when it launches, or something else? I made a local
>>>> copy of the systemd service file and edited out the rpcbind
>>>> dependency, so it's not that.
>>>
>>> Do you have v2 and v3 turned off in /etc/nfs.conf?
>>
>> It's an Ubuntu system, hence doesn't use /etc/nfs.conf; however I do
>> have these variables set in /etc/default/nfs-kernel-server :
>>
>> MOUNTD_NFS_V2="no"
>> MOUNTD_NFS_V3="no"
>> RPCMOUNTDOPTS="--manage-gids -N 2 -N 3"
>>
>> maybe this isn't the correct way to disable NFSv2/3, but it's all I
>> could find documented.
>
> That should do it, but if you want to verify that it worked, you can
> read /proc/fs/nfsd/versions.
That's it. The syntax above is *not* disabling NFSv3:
root@helios:~# cat /proc/fs/nfsd/versions
-2 +3 +4 +4.1 +4.2
>
>> The linux kernel version is 5.4.0, and the nfs-kernel-server package
>> version is 1:1.3.4-2.5ubuntu3.3 (so upstream 1.3.4), but I'm not
>> sure this is relevant.
>
> I can't reproduce the problem on my 5.9-ish server, but I also can't
> recall any relevant changes here.
>
> Looking back through the history.... Kinglong Mee fixed the server to
> ignore rpbind failures in the v4-only case about 7 years ago, back in
> 4.13.
>
> --b.
>
On Thu, Oct 01, 2020 at 04:05:13PM -0500, Patrick Goetz wrote:
>
>
> On 10/1/20 3:06 PM, J. Bruce Fields wrote:
> >On Thu, Oct 01, 2020 at 01:41:39PM -0500, Patrick Goetz wrote:
> >>Hi Bruce,
> >>
> >>Thanks for the reply. See below.
> >>
> >>On 10/1/20 1:30 PM, J. Bruce Fields wrote:
> >>>On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
> >>>>My University information security office does not like rpcbind and
> >>>>will automatically quarantine any system for which they detect a
> >>>>portmapper running on an exposed port.
> >>>>
> >>>>Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
> >>>>doesn't require rpcbind any more. For example, here's what it says
> >>>>in the current RHEL documentation:
> >>>>
> >>>>"NFS version 4 (NFSv4) works through firewalls and on the Internet,
> >>>>no longer requires an rpcbind service, supports Access Control Lists
> >>>>(ACLs), and utilizes stateful operations."
> >>>>
> >>>>https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
> >>>>
> >>>>I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
> >>>>absolutely will not start if it can't launch rpcbind as a precursor:
> >>>>
> >>>>-----------------------------
> >>>>root@helios:~# systemctl stop rpcbind
> >>>>Warning: Stopping rpcbind.service, but it can still be activated by:
> >>>> rpcbind.socket
> >>>>root@helios:~# systemctl mask rpcbind
> >>>>Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
> >>>>
> >>>>root@helios:~# systemctl restart nfs-server
> >>>>Job for nfs-server.service canceled.
> >>>>root@helios:~# systemctl status nfs-server
> >>>>● nfs-server.service - NFS server and services
> >>>> Loaded: loaded (/lib/systemd/system/nfs-server.service;
> >>>>enabled; vendor preset: enabled)
> >>>> Drop-In: /run/systemd/generator/nfs-server.service.d
> >>>> └─order-with-mounts.conf
> >>>> Active: failed (Result: exit-code) since Fri 2020-09-25
> >>>>14:21:46 UTC; 10s ago
> >>>> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
> >>>>status=0/SUCCESS)
> >>>> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
> >>>>(code=exited, status=1/FAILURE)
> >>>> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
> >>>>status=0/SUCCESS)
> >>>> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
> >>>>status=0/SUCCESS)
> >>>> Main PID: 3925 (code=exited, status=1/FAILURE)
> >>>>
> >>>>Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
> >>>>Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
> >>>>kernel failed: errno 111 (Connection refused)
> >>>>Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
> >>>>sockets for nfsd
> >>>>Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
> >>>>exited, code=exited, status=1/FAILURE
> >>>>Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
> >>>>result 'exit-code'.
> >>>>Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
> >>>>-----------------------------
> >>>>
> >>>>So, now I'm confused. Does NFSv4 need rpcbind to be running, does
> >>>>it just need it when it launches, or something else? I made a local
> >>>>copy of the systemd service file and edited out the rpcbind
> >>>>dependency, so it's not that.
> >>>
> >>>Do you have v2 and v3 turned off in /etc/nfs.conf?
> >>
> >>It's an Ubuntu system, hence doesn't use /etc/nfs.conf; however I do
> >>have these variables set in /etc/default/nfs-kernel-server :
> >>
> >> MOUNTD_NFS_V2="no"
> >> MOUNTD_NFS_V3="no"
> >> RPCMOUNTDOPTS="--manage-gids -N 2 -N 3"
> >>
> >>maybe this isn't the correct way to disable NFSv2/3, but it's all I
> >>could find documented.
> >
> >That should do it, but if you want to verify that it worked, you can
> >read /proc/fs/nfsd/versions.
>
> That's it. The syntax above is *not* disabling NFSv3:
>
> root@helios:~# cat /proc/fs/nfsd/versions
> -2 +3 +4 +4.1 +4.2
Looking more closely.... Does nfs-kernel-server have an RPCNFSDOPTS
variable or something? rpc.nfsd needs to be run with -N 2 -N 3 as well.
--b.
>
>
>
> >
> >>The linux kernel version is 5.4.0, and the nfs-kernel-server package
> >>version is 1:1.3.4-2.5ubuntu3.3 (so upstream 1.3.4), but I'm not
> >>sure this is relevant.
> >
> >I can't reproduce the problem on my 5.9-ish server, but I also can't
> >recall any relevant changes here.
> >
> >Looking back through the history.... Kinglong Mee fixed the server to
> >ignore rpbind failures in the v4-only case about 7 years ago, back in
> >4.13.
> >
> >--b.
> >
On 10/1/20 4:43 PM, J. Bruce Fields wrote:
> On Thu, Oct 01, 2020 at 04:05:13PM -0500, Patrick Goetz wrote:
>>
>>
>> On 10/1/20 3:06 PM, J. Bruce Fields wrote:
>>> On Thu, Oct 01, 2020 at 01:41:39PM -0500, Patrick Goetz wrote:
>>>> Hi Bruce,
>>>>
>>>> Thanks for the reply. See below.
>>>>
>>>> On 10/1/20 1:30 PM, J. Bruce Fields wrote:
>>>>> On Fri, Sep 25, 2020 at 09:40:16AM -0500, Patrick Goetz wrote:
>>>>>> My University information security office does not like rpcbind and
>>>>>> will automatically quarantine any system for which they detect a
>>>>>> portmapper running on an exposed port.
>>>>>>
>>>>>> Since I exclusively use NFSv4 I was happy to "learn" that NFSv4
>>>>>> doesn't require rpcbind any more. For example, here's what it says
>>>>>> in the current RHEL documentation:
>>>>>>
>>>>>> "NFS version 4 (NFSv4) works through firewalls and on the Internet,
>>>>>> no longer requires an rpcbind service, supports Access Control Lists
>>>>>> (ACLs), and utilizes stateful operations."
>>>>>>
>>>>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/exporting-nfs-shares_managing-file-systems#introduction-to-nfs_exporting-nfs-shares
>>>>>>
>>>>>> I'm using Ubuntu 20.04 rather than RHEL, but the nfs-server service
>>>>>> absolutely will not start if it can't launch rpcbind as a precursor:
>>>>>>
>>>>>> -----------------------------
>>>>>> root@helios:~# systemctl stop rpcbind
>>>>>> Warning: Stopping rpcbind.service, but it can still be activated by:
>>>>>> rpcbind.socket
>>>>>> root@helios:~# systemctl mask rpcbind
>>>>>> Created symlink /etc/systemd/system/rpcbind.service → /dev/null.
>>>>>>
>>>>>> root@helios:~# systemctl restart nfs-server
>>>>>> Job for nfs-server.service canceled.
>>>>>> root@helios:~# systemctl status nfs-server
>>>>>> ● nfs-server.service - NFS server and services
>>>>>> Loaded: loaded (/lib/systemd/system/nfs-server.service;
>>>>>> enabled; vendor preset: enabled)
>>>>>> Drop-In: /run/systemd/generator/nfs-server.service.d
>>>>>> └─order-with-mounts.conf
>>>>>> Active: failed (Result: exit-code) since Fri 2020-09-25
>>>>>> 14:21:46 UTC; 10s ago
>>>>>> Process: 3923 ExecStartPre=/usr/sbin/exportfs -r (code=exited,
>>>>>> status=0/SUCCESS)
>>>>>> Process: 3925 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
>>>>>> (code=exited, status=1/FAILURE)
>>>>>> Process: 3931 ExecStopPost=/usr/sbin/exportfs -au (code=exited,
>>>>>> status=0/SUCCESS)
>>>>>> Process: 3932 ExecStopPost=/usr/sbin/exportfs -f (code=exited,
>>>>>> status=0/SUCCESS)
>>>>>> Main PID: 3925 (code=exited, status=1/FAILURE)
>>>>>>
>>>>>> Sep 25 14:21:46 helios systemd[1]: Starting NFS server and services...
>>>>>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: writing fd to
>>>>>> kernel failed: errno 111 (Connection refused)
>>>>>> Sep 25 14:21:46 helios rpc.nfsd[3925]: rpc.nfsd: unable to set any
>>>>>> sockets for nfsd
>>>>>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Main process
>>>>>> exited, code=exited, status=1/FAILURE
>>>>>> Sep 25 14:21:46 helios systemd[1]: nfs-server.service: Failed with
>>>>>> result 'exit-code'.
>>>>>> Sep 25 14:21:46 helios systemd[1]: Stopped NFS server and services.
>>>>>> -----------------------------
>>>>>>
>>>>>> So, now I'm confused. Does NFSv4 need rpcbind to be running, does
>>>>>> it just need it when it launches, or something else? I made a local
>>>>>> copy of the systemd service file and edited out the rpcbind
>>>>>> dependency, so it's not that.
>>>>>
>>>>> Do you have v2 and v3 turned off in /etc/nfs.conf?
>>>>
>>>> It's an Ubuntu system, hence doesn't use /etc/nfs.conf; however I do
>>>> have these variables set in /etc/default/nfs-kernel-server :
>>>>
>>>> MOUNTD_NFS_V2="no"
>>>> MOUNTD_NFS_V3="no"
>>>> RPCMOUNTDOPTS="--manage-gids -N 2 -N 3"
>>>>
>>>> maybe this isn't the correct way to disable NFSv2/3, but it's all I
>>>> could find documented.
>>>
>>> That should do it, but if you want to verify that it worked, you can
>>> read /proc/fs/nfsd/versions.
>>
>> That's it. The syntax above is *not* disabling NFSv3:
>>
>> root@helios:~# cat /proc/fs/nfsd/versions
>> -2 +3 +4 +4.1 +4.2
>
> Looking more closely.... Does nfs-kernel-server have an RPCNFSDOPTS
> variable or something? rpc.nfsd needs to be run with -N 2 -N 3 as well.
>
> --b.
Hmmm, not exactly, but here are the relevant details from the
/usr/lib/systemd/system/nfs-server.service file:
-----------------------------
Wants=nfs-config.service
After=nfs-config.service
[Service]
EnvironmentFile=-/run/sysconfig/nfs-utils
Type=oneshot
RemainAfterExit=yes
ExecStartPre=/usr/sbin/exportfs -r
ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
-----------------------------
which I think explains why this isn't working properly, based on your
comment. The /run/sysconfig/nfs-utils file is assembled by
nfs-config.service from the /etc/default/nfs-kernel-server file:
root@helios:/run/sysconfig# cat nfs-utils
PIPEFS_MOUNTPOINT=/run/rpc_pipefs
RPCNFSDARGS=" 16"
RPCMOUNTDARGS="--manage-gids -N 2 -N 3"
STATDARGS=""
RPCSVCGSSDARGS=""
SVCGSSDARGS=""
So rpc.nfsd is only being started with $RPCNFSDARGS and not $RPCMOUNTDARGS
I think what you're saying is that I need to add $RPCMOUNTDARGS to the
service file command line for rpc.nfsd?
-
Ugh, lengthy aside: I'm finding so many bugs in Debian/Ubuntu packaging
based on packagers minimal understanding of how NFS/autofs work. We do
computation biology, and for almost a year were plagued by a performance
slow down which boiled down to these 2 lines in /etc/passwd:
syslog:x:102:106::/home/syslog:/usr/sbin/nologin
cups-pk-helper:x:124:118:user for cups-pk-helper
service,,,:/home/cups-pk-helper:/usr/sbin/nologin
Notice the invocation of non-existent home directories. On Arch Linux
systems these are set to /:
cups:x:209:209:cups helper user:/:/sbin/nologin
On non-network filesystem workstations this is harmless, but we use
autofs for home directory mounts, and the biologists run their software
from anaconda environments. A rather poor design decision, but when
launched, mini/anaconda scans through /etc/password looking for places
environments might be hidden away. autofs was hanging every time there
was an attempted access of a non-existent home directory. As experienced
by the researchers, they would try and run a program and it would just
hang 5-10 minutes loading some python module.
This is why I was complaining about documentation. There's now a whole
generation of IT professionals for whom NFS is entirely opaque due to a
lack of up to date documentation.
>
>>
>>
>>
>>>
>>>> The linux kernel version is 5.4.0, and the nfs-kernel-server package
>>>> version is 1:1.3.4-2.5ubuntu3.3 (so upstream 1.3.4), but I'm not
>>>> sure this is relevant.
>>>
>>> I can't reproduce the problem on my 5.9-ish server, but I also can't
>>> recall any relevant changes here.
>>>
>>> Looking back through the history.... Kinglong Mee fixed the server to
>>> ignore rpbind failures in the v4-only case about 7 years ago, back in
>>> 4.13.
>>>
>>> --b.
>>>
On Fri, Oct 02, 2020 at 10:12:24AM -0500, Patrick Goetz wrote:
> I think what you're saying is that I need to add $RPCMOUNTDARGS to
> the service file command line for rpc.nfsd?
Somehow you just need to make sure rpc.nfsd is also getting "-N 2 -N 3"
added to its commandline. I'm not sure of the right way to do that with
Debian's configuration.
--b.
On Mon, Oct 05, 2020 at 09:54:52AM -0400, J. Bruce Fields wrote:
>On Fri, Oct 02, 2020 at 10:12:24AM -0500, Patrick Goetz wrote:
>> I think what you're saying is that I need to add $RPCMOUNTDARGS to
>> the service file command line for rpc.nfsd?
>
>Somehow you just need to make sure rpc.nfsd is also getting "-N 2 -N 3"
>added to its commandline. I'm not sure of the right way to do that with
>Debian's configuration.
>
I think the canonical way to do this is to edit
/etc/default/nfs-kernel-server
and set
RPCNFSDOPTS="-N 2 -N3"
or similar.
There were some issues in the past with the nfs init scripts
not picking up settings in that file correctly, but I believe
they have been corrected.
Kind regards
Yep, just came here to post this myself and saw your message. The
problem is RPCNFSDOPTS is missing from the
/etc/default/nfs-kernel-server file.
The service file for the nfs server,
/usr/lib/systemd/system/nfs-config.service
includes this:
Wants=nfs-config.service
After=nfs-config.service
All nfs-config does is run this script:
ExecStart=/usr/lib/systemd/scripts/nfs-utils_env.sh
and the script reads the /etc/default/nfs* files and then
echo RPCNFSDARGS=\"$RPCNFSDOPTS ${RPCNFSDCOUNT:-8}\"
which then goes on the command line in the nfs-server service file:
ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
So the solution is to add this line to /etc/default/nfs-kernel-server:
RPCNFSDOPTS="-N 2 -N 3"
Still unaccounted for is this variable in /etc/default/nfs-kernel-server:
# Runtime priority of server (see nice(1))
RPCNFSDPRIORITY=0
Which I can't find used anywhere in the service files. But in any case,
adding the RPCNFSDOPTS variable resolved the issue:
root@helios:~# cat /proc/fs/nfsd/versions
-2 -3 +4 +4.1 +4.2
Thanks!
On 10/5/20 5:55 PM, McIntyre, Vincent (CASS, Marsfield) wrote:
> On Mon, Oct 05, 2020 at 09:54:52AM -0400, J. Bruce Fields wrote:
>> On Fri, Oct 02, 2020 at 10:12:24AM -0500, Patrick Goetz wrote:
>>> I think what you're saying is that I need to add $RPCMOUNTDARGS to
>>> the service file command line for rpc.nfsd?
>>
>> Somehow you just need to make sure rpc.nfsd is also getting "-N 2 -N 3"
>> added to its commandline. I'm not sure of the right way to do that with
>> Debian's configuration.
>>
>
> I think the canonical way to do this is to edit
> /etc/default/nfs-kernel-server
>
> and set
> RPCNFSDOPTS="-N 2 -N3"
>
> or similar.
>
> There were some issues in the past with the nfs init scripts
> not picking up settings in that file correctly, but I believe
> they have been corrected.
>
> Kind regards
>