Sadly, it took me a couple of days to track this down. The /etc/netgroup
file I'm using works perfectly on another NFS server (Ubuntu 18.04) in
production, so this wasn't an immediate suspicion. However, if I use
this /etc/exports:
/srv/nfs @cryo_em(rw,sync,fsid=0,crossmnt,no_subtree_check)
/srv/nfs/cryosparc @cryo_em(rw,sync,fsid=2,crossmnt,no_subtree_check)
Client mounts fail:
root@javelina:~# mount -vvvt nfs4 cerebro:/cryosparc /cryosparc
mount.nfs4: timeout set for Tue Jun 15 11:53:22 2021
mount.nfs4: trying text-based options
'vers=4.2,addr=128.xx.xx.xxx,clientaddr=129.xxx.xxx.xx'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting cerebro:/cryosparc
and if I switch to specifying the host explicitly:
/srv/nfs javelina.my.domain(rw,sync,fsid=0,crossmnt,no_subtree_check)
/srv/nfs/cryosparc
javelina.mydomain(rw,sync,fsid=2,crossmnt,no_subtree_check)
the mount just works. The tcpdump error message isn't terribly helpful
here:
11:14:02.856094 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
[.], ack 281, win 507, options [nop,nop,TS val 791638255 ecr
2576087678], length 0
11:14:02.856178 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
[P.], seq 1:25, ack 281, win 507, options [nop,nop,TS val 791638255 ecr
2576087678], length 24: NFS reply xid 2752089303 reply ERR 20: Auth
Bogus Credentials (seal broken)
but after figuring out the cause of the problem, I did find a
corroborating RHEL error report (which you'll need a RHEL account to
access):
https://access.redhat.com/solutions/3563601
I couldn't figure out how to determine the exact version of the NFS
server that ships with Ubuntu 20.04. Maybe someone could explain how to
do this. Running
/usr/sbin/rpc.nfsd --version
doesn't do it.
On Wed, 16 Jun 2021, Patrick Goetz wrote:
> Sadly, it took me a couple of days to track this down. The /etc/netgroup
> file I'm using works perfectly on another NFS server (Ubuntu 18.04) in
> production, so this wasn't an immediate suspicion. However, if I use
> this /etc/exports:
>
> /srv/nfs @cryo_em(rw,sync,fsid=0,crossmnt,no_subtree_check)
> /srv/nfs/cryosparc @cryo_em(rw,sync,fsid=2,crossmnt,no_subtree_check)
>
> Client mounts fail:
>
>
> root@javelina:~# mount -vvvt nfs4 cerebro:/cryosparc /cryosparc
> mount.nfs4: timeout set for Tue Jun 15 11:53:22 2021
> mount.nfs4: trying text-based options
> 'vers=4.2,addr=128.xx.xx.xxx,clientaddr=129.xxx.xxx.xx'
> mount.nfs4: mount(2): Permission denied
> mount.nfs4: access denied by server while mounting cerebro:/cryosparc
>
> and if I switch to specifying the host explicitly:
>
> /srv/nfs javelina.my.domain(rw,sync,fsid=0,crossmnt,no_subtree_check)
>
> /srv/nfs/cryosparc
> javelina.mydomain(rw,sync,fsid=2,crossmnt,no_subtree_check)
>
> the mount just works. The tcpdump error message isn't terribly helpful
> here:
>
> 11:14:02.856094 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
> [.], ack 281, win 507, options [nop,nop,TS val 791638255 ecr
> 2576087678], length 0
> 11:14:02.856178 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
> [P.], seq 1:25, ack 281, win 507, options [nop,nop,TS val 791638255 ecr
> 2576087678], length 24: NFS reply xid 2752089303 reply ERR 20: Auth
> Bogus Credentials (seal broken)
>
> but after figuring out the cause of the problem, I did find a
> corroborating RHEL error report (which you'll need a RHEL account to
> access):
>
> https://access.redhat.com/solutions/3563601
>
> I couldn't figure out how to determine the exact version of the NFS
> server that ships with Ubuntu 20.04. Maybe someone could explain how to
> do this. Running
> /usr/sbin/rpc.nfsd --version
> doesn't do it.
>
>
The problem is unlikely to be the implementation of netgroups - that
hasn't changed in a long time. It is more likely to be some subtle
configuration difference.
Can you provide the verbatim /etc/netgroups file, and the extact host
name that a DNS lookup of the client IP adress results in?
NeilBrown
Hi Neil -
This is extremely embarrassing, but chalk this one up to user error
(technically, PEBKAC). I'm used to /etc/nsswitch.conf always including
the files option, so didn't think to check this when in fact on Ubuntu
20.04 they ship the nsswitch.conf with this netgroup entry.
netgroup: nis
Who still uses NIS? Beats me; but once I added files to the option list
it starting working as advertised. Very sorry to dump random noise onto
the list.
On 6/16/21 10:11 PM, NeilBrown wrote:
> On Wed, 16 Jun 2021, Patrick Goetz wrote:
>> Sadly, it took me a couple of days to track this down. The /etc/netgroup
>> file I'm using works perfectly on another NFS server (Ubuntu 18.04) in
>> production, so this wasn't an immediate suspicion. However, if I use
>> this /etc/exports:
>>
>> /srv/nfs @cryo_em(rw,sync,fsid=0,crossmnt,no_subtree_check)
>> /srv/nfs/cryosparc @cryo_em(rw,sync,fsid=2,crossmnt,no_subtree_check)
>>
>> Client mounts fail:
>>
>>
>> root@javelina:~# mount -vvvt nfs4 cerebro:/cryosparc /cryosparc
>> mount.nfs4: timeout set for Tue Jun 15 11:53:22 2021
>> mount.nfs4: trying text-based options
>> 'vers=4.2,addr=128.xx.xx.xxx,clientaddr=129.xxx.xxx.xx'
>> mount.nfs4: mount(2): Permission denied
>> mount.nfs4: access denied by server while mounting cerebro:/cryosparc
>>
>> and if I switch to specifying the host explicitly:
>>
>> /srv/nfs javelina.my.domain(rw,sync,fsid=0,crossmnt,no_subtree_check)
>>
>> /srv/nfs/cryosparc
>> javelina.mydomain(rw,sync,fsid=2,crossmnt,no_subtree_check)
>>
>> the mount just works. The tcpdump error message isn't terribly helpful
>> here:
>>
>> 11:14:02.856094 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
>> [.], ack 281, win 507, options [nop,nop,TS val 791638255 ecr
>> 2576087678], length 0
>> 11:14:02.856178 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
>> [P.], seq 1:25, ack 281, win 507, options [nop,nop,TS val 791638255 ecr
>> 2576087678], length 24: NFS reply xid 2752089303 reply ERR 20: Auth
>> Bogus Credentials (seal broken)
>>
>> but after figuring out the cause of the problem, I did find a
>> corroborating RHEL error report (which you'll need a RHEL account to
>> access):
>>
>> https://access.redhat.com/solutions/3563601
>>
>> I couldn't figure out how to determine the exact version of the NFS
>> server that ships with Ubuntu 20.04. Maybe someone could explain how to
>> do this. Running
>> /usr/sbin/rpc.nfsd --version
>> doesn't do it.
>>
>>
>
> The problem is unlikely to be the implementation of netgroups - that
> hasn't changed in a long time. It is more likely to be some subtle
> configuration difference.
>
> Can you provide the verbatim /etc/netgroups file, and the extact host
> name that a DNS lookup of the client IP adress results in?
>
> NeilBrown
>
On Fri, 18 Jun 2021, Patrick Goetz wrote:
> Hi Neil -
>
> This is extremely embarrassing, but chalk this one up to user error
> (technically, PEBKAC). I'm used to /etc/nsswitch.conf always including
> the files option, so didn't think to check this when in fact on Ubuntu
> 20.04 they ship the nsswitch.conf with this netgroup entry.
>
> netgroup: nis
>
> Who still uses NIS? Beats me; but once I added files to the option list
> it starting working as advertised. Very sorry to dump random noise onto
> the list.
:-)
NIS has a certain elegant simplicity. LDAP is probably objective better
in every way ... except the elegant simplicity.
Glad to know I wasn't missing anything important, and that there was an
easy solution.
NeilBrown
>
>
> On 6/16/21 10:11 PM, NeilBrown wrote:
> > On Wed, 16 Jun 2021, Patrick Goetz wrote:
> >> Sadly, it took me a couple of days to track this down. The /etc/netgroup
> >> file I'm using works perfectly on another NFS server (Ubuntu 18.04) in
> >> production, so this wasn't an immediate suspicion. However, if I use
> >> this /etc/exports:
> >>
> >> /srv/nfs @cryo_em(rw,sync,fsid=0,crossmnt,no_subtree_check)
> >> /srv/nfs/cryosparc @cryo_em(rw,sync,fsid=2,crossmnt,no_subtree_check)
> >>
> >> Client mounts fail:
> >>
> >>
> >> root@javelina:~# mount -vvvt nfs4 cerebro:/cryosparc /cryosparc
> >> mount.nfs4: timeout set for Tue Jun 15 11:53:22 2021
> >> mount.nfs4: trying text-based options
> >> 'vers=4.2,addr=128.xx.xx.xxx,clientaddr=129.xxx.xxx.xx'
> >> mount.nfs4: mount(2): Permission denied
> >> mount.nfs4: access denied by server while mounting cerebro:/cryosparc
> >>
> >> and if I switch to specifying the host explicitly:
> >>
> >> /srv/nfs javelina.my.domain(rw,sync,fsid=0,crossmnt,no_subtree_check)
> >>
> >> /srv/nfs/cryosparc
> >> javelina.mydomain(rw,sync,fsid=2,crossmnt,no_subtree_check)
> >>
> >> the mount just works. The tcpdump error message isn't terribly helpful
> >> here:
> >>
> >> 11:14:02.856094 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
> >> [.], ack 281, win 507, options [nop,nop,TS val 791638255 ecr
> >> 2576087678], length 0
> >> 11:14:02.856178 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
> >> [P.], seq 1:25, ack 281, win 507, options [nop,nop,TS val 791638255 ecr
> >> 2576087678], length 24: NFS reply xid 2752089303 reply ERR 20: Auth
> >> Bogus Credentials (seal broken)
> >>
> >> but after figuring out the cause of the problem, I did find a
> >> corroborating RHEL error report (which you'll need a RHEL account to
> >> access):
> >>
> >> https://access.redhat.com/solutions/3563601
> >>
> >> I couldn't figure out how to determine the exact version of the NFS
> >> server that ships with Ubuntu 20.04. Maybe someone could explain how to
> >> do this. Running
> >> /usr/sbin/rpc.nfsd --version
> >> doesn't do it.
> >>
> >>
> >
> > The problem is unlikely to be the implementation of netgroups - that
> > hasn't changed in a long time. It is more likely to be some subtle
> > configuration difference.
> >
> > Can you provide the verbatim /etc/netgroups file, and the extact host
> > name that a DNS lookup of the client IP adress results in?
> >
> > NeilBrown
> >
>
>
On 6/17/21 4:24 PM, NeilBrown wrote:
> On Fri, 18 Jun 2021, Patrick Goetz wrote:
>> Hi Neil -
>>
>> This is extremely embarrassing, but chalk this one up to user error
>> (technically, PEBKAC). I'm used to /etc/nsswitch.conf always including
>> the files option, so didn't think to check this when in fact on Ubuntu
>> 20.04 they ship the nsswitch.conf with this netgroup entry.
>>
>> netgroup: nis
>>
>> Who still uses NIS? Beats me; but once I added files to the option list
>> it starting working as advertised. Very sorry to dump random noise onto
>> the list.
>
> :-)
> NIS has a certain elegant simplicity. LDAP is probably objective better
> in every way ... except the elegant simplicity.
>
I agree. I hate LDAP (so inelegant) and I find ASN.1 to be especially
inelegant. (I've never seen anything to indicate this is anything other
than an anachronism.)
> Glad to know I wasn't missing anything important, and that there was an
> easy solution.
>
> NeilBrown
>
>
>>
>>
>> On 6/16/21 10:11 PM, NeilBrown wrote:
>>> On Wed, 16 Jun 2021, Patrick Goetz wrote:
>>>> Sadly, it took me a couple of days to track this down. The /etc/netgroup
>>>> file I'm using works perfectly on another NFS server (Ubuntu 18.04) in
>>>> production, so this wasn't an immediate suspicion. However, if I use
>>>> this /etc/exports:
>>>>
>>>> /srv/nfs @cryo_em(rw,sync,fsid=0,crossmnt,no_subtree_check)
>>>> /srv/nfs/cryosparc @cryo_em(rw,sync,fsid=2,crossmnt,no_subtree_check)
>>>>
>>>> Client mounts fail:
>>>>
>>>>
>>>> root@javelina:~# mount -vvvt nfs4 cerebro:/cryosparc /cryosparc
>>>> mount.nfs4: timeout set for Tue Jun 15 11:53:22 2021
>>>> mount.nfs4: trying text-based options
>>>> 'vers=4.2,addr=128.xx.xx.xxx,clientaddr=129.xxx.xxx.xx'
>>>> mount.nfs4: mount(2): Permission denied
>>>> mount.nfs4: access denied by server while mounting cerebro:/cryosparc
>>>>
>>>> and if I switch to specifying the host explicitly:
>>>>
>>>> /srv/nfs javelina.my.domain(rw,sync,fsid=0,crossmnt,no_subtree_check)
>>>>
>>>> /srv/nfs/cryosparc
>>>> javelina.mydomain(rw,sync,fsid=2,crossmnt,no_subtree_check)
>>>>
>>>> the mount just works. The tcpdump error message isn't terribly helpful
>>>> here:
>>>>
>>>> 11:14:02.856094 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
>>>> [.], ack 281, win 507, options [nop,nop,TS val 791638255 ecr
>>>> 2576087678], length 0
>>>> 11:14:02.856178 IP cerebro.my.domain.nfs > javelina.my.domain.741: Flags
>>>> [P.], seq 1:25, ack 281, win 507, options [nop,nop,TS val 791638255 ecr
>>>> 2576087678], length 24: NFS reply xid 2752089303 reply ERR 20: Auth
>>>> Bogus Credentials (seal broken)
>>>>
>>>> but after figuring out the cause of the problem, I did find a
>>>> corroborating RHEL error report (which you'll need a RHEL account to
>>>> access):
>>>>
>>>> https://access.redhat.com/solutions/3563601
>>>>
>>>> I couldn't figure out how to determine the exact version of the NFS
>>>> server that ships with Ubuntu 20.04. Maybe someone could explain how to
>>>> do this. Running
>>>> /usr/sbin/rpc.nfsd --version
>>>> doesn't do it.
>>>>
>>>>
>>>
>>> The problem is unlikely to be the implementation of netgroups - that
>>> hasn't changed in a long time. It is more likely to be some subtle
>>> configuration difference.
>>>
>>> Can you provide the verbatim /etc/netgroups file, and the extact host
>>> name that a DNS lookup of the client IP adress results in?
>>>
>>> NeilBrown
>>>
>>
>>