Hi all,
I have a FreeBSD nfsv3 server that exports a filesystem with sec=krb5.
Mounting the share with sec=krb5 under a fbsd client works fine. I now
try to mount it under linux (archlinux, upgraded today) using nfs-utils.
Heimdal is configured on the linux-box, kinit -k linuxclient works fine.
I am also able to kinit to my user principals from it. When I try to
mount the nvsv3 sec=krb5 share, I get the following error:
# mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
# tail /var/log/messages
Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying clients;
Exiting!
Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root. chown
/var/lib/nfs to choose different user
Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying clients;
Exiting!
Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1 RPC
service (errno 111).
Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1 RPC
service (errno 111).
Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1 RPC
service (errno 111).
Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1 RPC
service (errno 111).
and it keeps on like this.
My nfs-common.conf reads:
[root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
# Parameters to be passed to nfs-common (nfs clients & server) init script.
#
# If you do not set values for the NEED_ options, they will be attempted
# autodetected; this should be sufficient for most people. Valid
alternatives
# for the NEED_ options are "yes" and "no".
# Do you want to start the statd daemon? It is not needed for NFSv4.
NEED_STATD=""
# Options to pass to rpc.statd.
# See rpc.statd(8) for more details.
# N.B. statd normally runs on both client and server, and run-time
# options should be specified accordingly.
# STATD_OPTS="-p 32765 -o 32766"
STATD_OPTS=""
# Options to pass to sm-notify
# e.g. SMNOTIFY_OPTS="-p 32764"
SMNOTIFY_OPTS=""
# Do you want to start the idmapd daemon? It is only needed for NFSv4.
NEED_IDMAPD=""
# Options to pass to rpc.idmapd.
# See rpc.idmapd(8) for more details.
IDMAPD_OPTS="-vvv"
# Do you want to start the gssd daemon? It is required for Kerberos mounts.
NEED_GSSD="yes"
# Options to pass to rpc.gssd.
# See rpc.gssd(8) for more details.
GSSD_OPTS="-vvv"
#RPCGSSDOPTS="-vvv"
# Where to mount rpc_pipefs filesystem; the default is
"/var/lib/nfs/rpc_pipefs".
PIPEFS_MOUNTPOINT=""
# Options used to mount rpc_pipefs filesystem; the default is "defaults".
PIPEFS_MOUNTOPTS=""
my rpc processes are:
[root@linuxclient ~]# ps axuww | grep -i rpc
root 1228 0.0 0.0 0 0 ? S 14:47 0:00 [rpciod/0]
root 27670 0.0 0.0 6232 908 ? Ss 16:05 0:00
/usr/bin/rpcbind
root 27683 0.0 0.1 6332 1236 ? Ss 16:05 0:00
/usr/sbin/rpc.statd
root 27699 0.0 0.1 6264 1180 ? Ss 16:05 0:00
/usr/sbin/rpc.gssd -vvv
root 27720 0.0 0.0 3776 476 pts/0 S+ 17:01 0:00 grep -i
rpc
And rpcinfo shows:
root@linuxclient ~]# rpcinfo
program version netid address service owner
100000 4 tcp6 ::.0.111 portmapper superuser
100000 3 tcp6 ::.0.111 portmapper superuser
100000 4 udp6 ::.0.111 portmapper superuser
100000 3 udp6 ::.0.111 portmapper superuser
100000 4 udp 0.0.0.0.0.111 portmapper superuser
100000 3 udp 0.0.0.0.0.111 portmapper superuser
100000 2 udp 0.0.0.0.0.111 portmapper superuser
100000 4 local /var/run/rpcbind.sock portmapper superuser
100000 3 local /var/run/rpcbind.sock portmapper superuser
100024 1 udp 0.0.0.0.228.144 status superuser
100024 1 tcp 0.0.0.0.198.8 status superuser
[root@linuxclient ~]# rpcinfo -s
program version(s) netid(s) service owner
100000 2,3,4 local,udp,udp6,tcp6 portmapper
superuser
100024 1 tcp,udp status
superuser
whereas on the fbsd box I have:
[root@fbsdserver ~]# rpcinfo -s
program version(s) netid(s) service owner
100000 2,3,4 local,udp6,tcp6,udp,tcp rpcbind
superuser
100024 1 tcp,udp,tcp6,udp6 status
superuser
100021 4,3,1,0 tcp,udp,tcp6,udp6 nlockmgr
superuser
100003 3,2 tcp6,tcp,udp6,udp nfs
superuser
100005 3,1 tcp,udp,tcp6,udp6 mountd
superuser
The versions I use are:
rpcbind-0.2.0-2
nfs-utils-1.2.2-3
And uname -a shows:
[root@linuxclient ~]# uname -a
Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC
2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel GNU/Linux
Does linux support RPCSEC_GSS security flavors over nvsv3? And if so,
could somebody direct me on how to establish mounting the remote share?
Thank you all for your time and attention in advance,
regards,
mamalos
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
On Sep 17, 2010, at 1:43 PM, J. Bruce Fields wrote:
> On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
>> Hi all,
>>
>> I have a FreeBSD nfsv3 server that exports a filesystem with
>> sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
>> fine. I now try to mount it under linux (archlinux, upgraded today)
>> using nfs-utils. Heimdal is configured on the linux-box, kinit -k
>> linuxclient works fine. I am also able to kinit to my user
>> principals from it. When I try to mount the nvsv3 sec=krb5 share, I
>> get the following error:
>>
>> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
>>
>
> Was there supposed to be some error output there? Or did the mount just
> hang?
>
>> # tail /var/log/messages
>> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
>> clients; Exiting!
>> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
>> chown /var/lib/nfs to choose different user
>> Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
>> Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
>> clients; Exiting!
>> Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
>> Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
>
> 111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
> to register lockd fail the whole mount?
I could be wrong, but I believe that yes, it should fail the mount operation in this case, unless you specified "nolock".
> I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?
--
chuck[dot]lever[at]oracle[dot]com
On Tue, Sep 21, 2010 at 11:36:18AM +0300, George Mamalakis wrote:
> Hi guys,
>
> Kevin:
> As stated in my first email, the linux box is running on a 2.6.35
> kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),
As Kevin says, it would be worth trying MIT kerberos instead of heimdal
if you can, as that is generally better tested.
--b.
> rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.
>
> Bruce: I would rather find an "easier" (configuration-specific) way
> to overcome my problem instead of delving into the code, since I am
> not a developer (I am an admin :)). Nevertheless, when I used
> google.com/searchcode for _gss_get_mechanism_cred I found that there
> is a libgss/g_glue.c that defines this function in the following way
> (the site is svn://svn.genunix.org/on/trunk <http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):
>
> /*
> * Glue routine for returning the mechanism-specific credential from a
> * external union credential.
> */
> gss_cred_id_t
> __gss_get_mechanism_cred(union_cred, mech_type)
> const gss_union_cred_t union_cred;
> const gss_OID mech_type;
> {
> int i;
>
> if (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
> return (GSS_C_NO_CREDENTIAL);
>
> for (i =0; i< union_cred->count; i++) {
> if (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
> return (union_cred->cred_array[i]);
> }
> return (GSS_C_NO_CREDENTIAL);
> }
>
>
> where you can see that the guy returns a cred_array[i] element, that
> obviously is null, because (most probably) the if statement above
> did not match any "compatible" mechanisms during the recursion.
>
> Maybe I am totally wrong, since I haven't checked the code
> thoroughly, but I think that the problem is located somewhere in
> this code segment.
>
> As I stated before, I was rather hoping to avoid this procedure,
> since I decided to use the linux distro for my clients due to its
> 'out-of-the-box' and 'everything-is-supported' benefits as
> (slightly-)opposed to fbsd (which I use in all my servers and
> personal computers, and "love" as an operating system) can be a bit
> more "demanding" when out-of-the-box-general-purpose issues are
> involved. The thing is that the client will be used as a
> general-purpose student pc in a university lab.
>
> Thanx again for your help,
>
> if anybody could direct me somewhere else I would be delighted to know so!
>
> mamalos
>
> --
> George Mamalakis
>
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
>
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
>
> phone number : +30 (2310) 994379
>
On Mon, Sep 20, 2010 at 11:38 AM, George Mamalakis <[email protected]> wrote:
> ?On 20/09/2010 17:56, J. Bruce Fields wrote:
>>
>> Ouch. ?Well, that's a problem.
>>
>> (And have you checked the log files to make sure it's gssd segfaulting
>> and not some kind of crash in the kernel?)
>>
>> --b.
>>
>
> Yes, it segfaults, even the log shows it. It's a null pointer exception. I
> used gdb in gssd and this is my outcome:
>
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> (gdb) bt
> #0 ?0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> #1 ?0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
> #2 ?0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
> #3 ?0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
> #4 ?0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
> #5 ?0x0804cbdc in ?? ()
> #6 ?0x0804cf9f in ?? ()
> #7 ?0x0804d7c8 in ?? ()
> #8 ?0x0804b7a8 in ?? ()
> #9 ?0x0804b309 in ?? ()
> #10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
> #11 0x0804a621 in ?? ()
> (gdb) info r
> eax ? ? ? ? ? ?0x0 ? ?0
> ecx ? ? ? ? ? ?0x9 ? ?9
> edx ? ? ? ? ? ?0x0 ? ?0
> ebx ? ? ? ? ? ?0xb72b9710 ? ?-1221880048
> esp ? ? ? ? ? ?0xbfe0b844 ? ?0xbfe0b844
> ebp ? ? ? ? ? ?0xbfe0b858 ? ?0xbfe0b858
> esi ? ? ? ? ? ?0x9 ? ?9
> edi ? ? ? ? ? ?0x9370928 ? ?154601768
> eip ? ? ? ? ? ?0xb72b4c8e ? ?0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags ? ? ? ? 0x210246 ? ?[ PF ZF IF RF ID ]
> cs ? ? ? ? ? ? 0x73 ? ?115
> ss ? ? ? ? ? ? 0x7b ? ?123
> ds ? ? ? ? ? ? 0x7b ? ?123
> es ? ? ? ? ? ? 0x7b ? ?123
> fs ? ? ? ? ? ? 0x0 ? ?0
> gs ? ? ? ? ? ? 0x33 ? ?51
> (gdb) disass $eip
> Dump of assembler code for function __gss_get_mechanism_cred:
> ? 0xb72b4c50 <+0>: ? ?push ? %ebp
> ? 0xb72b4c51 <+1>: ? ?mov ? ?%esp,%ebp
> ? 0xb72b4c53 <+3>: ? ?push ? %edi
> ? 0xb72b4c54 <+4>: ? ?push ? %esi
> ? 0xb72b4c55 <+5>: ? ?sub ? ?$0xc,%esp
> ? 0xb72b4c58 <+8>: ? ?mov ? ?0x8(%ebp),%ecx
> ? 0xb72b4c5b <+11>: ? ?test ? %ecx,%ecx
> ? 0xb72b4c5d <+13>: ? ?jne ? ?0xb72b4c68 <__gss_get_mechanism_cred+24>
> ? 0xb72b4c5f <+15>: ? ?add ? ?$0xc,%esp
> ? 0xb72b4c62 <+18>: ? ?xor ? ?%eax,%eax
> ? 0xb72b4c64 <+20>: ? ?pop ? ?%esi
> ? 0xb72b4c65 <+21>: ? ?pop ? ?%edi
> ? 0xb72b4c66 <+22>: ? ?pop ? ?%ebp
> ? 0xb72b4c67 <+23>: ? ?ret
> ? 0xb72b4c68 <+24>: ? ?mov ? ?0x8(%ebp),%eax
> ? 0xb72b4c6b <+27>: ? ?mov ? ?(%eax),%eax
> ? 0xb72b4c6d <+29>: ? ?test ? %eax,%eax
> ? 0xb72b4c6f <+31>: ? ?mov ? ?%eax,-0x10(%ebp)
> ? 0xb72b4c72 <+34>: ? ?jle ? ?0xb72b4c5f <__gss_get_mechanism_cred+15>
> ? 0xb72b4c74 <+36>: ? ?mov ? ?0xc(%ebp),%ecx
> ? 0xb72b4c77 <+39>: ? ?mov ? ?0x8(%ebp),%eax
> ? 0xb72b4c7a <+42>: ? ?mov ? ?(%ecx),%edx
> ? 0xb72b4c7c <+44>: ? ?mov ? ?0x4(%eax),%eax
> ? 0xb72b4c7f <+47>: ? ?mov ? ?%edx,-0x14(%ebp)
> ? 0xb72b4c82 <+50>: ? ?mov ? ?%eax,-0xc(%ebp)
> ? 0xb72b4c85 <+53>: ? ?xor ? ?%eax,%eax
> ? 0xb72b4c87 <+55>: ? ?nop
> ? 0xb72b4c88 <+56>: ? ?mov ? ?-0xc(%ebp),%edx
> ? 0xb72b4c8b <+59>: ? ?mov ? ?-0x14(%ebp),%ecx
> => 0xb72b4c8e <+62>: ? ?cmp ? ?(%edx,%eax,8),%ecx
> ? 0xb72b4c91 <+65>: ? ?jne ? ?0xb72b4cab <__gss_get_mechanism_cred+91>
> ? 0xb72b4c93 <+67>: ? ?mov ? ?0xc(%ebp),%edx
> ? 0xb72b4c96 <+70>: ? ?mov ? ?-0xc(%ebp),%ecx
> ? 0xb72b4c99 <+73>: ? ?mov ? ?0x4(%edx),%esi
> ? 0xb72b4c9c <+76>: ? ?mov ? ?-0x14(%ebp),%edx
> ---Type <return> to continue, or q <return> to quit---q
> Quit
> (gdb) info r
> eax ? ? ? ? ? ?0x0 ? ?0
> ecx ? ? ? ? ? ?0x9 ? ?9
> edx ? ? ? ? ? ?0x0 ? ?0
> ebx ? ? ? ? ? ?0xb72b9710 ? ?-1221880048
> esp ? ? ? ? ? ?0xbfe0b844 ? ?0xbfe0b844
> ebp ? ? ? ? ? ?0xbfe0b858 ? ?0xbfe0b858
> esi ? ? ? ? ? ?0x9 ? ?9
> edi ? ? ? ? ? ?0x9370928 ? ?154601768
> eip ? ? ? ? ? ?0xb72b4c8e ? ?0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags ? ? ? ? 0x210246 ? ?[ PF ZF IF RF ID ]
> cs ? ? ? ? ? ? 0x73 ? ?115
> ss ? ? ? ? ? ? 0x7b ? ?123
> ds ? ? ? ? ? ? 0x7b ? ?123
> es ? ? ? ? ? ? 0x7b ? ?123
> fs ? ? ? ? ? ? 0x0 ? ?0
> gs ? ? ? ? ? ? 0x33 ? ?51
>
>
> Where you can see the null reference.
>
> This, on the other hand, reminded me a discussion I had in the fbsd-stable
> list, where somebody advised me to use des-cbc-crc keytabs, and then
> experiment with other encryption types. Even though I didn't have such
> issues in fbsd, this might be the case on this problem. Maybe linux tries to
> compare an inexistent mech-type (or enc-type, whatever) with one of it's
> list, and hence it segfaults. So, to shed a little more light into the
> issue:
>
> - the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
> - on the linuxbox when I "kinit -k linuxclient" everything works fine.
> - on the linux client "ktutil -k /etc/krb5.keytab list" shows:
> /etc/krb5.keytab:
>
> Vno ?Type ? ? ? ? ? ? ? ? ? ? Principal
> Aliases
> ?1 ?des-cbc-md5 ? ? ? ? ? ? ?host/linuxclient@EXAMPLE
> ?1 ?des-cbc-md4 ? ? ? ? ? ? ?host/linuxclient@EXAMPLE
> ?1 ?des-cbc-crc ? ? ? ? ? ? ?host/linuxclient@EXAMPLE
> ?1 ?aes256-cts-hmac-sha1-96 ?host/linuxclient@EXAMPLE
> ?1 ?des3-cbc-sha1 ? ? ? ? ? ?host/linuxclient@EXAMPLE
> ?1 ?arcfour-hmac-md5 ? ? ? ? host/linuxclient@EXAMPLE
>
> So maybe what I have to do next is to create a keytab of des-cbc-crc
> encryption-type only for the client (maybe also for the nfsserver as well..)
> and see how it'll behave...but I will do it tomorrow.
>
> Until then,
>
> mamalos
Kernels before 2.6.35:
gssd has code to limit the encryption types negotiated, so limiting
its keytab to only des is not strictly required. (callbacks can
complicate this) It is required for a linux NFS server.
Kernels 2.6.35 and after:
However I saw from your previous message that the kernel is passing up
"enctypes=18,17,16,23,3,1,2", which indicates it is a rather new
kernel. What version is your linux kernel? AFAIK, a version of
nfs-utils with the changes to support newer enctypes has not yet been
released.
What kerberos libraries are on the linux client machine? (MIT or
Heimdal?) And what version? I've seen people successfully using
Heimdal libraries with gssd, but AFAIK it has not been tested like it
has with MIT.
On 20/09/2010 17:56, J. Bruce Fields wrote:
>
> Ouch. Well, that's a problem.
>
> (And have you checked the log files to make sure it's gssd segfaulting
> and not some kind of crash in the kernel?)
>
> --b.
>
Yes, it segfaults, even the log shows it. It's a null pointer exception.
I used gdb in gssd and this is my outcome:
0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.
Program received signal SIG37, Real-time event 37.
0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.
Program received signal SIG37, Real-time event 37.
0xb783f424 in __kernel_vsyscall ()
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
(gdb) bt
#0 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
#1 0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
#2 0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
#3 0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
#4 0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
#5 0x0804cbdc in ?? ()
#6 0x0804cf9f in ?? ()
#7 0x0804d7c8 in ?? ()
#8 0x0804b7a8 in ?? ()
#9 0x0804b309 in ?? ()
#10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
#11 0x0804a621 in ?? ()
(gdb) info r
eax 0x0 0
ecx 0x9 9
edx 0x0 0
ebx 0xb72b9710 -1221880048
esp 0xbfe0b844 0xbfe0b844
ebp 0xbfe0b858 0xbfe0b858
esi 0x9 9
edi 0x9370928 154601768
eip 0xb72b4c8e 0xb72b4c8e <__gss_get_mechanism_cred+62>
eflags 0x210246 [ PF ZF IF RF ID ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x33 51
(gdb) disass $eip
Dump of assembler code for function __gss_get_mechanism_cred:
0xb72b4c50 <+0>: push %ebp
0xb72b4c51 <+1>: mov %esp,%ebp
0xb72b4c53 <+3>: push %edi
0xb72b4c54 <+4>: push %esi
0xb72b4c55 <+5>: sub $0xc,%esp
0xb72b4c58 <+8>: mov 0x8(%ebp),%ecx
0xb72b4c5b <+11>: test %ecx,%ecx
0xb72b4c5d <+13>: jne 0xb72b4c68 <__gss_get_mechanism_cred+24>
0xb72b4c5f <+15>: add $0xc,%esp
0xb72b4c62 <+18>: xor %eax,%eax
0xb72b4c64 <+20>: pop %esi
0xb72b4c65 <+21>: pop %edi
0xb72b4c66 <+22>: pop %ebp
0xb72b4c67 <+23>: ret
0xb72b4c68 <+24>: mov 0x8(%ebp),%eax
0xb72b4c6b <+27>: mov (%eax),%eax
0xb72b4c6d <+29>: test %eax,%eax
0xb72b4c6f <+31>: mov %eax,-0x10(%ebp)
0xb72b4c72 <+34>: jle 0xb72b4c5f <__gss_get_mechanism_cred+15>
0xb72b4c74 <+36>: mov 0xc(%ebp),%ecx
0xb72b4c77 <+39>: mov 0x8(%ebp),%eax
0xb72b4c7a <+42>: mov (%ecx),%edx
0xb72b4c7c <+44>: mov 0x4(%eax),%eax
0xb72b4c7f <+47>: mov %edx,-0x14(%ebp)
0xb72b4c82 <+50>: mov %eax,-0xc(%ebp)
0xb72b4c85 <+53>: xor %eax,%eax
0xb72b4c87 <+55>: nop
0xb72b4c88 <+56>: mov -0xc(%ebp),%edx
0xb72b4c8b <+59>: mov -0x14(%ebp),%ecx
=> 0xb72b4c8e <+62>: cmp (%edx,%eax,8),%ecx
0xb72b4c91 <+65>: jne 0xb72b4cab <__gss_get_mechanism_cred+91>
0xb72b4c93 <+67>: mov 0xc(%ebp),%edx
0xb72b4c96 <+70>: mov -0xc(%ebp),%ecx
0xb72b4c99 <+73>: mov 0x4(%edx),%esi
0xb72b4c9c <+76>: mov -0x14(%ebp),%edx
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) info r
eax 0x0 0
ecx 0x9 9
edx 0x0 0
ebx 0xb72b9710 -1221880048
esp 0xbfe0b844 0xbfe0b844
ebp 0xbfe0b858 0xbfe0b858
esi 0x9 9
edi 0x9370928 154601768
eip 0xb72b4c8e 0xb72b4c8e <__gss_get_mechanism_cred+62>
eflags 0x210246 [ PF ZF IF RF ID ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x33 51
Where you can see the null reference.
This, on the other hand, reminded me a discussion I had in the
fbsd-stable list, where somebody advised me to use des-cbc-crc keytabs,
and then experiment with other encryption types. Even though I didn't
have such issues in fbsd, this might be the case on this problem. Maybe
linux tries to compare an inexistent mech-type (or enc-type, whatever)
with one of it's list, and hence it segfaults. So, to shed a little more
light into the issue:
- the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
- on the linuxbox when I "kinit -k linuxclient" everything works fine.
- on the linux client "ktutil -k /etc/krb5.keytab list" shows:
/etc/krb5.keytab:
Vno Type Principal
Aliases
1 des-cbc-md5 host/linuxclient@EXAMPLE
1 des-cbc-md4 host/linuxclient@EXAMPLE
1 des-cbc-crc host/linuxclient@EXAMPLE
1 aes256-cts-hmac-sha1-96 host/linuxclient@EXAMPLE
1 des3-cbc-sha1 host/linuxclient@EXAMPLE
1 arcfour-hmac-md5 host/linuxclient@EXAMPLE
So maybe what I have to do next is to create a keytab of des-cbc-crc
encryption-type only for the client (maybe also for the nfsserver as
well..) and see how it'll behave...but I will do it tomorrow.
Until then,
mamalos
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
I did some more testing, so I run a new kdc (heimdal again) on the
fbsd server (just to eliminate one candidate from my problematic-boxes);
this one is from the fbsd base system (the previous one was from ports),
and the version is 1.1.0. I init'd my realm, and changed the
/etc/krb.conf files on server and client to reflect my new kerberos
implementation.
When I tried to mount the remote share using sec=krb5, gssd segfaulted
again; I was on the exact place as before. Then I started deleting
encryption types from the keytab (using kadmin and del_enctype) to see
if this would help, until only one encryption type was left
(des3-cbc-sha1), but gssd segfaulted repeatedly. Two funny things that I
noticed:
1) when I del_enctype'd des3-cbc-sha1 and tried to kinit -t keytab -k
host/linclient on the linuxbox I got the error:
[root@linuxclient ]# kinit -t keytab -k host/linclient
kinit: krb5_get_init_creds: Looping 11 times while getting initial
credentials
[root@linuxclient ]#
and on the server tail /var/heimdal/kdc.log showed:
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
2010-09-21T16:26:36 AS-REQ host/linclient@EXAMPLE from
IPv4:192.168.0.235 for krbtgt/EXAMPLE@EXAMPLE
2010-09-21T16:26:36 No preauth found, returning PREAUTH-REQUIRED --
host/linclient@EXAMPLE
which means that there is
a) some strange "incompatibility" issue with the keytabs provided by
fbsd's heimdal version and the one that linux ships with.
b) the way linux kinit renders a keytab file becomes "problematic" when
des3-cbc-sha1 is absent (on fbsd kinit worked fine with the same
keytab). Haven't setup a linux heimdal server (yet...) to clear this.
but this is a complete different discussion.
The thing is that gssd creates it's krb5 cache in /tmp without any
issues, and dies only after trying to "create context with the server".
This is illustrated below, where gssd is run manually with verbose flags:
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt49
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt4b)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt4b)
process_krb5_upcall: service is '<null>'
Full hostname for 'filesrv' is 'filesrv'
Full hostname for 'linclient' is 'linclient'
Failed to find root/linclient@EXAMPLE in keytab FILE:/etc/krb5.keytab
(null) while getting keytab entry for 'root/linclient@EXAMPLE'
Failed to find nfs/linclient@EXAMPLE in keytab FILE:/etc/krb5.keytab
(null) while getting keytab entry for 'nfs/linclient@EXAMPLE'
Success getting keytab entry for 'host/linclient@EXAMPLE'
Successfully obtained machine credentials for principal
'host/linclient@EXAMPLE' stored in ccache 'FILE:/tmp/krb5cc_machine_EXAMPLE'
INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_EXAMPLE' are good
until 1285112422
using FILE:/tmp/krb5cc_machine_EXAMPLE as credentials cache for machine
creds
using gss_krb5_ccache_name to select krb5 ccache
FILE:/tmp/krb5cc_machine_EXAMPLE
creating context using fsuid 0 (save_uid 0)
creating tcp client for server filesrv
DEBUG: port already set to 2049
creating context with server nfs@filesrv
Segmentation fault
The cache in /tmp/krb5cc_machine_EXAMPLE file is ok, since I am able to
klist -c /tmp/krb5cc_machine_EXAMPLE and read its content. The problem
comes after that step, when the client tries to find available
mechanisms (stated in my last email).
2) The second funny thing was that when I searched the list for relevant
problems I found a mail I had sent on the 5th March 2010, where I was
facing some analogous problems, only this time with nfsv4 and a solaris
nfs server. The subject was "mount.nfs4: Broken pipe", and from my
"investigation" that far, I saw that there had to be a mix with MIT and
heimdal on arch-linux. I think that this must be the case now, since
when I read /etc/gssapi_mech.conf, I realized that it discussed about
MIT kerberos5, and /usr/lib/libgssapi.so must be relevant to MIT too.
But pacman -Qo /usr/lib/libgssapi.so showed that it is installed by
heimdal 1.3.3-1...
...I don't know, I am confused! Is there a config
(/etc/conf.d/nfs-common.conf, relevant packages, etc.) with which
someone is able to mount nfsv3 shares using sec=krb5? If anyone is able
to shed some light, it would be reaaally helpful.
Regards,
mamalos
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
On Mon, Sep 20, 2010 at 06:38:48PM +0300, George Mamalakis wrote:
> On 20/09/2010 17:56, J. Bruce Fields wrote:
> >
> >Ouch. Well, that's a problem.
> >
> >(And have you checked the log files to make sure it's gssd segfaulting
> >and not some kind of crash in the kernel?)
> >
> >--b.
> >
>
> Yes, it segfaults, even the log shows it. It's a null pointer
> exception. I used gdb in gssd and this is my outcome:
>
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIG37, Real-time event 37.
> 0xb783f424 in __kernel_vsyscall ()
> (gdb) c
> Continuing.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> (gdb) bt
> #0 0xb72b4c8e in __gss_get_mechanism_cred () from /usr/lib/libgssglue.so.1
> #1 0xb72b5c2c in gss_init_sec_context () from /usr/lib/libgssglue.so.1
> #2 0xb776c215 in authgss_refresh () from /usr/lib/libtirpc.so.1
> #3 0xb776c67d in authgss_create () from /usr/lib/libtirpc.so.1
> #4 0xb776c779 in authgss_create_default () from /usr/lib/libtirpc.so.1
> #5 0x0804cbdc in ?? ()
> #6 0x0804cf9f in ?? ()
> #7 0x0804d7c8 in ?? ()
> #8 0x0804b7a8 in ?? ()
> #9 0x0804b309 in ?? ()
> #10 0xb761cc76 in __libc_start_main () from /lib/libc.so.6
> #11 0x0804a621 in ?? ()
> (gdb) info r
> eax 0x0 0
> ecx 0x9 9
> edx 0x0 0
> ebx 0xb72b9710 -1221880048
> esp 0xbfe0b844 0xbfe0b844
> ebp 0xbfe0b858 0xbfe0b858
> esi 0x9 9
> edi 0x9370928 154601768
> eip 0xb72b4c8e 0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags 0x210246 [ PF ZF IF RF ID ]
> cs 0x73 115
> ss 0x7b 123
> ds 0x7b 123
> es 0x7b 123
> fs 0x0 0
> gs 0x33 51
> (gdb) disass $eip
> Dump of assembler code for function __gss_get_mechanism_cred:
> 0xb72b4c50 <+0>: push %ebp
> 0xb72b4c51 <+1>: mov %esp,%ebp
> 0xb72b4c53 <+3>: push %edi
> 0xb72b4c54 <+4>: push %esi
> 0xb72b4c55 <+5>: sub $0xc,%esp
> 0xb72b4c58 <+8>: mov 0x8(%ebp),%ecx
> 0xb72b4c5b <+11>: test %ecx,%ecx
> 0xb72b4c5d <+13>: jne 0xb72b4c68 <__gss_get_mechanism_cred+24>
> 0xb72b4c5f <+15>: add $0xc,%esp
> 0xb72b4c62 <+18>: xor %eax,%eax
> 0xb72b4c64 <+20>: pop %esi
> 0xb72b4c65 <+21>: pop %edi
> 0xb72b4c66 <+22>: pop %ebp
> 0xb72b4c67 <+23>: ret
> 0xb72b4c68 <+24>: mov 0x8(%ebp),%eax
> 0xb72b4c6b <+27>: mov (%eax),%eax
> 0xb72b4c6d <+29>: test %eax,%eax
> 0xb72b4c6f <+31>: mov %eax,-0x10(%ebp)
> 0xb72b4c72 <+34>: jle 0xb72b4c5f <__gss_get_mechanism_cred+15>
> 0xb72b4c74 <+36>: mov 0xc(%ebp),%ecx
> 0xb72b4c77 <+39>: mov 0x8(%ebp),%eax
> 0xb72b4c7a <+42>: mov (%ecx),%edx
> 0xb72b4c7c <+44>: mov 0x4(%eax),%eax
> 0xb72b4c7f <+47>: mov %edx,-0x14(%ebp)
> 0xb72b4c82 <+50>: mov %eax,-0xc(%ebp)
> 0xb72b4c85 <+53>: xor %eax,%eax
> 0xb72b4c87 <+55>: nop
> 0xb72b4c88 <+56>: mov -0xc(%ebp),%edx
> 0xb72b4c8b <+59>: mov -0x14(%ebp),%ecx
> => 0xb72b4c8e <+62>: cmp (%edx,%eax,8),%ecx
> 0xb72b4c91 <+65>: jne 0xb72b4cab <__gss_get_mechanism_cred+91>
> 0xb72b4c93 <+67>: mov 0xc(%ebp),%edx
> 0xb72b4c96 <+70>: mov -0xc(%ebp),%ecx
> 0xb72b4c99 <+73>: mov 0x4(%edx),%esi
> 0xb72b4c9c <+76>: mov -0x14(%ebp),%edx
> ---Type <return> to continue, or q <return> to quit---q
> Quit
> (gdb) info r
> eax 0x0 0
> ecx 0x9 9
> edx 0x0 0
> ebx 0xb72b9710 -1221880048
> esp 0xbfe0b844 0xbfe0b844
> ebp 0xbfe0b858 0xbfe0b858
> esi 0x9 9
> edi 0x9370928 154601768
> eip 0xb72b4c8e 0xb72b4c8e <__gss_get_mechanism_cred+62>
> eflags 0x210246 [ PF ZF IF RF ID ]
> cs 0x73 115
> ss 0x7b 123
> ds 0x7b 123
> es 0x7b 123
> fs 0x0 0
> gs 0x33 51
>
>
> Where you can see the null reference.
>
> This, on the other hand, reminded me a discussion I had in the
> fbsd-stable list, where somebody advised me to use des-cbc-crc
> keytabs, and then experiment with other encryption types. Even
> though I didn't have such issues in fbsd, this might be the case on
> this problem. Maybe linux tries to compare an inexistent mech-type
> (or enc-type, whatever) with one of it's list, and hence it
> segfaults. So, to shed a little more light into the issue:
>
> - the kdc (not the nfs-server) runs fbsd8, heimdal-1.2.1.
> - on the linuxbox when I "kinit -k linuxclient" everything works fine.
> - on the linux client "ktutil -k /etc/krb5.keytab list" shows:
> /etc/krb5.keytab:
>
> Vno Type Principal
> Aliases
> 1 des-cbc-md5 host/linuxclient@EXAMPLE
> 1 des-cbc-md4 host/linuxclient@EXAMPLE
> 1 des-cbc-crc host/linuxclient@EXAMPLE
> 1 aes256-cts-hmac-sha1-96 host/linuxclient@EXAMPLE
> 1 des3-cbc-sha1 host/linuxclient@EXAMPLE
> 1 arcfour-hmac-md5 host/linuxclient@EXAMPLE
>
> So maybe what I have to do next is to create a keytab of des-cbc-crc
> encryption-type only for the client (maybe also for the nfsserver as
> well..) and see how it'll behave...but I will do it tomorrow.
OK, thanks for the investigation so far!
Unfortunately, I don't know who's going to follow up on this soon, so
you may also want to get out the source that was built from, try to
figure exactly what line the NULL dereference is happening on, which
caller passed in the bad data, etc.
--b.
Hi guys,
Kevin:
As stated in my first email, the linux box is running on a 2.6.35
kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),
rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.
Bruce: I would rather find an "easier" (configuration-specific) way to
overcome my problem instead of delving into the code, since I am not a
developer (I am an admin :)). Nevertheless, when I used
google.com/searchcode for _gss_get_mechanism_cred I found that there is
a libgss/g_glue.c that defines this function in the following way (the
site is svn://svn.genunix.org/on/trunk
<http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):
/*
* Glue routine for returning the mechanism-specific credential from a
* external union credential.
*/
gss_cred_id_t
__gss_get_mechanism_cred(union_cred, mech_type)
const gss_union_cred_t union_cred;
const gss_OID mech_type;
{
int i;
if (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
return (GSS_C_NO_CREDENTIAL);
for (i =0; i< union_cred->count; i++) {
if (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
return (union_cred->cred_array[i]);
}
return (GSS_C_NO_CREDENTIAL);
}
where you can see that the guy returns a cred_array[i] element, that
obviously is null, because (most probably) the if statement above did
not match any "compatible" mechanisms during the recursion.
Maybe I am totally wrong, since I haven't checked the code thoroughly,
but I think that the problem is located somewhere in this code segment.
As I stated before, I was rather hoping to avoid this procedure, since I
decided to use the linux distro for my clients due to its
'out-of-the-box' and 'everything-is-supported' benefits as
(slightly-)opposed to fbsd (which I use in all my servers and personal
computers, and "love" as an operating system) can be a bit more
"demanding" when out-of-the-box-general-purpose issues are involved. The
thing is that the client will be used as a general-purpose student pc in
a university lab.
Thanx again for your help,
if anybody could direct me somewhere else I would be delighted to know so!
mamalos
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
Hi Jeff, and thanx for your answer,
>> 111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
>> to register lockd fail the whole mount? I thought it would at worst
>> result in ENOLCK on lock requests?) Do you get better results if you
>> mount with nolock?
>>
> Firewalling issues maybe?
>
The fbsd box runs no firewall whatsoever, and nvsv3 mounts work just
fine if sec=sys is given in the mount command instead of sec=krb5(ip).
On the linux box I haven't setup any iptables whatsoever. My
/etc/hosts.allow on the linuxbox reads:
[root@linuxclient ~]# cat /etc/hosts.allow
#
# /etc/hosts.allow
#
ALL:ALL
So I guess that there is no problem from this side too.
Thanx again,
mamalos
PS. Sorry I haven't answered to your question in my previous email, I
saw your email after sending the previous answer.
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
> Hi all,
>
> I have a FreeBSD nfsv3 server that exports a filesystem with
> sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
> fine. I now try to mount it under linux (archlinux, upgraded today)
> using nfs-utils. Heimdal is configured on the linux-box, kinit -k
> linuxclient works fine. I am also able to kinit to my user
> principals from it. When I try to mount the nvsv3 sec=krb5 share, I
> get the following error:
>
> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
>
Was there supposed to be some error output there? Or did the mount just
hang?
> # tail /var/log/messages
> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
> clients; Exiting!
> Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
> chown /var/lib/nfs to choose different user
> Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
> Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
> clients; Exiting!
> Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
> Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> RPC service (errno 111).
111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
to register lockd fail the whole mount? I thought it would at worst
result in ENOLCK on lock requests?) Do you get better results if you
mount with nolock?
> and it keeps on like this.
>
> My nfs-common.conf reads:
>
> [root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
> # Parameters to be passed to nfs-common (nfs clients & server) init script.
> #
>
> # If you do not set values for the NEED_ options, they will be attempted
> # autodetected; this should be sufficient for most people. Valid
> alternatives
> # for the NEED_ options are "yes" and "no".
>
> # Do you want to start the statd daemon? It is not needed for NFSv4.
> NEED_STATD=""
>
> # Options to pass to rpc.statd.
> # See rpc.statd(8) for more details.
> # N.B. statd normally runs on both client and server, and run-time
> # options should be specified accordingly.
> # STATD_OPTS="-p 32765 -o 32766"
> STATD_OPTS=""
>
> # Options to pass to sm-notify
> # e.g. SMNOTIFY_OPTS="-p 32764"
> SMNOTIFY_OPTS=""
>
> # Do you want to start the idmapd daemon? It is only needed for NFSv4.
> NEED_IDMAPD=""
>
> # Options to pass to rpc.idmapd.
> # See rpc.idmapd(8) for more details.
> IDMAPD_OPTS="-vvv"
>
> # Do you want to start the gssd daemon? It is required for Kerberos mounts.
> NEED_GSSD="yes"
>
> # Options to pass to rpc.gssd.
> # See rpc.gssd(8) for more details.
> GSSD_OPTS="-vvv"
> #RPCGSSDOPTS="-vvv"
> # Where to mount rpc_pipefs filesystem; the default is
> "/var/lib/nfs/rpc_pipefs".
> PIPEFS_MOUNTPOINT=""
>
> # Options used to mount rpc_pipefs filesystem; the default is "defaults".
> PIPEFS_MOUNTOPTS=""
>
> my rpc processes are:
> [root@linuxclient ~]# ps axuww | grep -i rpc
> root 1228 0.0 0.0 0 0 ? S 14:47 0:00 [rpciod/0]
> root 27670 0.0 0.0 6232 908 ? Ss 16:05 0:00
> /usr/bin/rpcbind
> root 27683 0.0 0.1 6332 1236 ? Ss 16:05 0:00
> /usr/sbin/rpc.statd
> root 27699 0.0 0.1 6264 1180 ? Ss 16:05 0:00
> /usr/sbin/rpc.gssd -vvv
> root 27720 0.0 0.0 3776 476 pts/0 S+ 17:01 0:00
> grep -i rpc
>
> And rpcinfo shows:
> root@linuxclient ~]# rpcinfo
> program version netid address service owner
> 100000 4 tcp6 ::.0.111 portmapper superuser
> 100000 3 tcp6 ::.0.111 portmapper superuser
> 100000 4 udp6 ::.0.111 portmapper superuser
> 100000 3 udp6 ::.0.111 portmapper superuser
> 100000 4 udp 0.0.0.0.0.111 portmapper superuser
> 100000 3 udp 0.0.0.0.0.111 portmapper superuser
> 100000 2 udp 0.0.0.0.0.111 portmapper superuser
> 100000 4 local /var/run/rpcbind.sock portmapper superuser
> 100000 3 local /var/run/rpcbind.sock portmapper superuser
> 100024 1 udp 0.0.0.0.228.144 status superuser
> 100024 1 tcp 0.0.0.0.198.8 status superuser
> [root@linuxclient ~]# rpcinfo -s
> program version(s) netid(s) service owner
> 100000 2,3,4 local,udp,udp6,tcp6 portmapper
> superuser
> 100024 1 tcp,udp status
> superuser
>
> whereas on the fbsd box I have:
> [root@fbsdserver ~]# rpcinfo -s
> program version(s) netid(s) service owner
> 100000 2,3,4 local,udp6,tcp6,udp,tcp rpcbind
> superuser
> 100024 1 tcp,udp,tcp6,udp6 status
> superuser
> 100021 4,3,1,0 tcp,udp,tcp6,udp6 nlockmgr
> superuser
> 100003 3,2 tcp6,tcp,udp6,udp nfs
> superuser
> 100005 3,1 tcp,udp,tcp6,udp6 mountd
> superuser
>
>
> The versions I use are:
> rpcbind-0.2.0-2
> nfs-utils-1.2.2-3
>
> And uname -a shows:
>
> [root@linuxclient ~]# uname -a
> Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC
> 2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel
> GNU/Linux
>
> Does linux support RPCSEC_GSS security flavors over nvsv3?
Yes, and this is something I test regularly.
> And if
> so, could somebody direct me on how to establish mounting the remote
> share?
Looks like the server advertises ipv6. I wonder if anyone's tested gss
in that case?
--b.
>
> Thank you all for your time and attention in advance,
>
> regards,
>
> mamalos
>
> --
> George Mamalakis
>
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
>
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
>
> phone number : +30 (2310) 994379
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 20, 2010 at 12:47:31PM +0300, George Mamalakis wrote:
> Hi Bruce and thanx for the reply,
>
> to your questions now:
>
> On 17/09/2010 20:43, J. Bruce Fields wrote:
> >
> >># mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
> >>
> >Was there supposed to be some error output there? Or did the mount just
> >hang?
> >
> The mount command stalls and no output is presented; only the
> messages on /var/log/messages.
> >>Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> >>RPC service (errno 111).
> >111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
> >to register lockd fail the whole mount? I thought it would at worst
> >result in ENOLCK on lock requests?) Do you get better results if you
> >mount with nolock?
> >
> Now this is peculiar. If run mount -o nolock then gssd segfaults.
> This is because it creates a /tmp/krb5cc_machine_REALM ticket, and
> when it access it, it segfaults.
Ouch. Well, that's a problem.
(And have you checked the log files to make sure it's gssd segfaulting
and not some kind of crash in the kernel?)
--b.
> To overcome this issue I deleted
> the temporary file, and ran gssd manually to see its output. So,
> mount returned the following:
>
> [root@linuxclient ~]# mount -t nfs -o sec=krb5,nolock
> fbsdserver:/exports /mnt
> mount.nfs: access denied by server while mounting fbsdserver:/exports
>
> and gssd console output read:
>
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt110
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdserver
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt113
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdserver
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
> handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
> process_krb5_upcall: service is '<null>'
> getting credentials for client with uid 0 for server fbsdclient.ee.auth.gr
> WARNING: Failed to create krb5 context for user with uid 0 for
> server fbsdserver
> doing error downcall
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt112
> destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt111
>
> /var/log/messages shows nothing.
>
>
> I think that's all regarding your questions.
>
> Thanx again,
>
> mamalos.
>
> --
> George Mamalakis
>
> IT Officer
> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
> MSc (Imperial College of London)
>
> Department of Electrical and Computer Engineering
> Faculty of Engineering
> Aristotle University of Thessaloniki
>
> phone number : +30 (2310) 994379
>
Hi Bruce and thanx for the reply,
to your questions now:
On 17/09/2010 20:43, J. Bruce Fields wrote:
>
>> # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
>>
> Was there supposed to be some error output there? Or did the mount just
> hang?
>
The mount command stalls and no output is presented; only the messages
on /var/log/messages.
>> Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
>> RPC service (errno 111).
> 111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
> to register lockd fail the whole mount? I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?
>
Now this is peculiar. If run mount -o nolock then gssd segfaults. This
is because it creates a /tmp/krb5cc_machine_REALM ticket, and when it
access it, it segfaults. To overcome this issue I deleted the temporary
file, and ran gssd manually to see its output. So, mount returned the
following:
[root@linuxclient ~]# mount -t nfs -o sec=krb5,nolock
fbsdserver:/exports /mnt
mount.nfs: access denied by server while mounting fbsdserver:/exports
and gssd console output read:
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt110
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdserver
WARNING: Failed to create krb5 context for user with uid 0 for server
fbsdserver
doing error downcall
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt113
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdserver
WARNING: Failed to create krb5 context for user with uid 0 for server
fbsdserver
doing error downcall
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt112)
process_krb5_upcall: service is '<null>'
getting credentials for client with uid 0 for server fbsdclient.ee.auth.gr
WARNING: Failed to create krb5 context for user with uid 0 for server
fbsdserver
doing error downcall
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt112
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt111
/var/log/messages shows nothing.
I think that's all regarding your questions.
Thanx again,
mamalos.
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379
On Fri, 17 Sep 2010 13:43:28 -0400
"J. Bruce Fields" <[email protected]> wrote:
> On Fri, Sep 17, 2010 at 06:54:35PM +0300, George Mamalakis wrote:
> > Hi all,
> >
> > I have a FreeBSD nfsv3 server that exports a filesystem with
> > sec=krb5. Mounting the share with sec=krb5 under a fbsd client works
> > fine. I now try to mount it under linux (archlinux, upgraded today)
> > using nfs-utils. Heimdal is configured on the linux-box, kinit -k
> > linuxclient works fine. I am also able to kinit to my user
> > principals from it. When I try to mount the nvsv3 sec=krb5 share, I
> > get the following error:
> >
> > # mount -t nfs -o sec=krb5 fbsdserver:/exports /mnt
> >
>
> Was there supposed to be some error output there? Or did the mount just
> hang?
>
> > # tail /var/log/messages
> > Sep 17 16:05:31 linuxclient rpc.statd[27683]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27684]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27684]: Already notifying
> > clients; Exiting!
> > Sep 17 16:05:31 linuxclient rpc.statd[27683]: Running as root.
> > chown /var/lib/nfs to choose different user
> > Sep 17 16:05:31 linuxclient sm-notify[27687]: Version 1.2.2 starting
> > Sep 17 16:05:31 linuxclient sm-notify[27687]: Already notifying
> > clients; Exiting!
> > Sep 17 16:05:53 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:05:53 linuxclient kernel: lockd_up: makesock failed, error=-111
> > Sep 17 16:05:54 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:05:57 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
> > Sep 17 16:06:01 linuxclient kernel: svc: failed to register lockdv1
> > RPC service (errno 111).
>
> 111 is ECONNREFUSED. I'm not sure why that's failing. (Should failure
> to register lockd fail the whole mount? I thought it would at worst
> result in ENOLCK on lock requests?) Do you get better results if you
> mount with nolock?
>
Firewalling issues maybe?
> > and it keeps on like this.
> >
> > My nfs-common.conf reads:
> >
> > [root@linuxclient ~]# cat /etc/conf.d/nfs-common.conf
> > # Parameters to be passed to nfs-common (nfs clients & server) init script.
> > #
> >
> > # If you do not set values for the NEED_ options, they will be attempted
> > # autodetected; this should be sufficient for most people. Valid
> > alternatives
> > # for the NEED_ options are "yes" and "no".
> >
> > # Do you want to start the statd daemon? It is not needed for NFSv4.
> > NEED_STATD=""
> >
> > # Options to pass to rpc.statd.
> > # See rpc.statd(8) for more details.
> > # N.B. statd normally runs on both client and server, and run-time
> > # options should be specified accordingly.
> > # STATD_OPTS="-p 32765 -o 32766"
> > STATD_OPTS=""
> >
> > # Options to pass to sm-notify
> > # e.g. SMNOTIFY_OPTS="-p 32764"
> > SMNOTIFY_OPTS=""
> >
> > # Do you want to start the idmapd daemon? It is only needed for NFSv4.
> > NEED_IDMAPD=""
> >
> > # Options to pass to rpc.idmapd.
> > # See rpc.idmapd(8) for more details.
> > IDMAPD_OPTS="-vvv"
> >
> > # Do you want to start the gssd daemon? It is required for Kerberos mounts.
> > NEED_GSSD="yes"
> >
> > # Options to pass to rpc.gssd.
> > # See rpc.gssd(8) for more details.
> > GSSD_OPTS="-vvv"
> > #RPCGSSDOPTS="-vvv"
> > # Where to mount rpc_pipefs filesystem; the default is
> > "/var/lib/nfs/rpc_pipefs".
> > PIPEFS_MOUNTPOINT=""
> >
> > # Options used to mount rpc_pipefs filesystem; the default is "defaults".
> > PIPEFS_MOUNTOPTS=""
> >
> > my rpc processes are:
> > [root@linuxclient ~]# ps axuww | grep -i rpc
> > root 1228 0.0 0.0 0 0 ? S 14:47 0:00 [rpciod/0]
> > root 27670 0.0 0.0 6232 908 ? Ss 16:05 0:00
> > /usr/bin/rpcbind
> > root 27683 0.0 0.1 6332 1236 ? Ss 16:05 0:00
> > /usr/sbin/rpc.statd
> > root 27699 0.0 0.1 6264 1180 ? Ss 16:05 0:00
> > /usr/sbin/rpc.gssd -vvv
> > root 27720 0.0 0.0 3776 476 pts/0 S+ 17:01 0:00
> > grep -i rpc
> >
> > And rpcinfo shows:
> > root@linuxclient ~]# rpcinfo
> > program version netid address service owner
> > 100000 4 tcp6 ::.0.111 portmapper superuser
> > 100000 3 tcp6 ::.0.111 portmapper superuser
> > 100000 4 udp6 ::.0.111 portmapper superuser
> > 100000 3 udp6 ::.0.111 portmapper superuser
> > 100000 4 udp 0.0.0.0.0.111 portmapper superuser
> > 100000 3 udp 0.0.0.0.0.111 portmapper superuser
> > 100000 2 udp 0.0.0.0.0.111 portmapper superuser
> > 100000 4 local /var/run/rpcbind.sock portmapper superuser
> > 100000 3 local /var/run/rpcbind.sock portmapper superuser
> > 100024 1 udp 0.0.0.0.228.144 status superuser
> > 100024 1 tcp 0.0.0.0.198.8 status superuser
> > [root@linuxclient ~]# rpcinfo -s
> > program version(s) netid(s) service owner
> > 100000 2,3,4 local,udp,udp6,tcp6 portmapper
> > superuser
> > 100024 1 tcp,udp status
> > superuser
> >
> > whereas on the fbsd box I have:
> > [root@fbsdserver ~]# rpcinfo -s
> > program version(s) netid(s) service owner
> > 100000 2,3,4 local,udp6,tcp6,udp,tcp rpcbind
> > superuser
> > 100024 1 tcp,udp,tcp6,udp6 status
> > superuser
> > 100021 4,3,1,0 tcp,udp,tcp6,udp6 nlockmgr
> > superuser
> > 100003 3,2 tcp6,tcp,udp6,udp nfs
> > superuser
> > 100005 3,1 tcp,udp,tcp6,udp6 mountd
> > superuser
> >
> >
> > The versions I use are:
> > rpcbind-0.2.0-2
> > nfs-utils-1.2.2-3
> >
> > And uname -a shows:
> >
> > [root@linuxclient ~]# uname -a
> > Linux linuxclient 2.6.35-ARCH #1 SMP PREEMPT Fri Aug 27 16:22:18 UTC
> > 2010 i686 Intel(R) Xeon(R) CPU E5310 @ 1.60GHz GenuineIntel
> > GNU/Linux
> >
> > Does linux support RPCSEC_GSS security flavors over nvsv3?
>
> Yes, and this is something I test regularly.
>
> > And if
> > so, could somebody direct me on how to establish mounting the remote
> > share?
>
> Looks like the server advertises ipv6. I wonder if anyone's tested gss
> in that case?
>
Not in a while, but I did test it when I did the IPv6 changes for
rpc.gssd. It should work fine, AFAIK. Based on the logs above,
RPCSEC_GSS is probably a red herring. The real problem seems to be that
there are problems communicating with the portmapper.
--
Jeff Layton <[email protected]>
On 21/09/2010 16:51, J. Bruce Fields wrote:
> On Tue, Sep 21, 2010 at 11:36:18AM +0300, George Mamalakis wrote:
>> Hi guys,
>>
>> Kevin:
>> As stated in my first email, the linux box is running on a 2.6.35
>> kernel, heimdal 1.3.3-1 (the version was not mentioned in my email),
> As Kevin says, it would be worth trying MIT kerberos instead of heimdal
> if you can, as that is generally better tested.
>
> --b.
>
>> rpcbind-0.2.0-2, nfs-utils-1.2.2-3, and the distro is arch-linux.
>>
>> Bruce: I would rather find an "easier" (configuration-specific) way
>> to overcome my problem instead of delving into the code, since I am
>> not a developer (I am an admin :)). Nevertheless, when I used
>> google.com/searchcode for _gss_get_mechanism_cred I found that there
>> is a libgss/g_glue.c that defines this function in the following way
>> (the site is svn://svn.genunix.org/on/trunk<http://google.com/codesearch/p?hl=en#m0BoFGhUuEY/&q=_gss_get_mechanism_cred>):
>>
>> /*
>> * Glue routine for returning the mechanism-specific credential from a
>> * external union credential.
>> */
>> gss_cred_id_t
>> __gss_get_mechanism_cred(union_cred, mech_type)
>> const gss_union_cred_t union_cred;
>> const gss_OID mech_type;
>> {
>> int i;
>>
>> if (union_cred == (gss_union_cred_t)GSS_C_NO_CREDENTIAL)
>> return (GSS_C_NO_CREDENTIAL);
>>
>> for (i =0; i< union_cred->count; i++) {
>> if (g_OID_equal(mech_type,&union_cred->mechs_array[i]))
>> return (union_cred->cred_array[i]);
>> }
>> return (GSS_C_NO_CREDENTIAL);
>> }
>>
>>
>> where you can see that the guy returns a cred_array[i] element, that
>> obviously is null, because (most probably) the if statement above
>> did not match any "compatible" mechanisms during the recursion.
>>
>> Maybe I am totally wrong, since I haven't checked the code
>> thoroughly, but I think that the problem is located somewhere in
>> this code segment.
>>
>> As I stated before, I was rather hoping to avoid this procedure,
>> since I decided to use the linux distro for my clients due to its
>> 'out-of-the-box' and 'everything-is-supported' benefits as
>> (slightly-)opposed to fbsd (which I use in all my servers and
>> personal computers, and "love" as an operating system) can be a bit
>> more "demanding" when out-of-the-box-general-purpose issues are
>> involved. The thing is that the client will be used as a
>> general-purpose student pc in a university lab.
>>
>> Thanx again for your help,
>>
>> if anybody could direct me somewhere else I would be delighted to know so!
>>
>> mamalos
>>
>> --
>> George Mamalakis
>>
>> IT Officer
>> Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
>> MSc (Imperial College of London)
>>
>> Department of Electrical and Computer Engineering
>> Faculty of Engineering
>> Aristotle University of Thessaloniki
>>
>> phone number : +30 (2310) 994379
>>
Ok Bruce and Kevin,
I'll try and do that. I only hope that MIT-client is compatible with
Heimdal-server (if I manage to install it on archlinux through AUR...:)).
--
George Mamalakis
IT Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
Aristotle University of Thessaloniki
phone number : +30 (2310) 994379