I've just set up a Linux KDC with a Linux NFS server (Fedora 13 with the latest updates).
rpc.svcgssd fails to start on the NFS server.
ERROR: GSS-API: error in gss_acquire_cred(): Unspecified GSS failure. Minor code may provide more information - Key table entry not found
unable to obtain root (machine) credentials
do you have a keytab entry for nfs/<your.host>@<YOUR.REALM> in /etc/krb5.keytab?
I do have an entry for nfs/<your.host>@<YOUR.REALM> in /etc/krb5.keytab. The problem is that /etc/hosts looks like this:
192.168.1.58 your.host your # Added by NetworkManager
127.0.0.1 localhost.localdomain localhost
::1 your.host your localhost6.localdomain6 localhost6
Removing "your.host your" from the "::1" entry makes this problem go away -- rpc.svcgssd starts up as expected.
Now I reboot, and NetworkManager happily adds "your.host your" back to the "::1" entry, and rpc.svcgssd fails again. I haven't tried this, but I suspect if the ::1 entry weren't there, NM would add "your.host.net your" to the IPv4 loopback entry, and we'd have the same problem.
At a glance, it looks like the local hostname is determined in a library, and not in rpc.svcgssd. This really needs to be more robust.
I see the "-p principal" option in the latest nfs-utils, but it doesn't seem to be supported in Fedora 13's rpc.svcgssd. Is this the workaround?
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Hi Chuck,
We have been disabling NM for years now because of this. As far as I can
see from it's behaviour, it sets the hostname all by itself, during
startup (but I could be wrong here, didn't look in the source code).
V.
Chuck Lever schreef:
> ERROR: GSS-API: error in gss_acquire_cred(): Unspecified GSS failure. Minor code may provide more information - Key table entry not found
> unable to obtain root (machine) credentials
> do you have a keytab entry for nfs/<your.host>@<YOUR.REALM> in /etc/krb5.keytab?
[...]
> Removing "your.host your" from the "::1" entry makes this problem go away -- rpc.svcgssd starts up as expected.
>
> Now I reboot, and NetworkManager happily adds "your.host your" back to the "::1" entry, and rpc.svcgssd fails again. I haven't tried this, but I suspect if the ::1 entry weren't there, NM would add "your.host.net your" to the IPv4 loopback entry, and we'd have the same problem.
--
http://www.openoffice.nl/ Open Office - Linux Office Solutions
Valentijn Sessink [email protected] +31(0)20-4214059
On Nov 17, 2010, at 10:54 AM, Kevin Coffman wrote:
> On Wed, Nov 17, 2010 at 10:30 AM, Chuck Lever <[email protected]> wrote:
>> Hey-
>>
>> On Nov 17, 2010, at 10:18 AM, Steve Dickson wrote:
>>
>>> Sorry for the delayed response... I had my
>>> head down for last couple of days...
>>>
>>> On 11/16/2010 04:42 PM, Chuck Lever wrote:
>>>>
>>>> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>>>>
>>>>> Chuck Lever wrote:
>>>>>
>>>>> Before we go too far down the NM path of no return, I was under the
>>>>> impression that some applications require the host's name on the localhost
>>>>> entries in /etc/hosts. That's why NM puts it there.
>>>>>
>>>>> There's nothing invalid about having a hostname on the localhost entries
>>>>> in /etc/hosts, is there?
>>>>>
>>>>> So I wonder if removing NM is really the solution here.
>>>>>
>>>>> No, it's not. I just like to complain about NM.
>>>>>
>>>>> The original problem was that rpc.svcgssd couldn't figure out the correct
>>>>> kerberos realm. The fix in this particular case, I think, is to set the
>>>>> realm explicitly in /etc/idmapd.conf.
>>>>
>>>> It's having trouble determining the NFS server's hostname. It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>>>>
>>>> I don't know if realm self-discovery is an issue too.
>>> I think the problem is a reverse lookup is done on hostname that
>>> is found in the /etc/krb5.keytab. Instead of the FQDN being
>>> returned, localhost is returned because the FQDN was added to
>>> the localhost line in /etc/hosts.
>>>
>>> Actually I didn't realize it was NM doing that... I thought
>>> it was the installer...
>>
>> No matter who does it, I think there are applications
>> (gdm? rusty recollection) that require this network configuration
>> in /etc/hosts, so our best bet IMO is to fix rpc.svcgssd, or more
>> likely the gss library it depends on, to get it right in this situation.
>> If we all agree this is a bug (and sounds like we do) then I can
>> create a bug report on bugzilla.linux-nfs.org, as a starting point.
>
> Hi Chuck and Steve,
> This issue affects gss authentication in sshd as well. I believe this
> is all the way down in the Kerberos code, which has been this way for
> years. I'm not sure what needs to be changed to "get it right".
I was afraid of that. Do you know if this has this ever been brought up with the upstream Kerberos maintainers?
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Chuck Lever wrote:
Before we go too far down the NM path of no return, I was under the
impression that some applications require the host's name on the localhost
entries in /etc/hosts. That's why NM puts it there.
There's nothing invalid about having a hostname on the localhost entries
in /etc/hosts, is there?
So I wonder if removing NM is really the solution here.
No, it's not. I just like to complain about NM.
The original problem was that rpc.svcgssd couldn't figure out the correct
kerberos realm. The fix in this particular case, I think, is to set the
realm explicitly in /etc/idmapd.conf.
But a more general problem is that if you don't set a realm in
/etc/idmapd.conf, the fallback is to whatever is returned by gethostname().
Shouldn't the fallback be to what is in krb5.conf?
In general, I think it's a mistake to assume that a host's security realm is
the same as its dns domain, especially given host mobility, the lack of
security in dns, and the existence of other methods (krb5.conf) to determine
the security realm.
On Wed, Nov 17, 2010 at 11:05 AM, Chuck Lever <[email protected]> wrote:
>
> On Nov 17, 2010, at 10:54 AM, Kevin Coffman wrote:
>
>> On Wed, Nov 17, 2010 at 10:30 AM, Chuck Lever <[email protected]> wrote:
>>> Hey-
>>>
>>> On Nov 17, 2010, at 10:18 AM, Steve Dickson wrote:
>>>
>>>> Sorry for the delayed response... I had my
>>>> head down for last couple of days...
>>>>
>>>> On 11/16/2010 04:42 PM, Chuck Lever wrote:
>>>>>
>>>>> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>>>>>
>>>>>> Chuck Lever wrote:
>>>>>>
>>>>>> Before we go too far down the NM path of no return, I was under the
>>>>>> impression that some applications require the host's name on the localhost
>>>>>> entries in /etc/hosts. ?That's why NM puts it there.
>>>>>>
>>>>>> There's nothing invalid about having a hostname on the localhost entries
>>>>>> in /etc/hosts, is there?
>>>>>>
>>>>>> So I wonder if removing NM is really the solution here.
>>>>>>
>>>>>> No, it's not. ?I just like to complain about NM.
>>>>>>
>>>>>> The original problem was that rpc.svcgssd couldn't figure out the correct
>>>>>> kerberos realm. ?The fix in this particular case, I think, is to set the
>>>>>> realm explicitly in /etc/idmapd.conf.
>>>>>
>>>>> It's having trouble determining the NFS server's hostname. ?It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>>>>>
>>>>> I don't know if realm self-discovery is an issue too.
>>>> I think the problem is a reverse lookup is done on hostname that
>>>> is found in the /etc/krb5.keytab. Instead of the FQDN being
>>>> returned, localhost is returned because the FQDN was added to
>>>> the localhost line in /etc/hosts.
>>>>
>>>> Actually I didn't realize it was NM doing that... I thought
>>>> it was the installer...
>>>
>>> No matter who does it, I think there are applications
>>> (gdm? ?rusty recollection) that require this network configuration
>>> in /etc/hosts, so our best bet IMO is to fix rpc.svcgssd, or more
>>> likely the gss library it depends on, to get it right in this situation.
>>> If we all agree this is a bug (and sounds like we do) then I can
>>> create a bug report on bugzilla.linux-nfs.org, as a starting point.
>>
>> Hi Chuck and Steve,
>> This issue affects gss authentication in sshd as well. ?I believe this
>> is all the way down in the Kerberos code, which has been this way for
>> years. ?I'm not sure what needs to be changed to "get it right".
>
> I was afraid of that. ?Do you know if this has this ever been brought
> up with the upstream Kerberos maintainers?
Not that I am aware of. Like Valentin, I believe it to be a
NetworkManager bug. (The Kerberos code works fine on all other Unix
platforms.)
K.C.
Sorry for the delayed response... I had my
head down for last couple of days...
On 11/16/2010 04:42 PM, Chuck Lever wrote:
>
> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>
>> Chuck Lever wrote:
>>
>> Before we go too far down the NM path of no return, I was under the
>> impression that some applications require the host's name on the localhost
>> entries in /etc/hosts. That's why NM puts it there.
>>
>> There's nothing invalid about having a hostname on the localhost entries
>> in /etc/hosts, is there?
>>
>> So I wonder if removing NM is really the solution here.
>>
>> No, it's not. I just like to complain about NM.
>>
>> The original problem was that rpc.svcgssd couldn't figure out the correct
>> kerberos realm. The fix in this particular case, I think, is to set the
>> realm explicitly in /etc/idmapd.conf.
>
> It's having trouble determining the NFS server's hostname. It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>
> I don't know if realm self-discovery is an issue too.
I think the problem is a reverse lookup is done on hostname that
is found in the /etc/krb5.keytab. Instead of the FQDN being
returned, localhost is returned because the FQDN was added to
the localhost line in /etc/hosts.
Actually I didn't realize it was NM doing that... I thought
it was the installer...
steved.
Kevin Coffman schreef:
> This issue affects gss authentication in sshd as well.
"The issue" being that "gethostname()" returns nonsense after
NetworkManager configures the nonsense with "sethostname()".
I looked at the source code of Networkmanager, briefly, and it does look
like they do their utter best to make the hostname anything that you did
not want it to be, they first look in "system-settings" - but I can't
see what that is, then they use "automatic hostname from DHCP, VPN
etc.", only then to fall back to "the original hostname when NM
started". But the "set_system_hostname()" function has
"FALLBACK_HOSTNAME" (being localhost.localdomain) all over the place, so
a single call to set_system_hostname with the wrong settings will mess
things up for the rest of the uptime of the machine.
> I believe this
> is all the way down in the Kerberos code, which has been this way for
> years. I'm not sure what needs to be changed to "get it right".
If something messes with "sethostname()", you can be sure that
"gethostname()" will return the mess.
Apart from that, IIRC, there was some discussion about the "Domain"
clause in the idmapd.conf and if that should come from the KRB domain
settings; problem with that, again IIRC, is, that then the idmapper
would become dependent on kerberos libraries.
If you'll ask me, I don't think there's such a big problem. The only
problem is that NetworkManager shouldn't mess with the hostname.
(But I could be wrong here).
Best regards,
Valentijn
Quoting myself: ...
Op 16-11-10 16:58, Valentijn Sessink schreef:
> We have been disabling NM for years now because of this.
>> Now I reboot, and NetworkManager happily adds "your.host your" back to the "::1" entry, and rpc.svcgssd fails again.
... here's the long standing bug report for Ubuntu:
https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/8980
(Been there since 2004). Nobody uses hostnames anymore, I guess ;)
V.
On Tue, Nov 16, 2010 at 03:54:36PM -0500, Jim Rees wrote:
> Chuck Lever wrote:
>
> Before we go too far down the NM path of no return, I was under the
> impression that some applications require the host's name on the localhost
> entries in /etc/hosts. That's why NM puts it there.
>
> There's nothing invalid about having a hostname on the localhost entries
> in /etc/hosts, is there?
>
> So I wonder if removing NM is really the solution here.
>
> No, it's not. I just like to complain about NM.
>
> The original problem was that rpc.svcgssd couldn't figure out the correct
> kerberos realm. The fix in this particular case, I think, is to set the
> realm explicitly in /etc/idmapd.conf.
>
> But a more general problem is that if you don't set a realm in
> /etc/idmapd.conf, the fallback is to whatever is returned by gethostname().
> Shouldn't the fallback be to what is in krb5.conf?
>
> In general, I think it's a mistake to assume that a host's security realm is
> the same as its dns domain, especially given host mobility, the lack of
> security in dns, and the existence of other methods (krb5.conf) to determine
> the security realm.
Probably so. Seems like hostname problems are one of the most frequent
stumbling blocks for nfs/krb5 setup, too.
But fixing this probably needs a volunteer.
--b.
On Nov 17, 2010, at 11:26 AM, Kevin Coffman wrote:
> On Wed, Nov 17, 2010 at 11:05 AM, Chuck Lever <[email protected]> wrote:
>>
>> On Nov 17, 2010, at 10:54 AM, Kevin Coffman wrote:
>>
>>> On Wed, Nov 17, 2010 at 10:30 AM, Chuck Lever <[email protected]> wrote:
>>>> Hey-
>>>>
>>>> On Nov 17, 2010, at 10:18 AM, Steve Dickson wrote:
>>>>
>>>>> Sorry for the delayed response... I had my
>>>>> head down for last couple of days...
>>>>>
>>>>> On 11/16/2010 04:42 PM, Chuck Lever wrote:
>>>>>>
>>>>>> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>>>>>>
>>>>>>> Chuck Lever wrote:
>>>>>>>
>>>>>>> Before we go too far down the NM path of no return, I was under the
>>>>>>> impression that some applications require the host's name on the localhost
>>>>>>> entries in /etc/hosts. That's why NM puts it there.
>>>>>>>
>>>>>>> There's nothing invalid about having a hostname on the localhost entries
>>>>>>> in /etc/hosts, is there?
>>>>>>>
>>>>>>> So I wonder if removing NM is really the solution here.
>>>>>>>
>>>>>>> No, it's not. I just like to complain about NM.
>>>>>>>
>>>>>>> The original problem was that rpc.svcgssd couldn't figure out the correct
>>>>>>> kerberos realm. The fix in this particular case, I think, is to set the
>>>>>>> realm explicitly in /etc/idmapd.conf.
>>>>>>
>>>>>> It's having trouble determining the NFS server's hostname. It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>>>>>>
>>>>>> I don't know if realm self-discovery is an issue too.
>>>>> I think the problem is a reverse lookup is done on hostname that
>>>>> is found in the /etc/krb5.keytab. Instead of the FQDN being
>>>>> returned, localhost is returned because the FQDN was added to
>>>>> the localhost line in /etc/hosts.
>>>>>
>>>>> Actually I didn't realize it was NM doing that... I thought
>>>>> it was the installer...
>>>>
>>>> No matter who does it, I think there are applications
>>>> (gdm? rusty recollection) that require this network configuration
>>>> in /etc/hosts, so our best bet IMO is to fix rpc.svcgssd, or more
>>>> likely the gss library it depends on, to get it right in this situation.
>>>> If we all agree this is a bug (and sounds like we do) then I can
>>>> create a bug report on bugzilla.linux-nfs.org, as a starting point.
>>>
>>> Hi Chuck and Steve,
>>> This issue affects gss authentication in sshd as well. I believe this
>>> is all the way down in the Kerberos code, which has been this way for
>>> years. I'm not sure what needs to be changed to "get it right".
>>
>> I was afraid of that. Do you know if this has this ever been brought
>> up with the upstream Kerberos maintainers?
>
> Not that I am aware of. Like Valentin, I believe it to be a
> NetworkManager bug. (The Kerberos code works fine on all other Unix
> platforms.)
The hostname is added to localhost entries in /etc/hosts because some applications require at least one entry with the local host's hostname in /etc/hosts. When a system comes up with all of its network interfaces offline and without any IP addresses assigned, what else can be done?
One way to address this via NetworkManager might be to have the hostname removed from the localhost entries when a real IP address for the host is added to /etc/hosts, and then added back to the localhost entries when all network interfaces are disabled.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Op 17-11-10 18:51, Chuck Lever schreef:
> The hostname is added to localhost entries in /etc/hosts because some
> applications require at least one entry with the local host's
> hostname in /etc/hosts. When a system comes up with all of its
> network interfaces offline and without any IP addresses assigned,
> what else can be done?
Read /etc/hostname?
Adding "localhost" entries to /etc/hosts does nothing for applications
that require an entry with the hostname in /etc/hosts, *unless* the
hostname is "localhost". Now as Networkmanager (apparently) sets the
hostname to "localhost", I can see why Networkmanager wants to add
"localhost" to /etc/hosts afterwards. But hey, we might as well set the
hostname to everyone.loves.networkmanager, then add *that* to /etc/hosts
and run into problems.
Please note, that /etc/hosts has nothing to do with the hostname, it's
only a way to statically (and locally) add things to the resolver.
[...]
> for the host is added to /etc/hosts, and then added back to the
> localhost entries when all network interfaces are disabled.
I don't think adding or not adding them to /etc/hosts makes any
difference. I'd guess that calling sethostname() causes problems.
V.
On Nov 16, 2010, at 3:17 PM, Jim Rees wrote:
> Valentijn Sessink wrote:
>
> Quoting myself: ...
> Op 16-11-10 16:58, Valentijn Sessink schreef:
>> We have been disabling NM for years now because of this.
>>> Now I reboot, and NetworkManager happily adds "your.host your" back to the "::1" entry, and rpc.svcgssd fails again.
>
> ... here's the long standing bug report for Ubuntu:
> https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/8980
>
> (Been there since 2004). Nobody uses hostnames anymore, I guess ;)
>
> It corrupts your resolv.conf too. You should always remove Network Manager
> except on mobile hosts that really need it.
Before we go too far down the NM path of no return, I was under the impression that some applications require the host's name on the localhost entries in /etc/hosts. That's why NM puts it there.
There's nothing invalid about having a hostname on the localhost entries in /etc/hosts, is there?
So I wonder if removing NM is really the solution here.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Valentijn Sessink wrote:
Quoting myself: ...
Op 16-11-10 16:58, Valentijn Sessink schreef:
>We have been disabling NM for years now because of this.
>>Now I reboot, and NetworkManager happily adds "your.host your" back to the "::1" entry, and rpc.svcgssd fails again.
... here's the long standing bug report for Ubuntu:
https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/8980
(Been there since 2004). Nobody uses hostnames anymore, I guess ;)
It corrupts your resolv.conf too. You should always remove Network Manager
except on mobile hosts that really need it.
On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
> Chuck Lever wrote:
>
> Before we go too far down the NM path of no return, I was under the
> impression that some applications require the host's name on the localhost
> entries in /etc/hosts. That's why NM puts it there.
>
> There's nothing invalid about having a hostname on the localhost entries
> in /etc/hosts, is there?
>
> So I wonder if removing NM is really the solution here.
>
> No, it's not. I just like to complain about NM.
>
> The original problem was that rpc.svcgssd couldn't figure out the correct
> kerberos realm. The fix in this particular case, I think, is to set the
> realm explicitly in /etc/idmapd.conf.
It's having trouble determining the NFS server's hostname. It needs to find the right nfs/your.host key in /etc/krb5.keytab.
I don't know if realm self-discovery is an issue too.
> But a more general problem is that if you don't set a realm in
> /etc/idmapd.conf, the fallback is to whatever is returned by gethostname().
> Shouldn't the fallback be to what is in krb5.conf?
> In general, I think it's a mistake to assume that a host's security realm is
> the same as its dns domain, especially given host mobility, the lack of
> security in dns, and the existence of other methods (krb5.conf) to determine
> the security realm.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Hey-
On Nov 17, 2010, at 10:18 AM, Steve Dickson wrote:
> Sorry for the delayed response... I had my
> head down for last couple of days...
>
> On 11/16/2010 04:42 PM, Chuck Lever wrote:
>>
>> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>>
>>> Chuck Lever wrote:
>>>
>>> Before we go too far down the NM path of no return, I was under the
>>> impression that some applications require the host's name on the localhost
>>> entries in /etc/hosts. That's why NM puts it there.
>>>
>>> There's nothing invalid about having a hostname on the localhost entries
>>> in /etc/hosts, is there?
>>>
>>> So I wonder if removing NM is really the solution here.
>>>
>>> No, it's not. I just like to complain about NM.
>>>
>>> The original problem was that rpc.svcgssd couldn't figure out the correct
>>> kerberos realm. The fix in this particular case, I think, is to set the
>>> realm explicitly in /etc/idmapd.conf.
>>
>> It's having trouble determining the NFS server's hostname. It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>>
>> I don't know if realm self-discovery is an issue too.
> I think the problem is a reverse lookup is done on hostname that
> is found in the /etc/krb5.keytab. Instead of the FQDN being
> returned, localhost is returned because the FQDN was added to
> the localhost line in /etc/hosts.
>
> Actually I didn't realize it was NM doing that... I thought
> it was the installer...
No matter who does it, I think there are applications (gdm? rusty recollection) that require this network configuration in /etc/hosts, so our best bet IMO is to fix rpc.svcgssd, or more likely the gss library it depends on, to get it right in this situation. If we all agree this is a bug (and sounds like we do) then I can create a bug report on bugzilla.linux-nfs.org, as a starting point.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
On Wed, Nov 17, 2010 at 10:30 AM, Chuck Lever <[email protected]> wrote:
> Hey-
>
> On Nov 17, 2010, at 10:18 AM, Steve Dickson wrote:
>
>> Sorry for the delayed response... I had my
>> head down for last couple of days...
>>
>> On 11/16/2010 04:42 PM, Chuck Lever wrote:
>>>
>>> On Nov 16, 2010, at 3:54 PM, Jim Rees wrote:
>>>
>>>> Chuck Lever wrote:
>>>>
>>>> Before we go too far down the NM path of no return, I was under the
>>>> impression that some applications require the host's name on the localhost
>>>> entries in /etc/hosts. ?That's why NM puts it there.
>>>>
>>>> There's nothing invalid about having a hostname on the localhost entries
>>>> in /etc/hosts, is there?
>>>>
>>>> So I wonder if removing NM is really the solution here.
>>>>
>>>> No, it's not. ?I just like to complain about NM.
>>>>
>>>> The original problem was that rpc.svcgssd couldn't figure out the correct
>>>> kerberos realm. ?The fix in this particular case, I think, is to set the
>>>> realm explicitly in /etc/idmapd.conf.
>>>
>>> It's having trouble determining the NFS server's hostname. ?It needs to find the right nfs/your.host key in /etc/krb5.keytab.
>>>
>>> I don't know if realm self-discovery is an issue too.
>> I think the problem is a reverse lookup is done on hostname that
>> is found in the /etc/krb5.keytab. Instead of the FQDN being
>> returned, localhost is returned because the FQDN was added to
>> the localhost line in /etc/hosts.
>>
>> Actually I didn't realize it was NM doing that... I thought
>> it was the installer...
>
> No matter who does it, I think there are applications
> (gdm? ?rusty recollection) that require this network configuration
> in /etc/hosts, so our best bet IMO is to fix rpc.svcgssd, or more
> likely the gss library it depends on, to get it right in this situation.
> If we all agree this is a bug (and sounds like we do) then I can
> create a bug report on bugzilla.linux-nfs.org, as a starting point.
Hi Chuck and Steve,
This issue affects gss authentication in sshd as well. I believe this
is all the way down in the Kerberos code, which has been this way for
years. I'm not sure what needs to be changed to "get it right".
K.C.