I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names.
In this scenario, svcgssd can be called with "-n" so that it acquires creds at context creation. After running this way I found svcgssd opens a file to the kerberos replay cache for every context/cred, eventually reaching ulimit. For a busy cluster with many different client-user pairs that becomes a problem. I am lost in the gss_krb5 code, but suspect that the kerberos code leaks credentials in this configuration.
Ondrej Palkovsky submitted a patch to specify multiple identities and acquire creds up-front using multiples of "-h": http://marc.info/?l=linux-nfsv4&m=123685185324902&w=2
I've updated that work to be current to nfs-utils-1.2.3 which solves our immediate problem, and it works well -- but running svcgssd with '-n' is still going to leak file handles to the replay cache. What's the best way to fix this? Can the created-on-the-fly cred can be re-used for subsequent contexts?
Ben
On Wed, Jul 13, 2011 at 12:03:03PM -0400, Benjamin Coddington wrote:
> I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names.
>
> In this scenario, svcgssd can be called with "-n" so that it acquires creds at context creation. After running this way I found svcgssd opens a file to the kerberos replay cache for every context/cred, eventually reaching ulimit. For a busy cluster with many different client-user pairs that becomes a problem. I am lost in the gss_krb5 code, but suspect that the kerberos code leaks credentials in this configuration.
>
> Ondrej Palkovsky submitted a patch to specify multiple identities and acquire creds up-front using multiples of "-h": http://marc.info/?l=linux-nfsv4&m=123685185324902&w=2
>
> I've updated that work to be current to nfs-utils-1.2.3 which solves our immediate problem, and it works well -- but running svcgssd with '-n' is still going to leak file handles to the replay cache. What's the best way to fix this? Can the created-on-the-fly cred can be re-used for subsequent contexts?
Sounds like a likely kerberos bug as well--may be use asking the
kerberos folks?
--b.
On Jul 13, 2011, at 1:35 PM, J. Bruce Fields wrote:
> On Wed, Jul 13, 2011 at 12:03:03PM -0400, Benjamin Coddington wrote:
>> I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names.
>>
>> In this scenario, svcgssd can be called with "-n" so that it acquires creds at context creation. After running this way I found svcgssd opens a file to the kerberos replay cache for every context/cred, eventually reaching ulimit. For a busy cluster with many different client-user pairs that becomes a problem. I am lost in the gss_krb5 code, but suspect that the kerberos code leaks credentials in this configuration.
>>
>> Ondrej Palkovsky submitted a patch to specify multiple identities and acquire creds up-front using multiples of "-h": http://marc.info/?l=linux-nfsv4&m=123685185324902&w=2
>>
>> I've updated that work to be current to nfs-utils-1.2.3 which solves our immediate problem, and it works well -- but running svcgssd with '-n' is still going to leak file handles to the replay cache. What's the best way to fix this? Can the created-on-the-fly cred can be re-used for subsequent contexts?
>
> Sounds like a likely kerberos bug as well--may be use asking the
> kerberos folks?
>
> --b.
It is a kerberos bug where the context is not cleaned up in gss_export_lucid_sec_context(), so svcgssd leaks a bit. Here's a reference to the kerberos problem: http://marc.info/?t=131068390400045&r=1&w=2
Thank you for the suggestion.
Ben
On 07/13/2011 12:03 PM, Benjamin Coddington wrote:
> I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names.
>
> In this scenario, svcgssd can be called with "-n" so that it acquires creds at context creation. After running this way I found svcgssd opens a file to the kerberos replay cache for every context/cred, eventually reaching ulimit. For a busy cluster with many different client-user pairs that becomes a problem. I am lost in the gss_krb5 code, but suspect that the kerberos code leaks credentials in this configuration.
>
> Ondrej Palkovsky submitted a patch to specify multiple identities and acquire creds up-front using multiples of "-h": http://marc.info/?l=linux-nfsv4&m=123685185324902&w=2
>
> I've updated that work to be current to nfs-utils-1.2.3 which solves our immediate problem, and it works well
Please go a head are repost the updated patch... Also it good if there was a man page
updated was well...
tia,
steved.
On Wed, Jul 20, 2011 at 08:56:47AM -0400, Benjamin Coddington wrote:
> On Jul 13, 2011, at 1:35 PM, J. Bruce Fields wrote:
> > On Wed, Jul 13, 2011 at 12:03:03PM -0400, Benjamin Coddington wrote:
> >> I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names.
> >>
> >> In this scenario, svcgssd can be called with "-n" so that it acquires creds at context creation. After running this way I found svcgssd opens a file to the kerberos replay cache for every context/cred, eventually reaching ulimit. For a busy cluster with many different client-user pairs that becomes a problem. I am lost in the gss_krb5 code, but suspect that the kerberos code leaks credentials in this configuration.
> >>
> >> Ondrej Palkovsky submitted a patch to specify multiple identities and acquire creds up-front using multiples of "-h": http://marc.info/?l=linux-nfsv4&m=123685185324902&w=2
> >>
> >> I've updated that work to be current to nfs-utils-1.2.3 which solves our immediate problem, and it works well -- but running svcgssd with '-n' is still going to leak file handles to the replay cache. What's the best way to fix this? Can the created-on-the-fly cred can be re-used for subsequent contexts?
> >
> > Sounds like a likely kerberos bug as well--may be use asking the
> > kerberos folks?
> >
> > --b.
>
> It is a kerberos bug where the context is not cleaned up in gss_export_lucid_sec_context(), so svcgssd leaks a bit. Here's a reference to the kerberos problem: http://marc.info/?t=131068390400045&r=1&w=2
Good, thanks for following up.
--b.