Upstream fedfs-utils has not been under active development for
two years or more, and there is a scant user base. I'd like to
propose making 0.10 the final major release of fedfs-utils.
The plan:
- Since 0.10 is in at least one major enterprise distribution,
I will remain available to integrate security fixes and make
new minor releases in the 0.10 line, as needed, for one to
two more years.
- Retire and remove fedfs-utils from upstream mirror distros
such as Fedora rawhide.
- Transfer utilities such as nfsref into nfs-utils, with
support for FedFS junctions removed.
- Announcements of the change in status will be made on
fedfs-utils-announce and on the wiki.linux-nfs.org site.
Comments welcome!
--
Chuck Lever
> On Jun 2, 2017, at 11:01 AM, Chuck Lever <[email protected]> wrote:
>
> Upstream fedfs-utils has not been under active development for
> two years or more, and there is a scant user base. I'd like to
> propose making 0.10 the final major release of fedfs-utils.
>
> The plan:
>
> - Since 0.10 is in at least one major enterprise distribution,
> I will remain available to integrate security fixes and make
> new minor releases in the 0.10 line, as needed, for one to
> two more years.
>
> - Retire and remove fedfs-utils from upstream mirror distros
> such as Fedora rawhide.
>
> - Transfer utilities such as nfsref into nfs-utils, with
> support for FedFS junctions removed.
>
> - Announcements of the change in status will be made on
> fedfs-utils-announce and on the wiki.linux-nfs.org site.
>
>
>
> Comments welcome!
This was meant as a Request For Comments, but the cat is
out of bag now:
https://lwn.net/Articles/725411/rss
;-)
I haven't heard any objections to this proposal, so let's
go with it. Consider this the official announcement of
the end-of-life of fedfs-utils.
--
Chuck Lever
Hi Chuck,
----- Original Message -----
> From: "Chuck Lever" <[email protected]>
> To: "J. Bruce Fields" <[email protected]>
> Cc: "fedfs-utils Developers" <[email protected]>, "linux-nfs" <[email protected]>, "NFSv4"
> <[email protected]>
> Sent: Wednesday, June 7, 2017 8:02:19 PM
> Subject: Re: [nfsv4] Proposal for end-of-life for fedfs-utils development
> Bruce, that's a good question, and an answer is worth sharing with
> the nfsv4 WG mailing list, whom I've cc'd.
>
>
>> On Jun 7, 2017, at 11:55 AM, [email protected] wrote:
>>
>> So if it's not too depressing I'd be curious what went wrong--did this
>> turn out to be harder than we thought to get stable, or are people happy
>> enough with automounting, or did we just not do a good job of explaining
>> it to people that might use it, or some combination of all those?
>
> If you're interested in an intriguing discussion of what makes a
> successful protocol, I recommend RFC 5218. Now, not in any particular
> order, the main reasons FedFS has not been widely adopted are (IMO of
> course):
>
>
> 1. Lack of vendor adoption
>
> After the specifications were completed, we anticipated two independent
> implementations. For various reasons the Solaris FedFS implementation
> was abandoned. One big reason was LDAP.
>
> NFS/Ganesha has expressed some interest in FedFS over the years, but
> I'm not aware of an implementation.
>
> NetApp abandoned FedFS before it was published as RFCs.
>
> So we were left with just the Linux implementation, just like what
> happened with SPKM.
>
>
> 2. LDAP is onerously complex
>
> The LDAP components of the Linux implementation were the worst to
> implement by far. This also proved to be the case in the Solaris
> prototype implementation. OpenLDAP is designed for massive scalability
> but aggressively shuns ease of administration.
>
> It was suggested that we integrate with FreeIPA, which is a Linux-based
> management suite that can provide an LDAP service, a KDC, and a
> certificate authority. But there was never enough user inertia to make
> that effort.
>
> There was resistance to integrating FedFS directly into nfs-utils
> because the LDAP components would have added complex library
> dependencies.
>
> The LDAP pieces of FedFS are specified to use TLS, but NFS operates
> using GSS and Kerberos. Getting these two worlds to work together was
> going to be the next step (and also, figuring out how to make the LDAP
> service use GSS instead, which we should have done before completing
> the standard).
>
> However, by the time the FedFS standards were complete, there was no
> longer WG interest to address its shortcomings. There were two small
> I-Ds published to start down that path. They went nowhere because the
> IETF's pool of LDAP expertise is difficult to consult, now that
> LDAP-related WGs have been disbanded.
>
> IMO LDAP, which was chosen early in the 2000s by the IBM prototypes
> that were to become FedFS, was ultimately a poor choice for what
> eventually became the public FedFS standard. This can be attributed
> to changing times.
>
>
> 3. Lack of consensus about how to store junctions
>
> There was never consensus in the Linux community about how to represent
> junction objects on disk. NFS wanted something that would look naturally
> like an empty directory to NFS versions that did not support referrals.
> Samba wanted something that could behave like symbolic links, as it
> had been using for its own DFS-style referrals. The filesystem folks
> were not interested in creating a new distinct object that could fill
> both roles.
>
> As you are probably aware, Bruce, I asked about this every year at
> LFS/MM for several years. I was always told "get back to us when you
> have users."
>
> Solaris went for reparse points in its implementation. Those are also
> supported by FreeBSD. I think RPs would have been a great direction
> but sadly these are not being actively pursued in the Linux FS
> community.
>
> Lack of user adoption sapped the energy out of the effort to find a
> consensus. Though, if FedFS had been widely adopted before a consensus
> was reached on junction object representation, we'd have a significant
> data conversion problem.
>
>
> 4. Existing implementations are capable enough
>
> This is mostly speculation on my part, but FedFS was a competitive
> response to the global namespace capabilities of AFS and SMB, not to
> any particular stated need in NFS communities.
>
> I've discussed the use of FedFS with various large enterprises, but
> quite often the underlying needs are able to be filled by some form
> of pNFS. I think this class of solution is what NetApp and Primary
> Data have adopted.
>
> Or put another way, pNFS seems capable of doing most anything that a
> referral-based mechanism can do. And in non-NFSv4 environments, the
> automounter is good enough for most people.
>
> Certainly we could design something from the ground up that addresses
> many of these shortcomings, but I get about one query about FedFS a
> year. I just wonder if it would be worth the effort.
We at DESY was interested in FedFS for two main reasons: one is to replace
AFS and second is to build a federation on independent NFS server.
The second option is not covered by pNFS as FedFS allows to run independent
pNFS instances with different administrative domains, implementations and
namespaces, but still provide a common mount point. NetApp or Primary Data
does not provide solution for that. Even we don't :-D.
Why we have failed? There was not clear how to set it up. LDAP management
was a mess. Schema was not standardized and not available with existing
LDAP server.
Currently we have 5 pNFS based instances with ~15PB in total. Some
nodes have direct mounts. But most of 'generic' clients automunter config
to fake FedFS as we don't know in advance which server will be used. Does
it works - Yes, does it solves all problems - No.
I would like to see a federated name space. We have the demand, but may be it's
only us.
For non NFS world we have a HTTP federation. It uses HTTP redirects, similar to
referrals. However, the tendency is to move towards distributed systems instead
of federated ones. If this project survive, then we can attempt to have a second
look at NFS federation. Life is a spiral.....
Regards,
Tigran.
>
>
>> --b.
>>
>> On Fri, Jun 02, 2017 at 11:01:05AM -0400, Chuck Lever wrote:
>>> Upstream fedfs-utils has not been under active development for
>>> two years or more, and there is a scant user base. I'd like to
>>> propose making 0.10 the final major release of fedfs-utils.
>>>
>>> The plan:
>>>
>>> - Since 0.10 is in at least one major enterprise distribution,
>>> I will remain available to integrate security fixes and make
>>> new minor releases in the 0.10 line, as needed, for one to
>>> two more years.
>>>
>>> - Retire and remove fedfs-utils from upstream mirror distros
>>> such as Fedora rawhide.
>>>
>>> - Transfer utilities such as nfsref into nfs-utils, with
>>> support for FedFS junctions removed.
>>>
>>> - Announcements of the change in status will be made on
>>> fedfs-utils-announce and on the wiki.linux-nfs.org site.
>
> --
> Chuck Lever
>
>
>
> _______________________________________________
> nfsv4 mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/nfsv4
> On Jun 14, 2017, at 5:57 PM, Mkrtchyan, Tigran <[email protected]> wrote:
>
> Hi Chuck,
>
> ----- Original Message -----
>> From: "Chuck Lever" <[email protected]>
>> To: "J. Bruce Fields" <[email protected]>
>> Cc: "fedfs-utils Developers" <[email protected]>, "linux-nfs" <[email protected]>, "NFSv4"
>> <[email protected]>
>> Sent: Wednesday, June 7, 2017 8:02:19 PM
>> Subject: Re: [nfsv4] Proposal for end-of-life for fedfs-utils development
>
>> Bruce, that's a good question, and an answer is worth sharing with
>> the nfsv4 WG mailing list, whom I've cc'd.
>>
>>
>>> On Jun 7, 2017, at 11:55 AM, [email protected] wrote:
>>>
>>> So if it's not too depressing I'd be curious what went wrong--did this
>>> turn out to be harder than we thought to get stable, or are people happy
>>> enough with automounting, or did we just not do a good job of explaining
>>> it to people that might use it, or some combination of all those?
>>
>> If you're interested in an intriguing discussion of what makes a
>> successful protocol, I recommend RFC 5218. Now, not in any particular
>> order, the main reasons FedFS has not been widely adopted are (IMO of
>> course):
>>
>>
>> 1. Lack of vendor adoption
>>
>> After the specifications were completed, we anticipated two independent
>> implementations. For various reasons the Solaris FedFS implementation
>> was abandoned. One big reason was LDAP.
>>
>> NFS/Ganesha has expressed some interest in FedFS over the years, but
>> I'm not aware of an implementation.
>>
>> NetApp abandoned FedFS before it was published as RFCs.
>>
>> So we were left with just the Linux implementation, just like what
>> happened with SPKM.
>>
>>
>> 2. LDAP is onerously complex
>>
>> The LDAP components of the Linux implementation were the worst to
>> implement by far. This also proved to be the case in the Solaris
>> prototype implementation. OpenLDAP is designed for massive scalability
>> but aggressively shuns ease of administration.
>>
>> It was suggested that we integrate with FreeIPA, which is a Linux-based
>> management suite that can provide an LDAP service, a KDC, and a
>> certificate authority. But there was never enough user inertia to make
>> that effort.
>>
>> There was resistance to integrating FedFS directly into nfs-utils
>> because the LDAP components would have added complex library
>> dependencies.
>>
>> The LDAP pieces of FedFS are specified to use TLS, but NFS operates
>> using GSS and Kerberos. Getting these two worlds to work together was
>> going to be the next step (and also, figuring out how to make the LDAP
>> service use GSS instead, which we should have done before completing
>> the standard).
>>
>> However, by the time the FedFS standards were complete, there was no
>> longer WG interest to address its shortcomings. There were two small
>> I-Ds published to start down that path. They went nowhere because the
>> IETF's pool of LDAP expertise is difficult to consult, now that
>> LDAP-related WGs have been disbanded.
>>
>> IMO LDAP, which was chosen early in the 2000s by the IBM prototypes
>> that were to become FedFS, was ultimately a poor choice for what
>> eventually became the public FedFS standard. This can be attributed
>> to changing times.
>>
>>
>> 3. Lack of consensus about how to store junctions
>>
>> There was never consensus in the Linux community about how to represent
>> junction objects on disk. NFS wanted something that would look naturally
>> like an empty directory to NFS versions that did not support referrals.
>> Samba wanted something that could behave like symbolic links, as it
>> had been using for its own DFS-style referrals. The filesystem folks
>> were not interested in creating a new distinct object that could fill
>> both roles.
>>
>> As you are probably aware, Bruce, I asked about this every year at
>> LFS/MM for several years. I was always told "get back to us when you
>> have users."
>>
>> Solaris went for reparse points in its implementation. Those are also
>> supported by FreeBSD. I think RPs would have been a great direction
>> but sadly these are not being actively pursued in the Linux FS
>> community.
>>
>> Lack of user adoption sapped the energy out of the effort to find a
>> consensus. Though, if FedFS had been widely adopted before a consensus
>> was reached on junction object representation, we'd have a significant
>> data conversion problem.
>>
>>
>> 4. Existing implementations are capable enough
>>
>> This is mostly speculation on my part, but FedFS was a competitive
>> response to the global namespace capabilities of AFS and SMB, not to
>> any particular stated need in NFS communities.
>>
>> I've discussed the use of FedFS with various large enterprises, but
>> quite often the underlying needs are able to be filled by some form
>> of pNFS. I think this class of solution is what NetApp and Primary
>> Data have adopted.
>>
>> Or put another way, pNFS seems capable of doing most anything that a
>> referral-based mechanism can do. And in non-NFSv4 environments, the
>> automounter is good enough for most people.
>>
>> Certainly we could design something from the ground up that addresses
>> many of these shortcomings, but I get about one query about FedFS a
>> year. I just wonder if it would be worth the effort.
>
> We at DESY was interested in FedFS for two main reasons: one is to replace
> AFS and second is to build a federation on independent NFS server.
> The second option is not covered by pNFS as FedFS allows to run independent
> pNFS instances with different administrative domains, implementations and
> namespaces, but still provide a common mount point. NetApp or Primary Data
> does not provide solution for that. Even we don't :-D.
The last time I visited DESY, I left with the impression that
there was a requirement that directories could be populated
programmatically (ie. one directory entry could represent a
group of files, perhaps on disparate NFS servers, and something
on each client determined which member of that group would be
opened). FedFS can't do that, but perhaps pNFS could.
> Why we have failed? There was not clear how to set it up. LDAP management
> was a mess. Schema was not standardized
The schema is now standardized in RFC 7532, but the pub date
for that document is recent (March 2015).
> and not available with existing LDAP server.
I recognized this could be an impediment to adoption, and
contacted the people responsible for 389ds and the OpenLDAP
community. I was told they could include the schema when
the schema was standardized and fedfs-utils had users.
fedfs-utils provides the schema in two forms, IIRC, ready
to copy to your favorite LDAP server.
> Currently we have 5 pNFS based instances with ~15PB in total. Some
> nodes have direct mounts. But most of 'generic' clients automunter config
> to fake FedFS as we don't know in advance which server will be used. Does
> it works - Yes, does it solves all problems - No.
>
> I would like to see a federated name space. We have the demand, but may be it's
> only us.
I can't speak authoritatively about that, but DESY is the
only contact I've had about fedfs-utils in years.
> For non NFS world we have a HTTP federation. It uses HTTP redirects, similar to
> referrals. However, the tendency is to move towards distributed systems instead
> of federated ones. If this project survive, then we can attempt to have a second
> look at NFS federation. Life is a spiral.....
There's no plan to take down the upstream fedfs-utils
source code repository.
> Regards,
> Tigran.
>
>>
>>
>>> --b.
>>>
>>> On Fri, Jun 02, 2017 at 11:01:05AM -0400, Chuck Lever wrote:
>>>> Upstream fedfs-utils has not been under active development for
>>>> two years or more, and there is a scant user base. I'd like to
>>>> propose making 0.10 the final major release of fedfs-utils.
>>>>
>>>> The plan:
>>>>
>>>> - Since 0.10 is in at least one major enterprise distribution,
>>>> I will remain available to integrate security fixes and make
>>>> new minor releases in the 0.10 line, as needed, for one to
>>>> two more years.
>>>>
>>>> - Retire and remove fedfs-utils from upstream mirror distros
>>>> such as Fedora rawhide.
>>>>
>>>> - Transfer utilities such as nfsref into nfs-utils, with
>>>> support for FedFS junctions removed.
>>>>
>>>> - Announcements of the change in status will be made on
>>>> fedfs-utils-announce and on the wiki.linux-nfs.org site.
>>
>> --
>> Chuck Lever
>>
>>
>>
>> _______________________________________________
>> nfsv4 mailing list
>> [email protected]
>> https://www.ietf.org/mailman/listinfo/nfsv4
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Chuck Lever
So if it's not too depressing I'd be curious what went wrong--did this
turn out to be harder than we thought to get stable, or are people happy
enough with automounting, or did we just not do a good job of explaining
it to people that might use it, or some combination of all those?
--b.
On Fri, Jun 02, 2017 at 11:01:05AM -0400, Chuck Lever wrote:
> Upstream fedfs-utils has not been under active development for
> two years or more, and there is a scant user base. I'd like to
> propose making 0.10 the final major release of fedfs-utils.
>
> The plan:
>
> - Since 0.10 is in at least one major enterprise distribution,
> I will remain available to integrate security fixes and make
> new minor releases in the 0.10 line, as needed, for one to
> two more years.
>
> - Retire and remove fedfs-utils from upstream mirror distros
> such as Fedora rawhide.
>
> - Transfer utilities such as nfsref into nfs-utils, with
> support for FedFS junctions removed.
>
> - Announcements of the change in status will be made on
> fedfs-utils-announce and on the wiki.linux-nfs.org site.
>
>
>
> Comments welcome!
>
>
> --
> Chuck Lever
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Bruce, that's a good question, and an answer is worth sharing with
the nfsv4 WG mailing list, whom I've cc'd.
> On Jun 7, 2017, at 11:55 AM, [email protected] wrote:
>
> So if it's not too depressing I'd be curious what went wrong--did this
> turn out to be harder than we thought to get stable, or are people happy
> enough with automounting, or did we just not do a good job of explaining
> it to people that might use it, or some combination of all those?
If you're interested in an intriguing discussion of what makes a
successful protocol, I recommend RFC 5218. Now, not in any particular
order, the main reasons FedFS has not been widely adopted are (IMO of
course):
1. Lack of vendor adoption
After the specifications were completed, we anticipated two independent
implementations. For various reasons the Solaris FedFS implementation
was abandoned. One big reason was LDAP.
NFS/Ganesha has expressed some interest in FedFS over the years, but
I'm not aware of an implementation.
NetApp abandoned FedFS before it was published as RFCs.
So we were left with just the Linux implementation, just like what
happened with SPKM.
2. LDAP is onerously complex
The LDAP components of the Linux implementation were the worst to
implement by far. This also proved to be the case in the Solaris
prototype implementation. OpenLDAP is designed for massive scalability
but aggressively shuns ease of administration.
It was suggested that we integrate with FreeIPA, which is a Linux-based
management suite that can provide an LDAP service, a KDC, and a
certificate authority. But there was never enough user inertia to make
that effort.
There was resistance to integrating FedFS directly into nfs-utils
because the LDAP components would have added complex library
dependencies.
The LDAP pieces of FedFS are specified to use TLS, but NFS operates
using GSS and Kerberos. Getting these two worlds to work together was
going to be the next step (and also, figuring out how to make the LDAP
service use GSS instead, which we should have done before completing
the standard).
However, by the time the FedFS standards were complete, there was no
longer WG interest to address its shortcomings. There were two small
I-Ds published to start down that path. They went nowhere because the
IETF's pool of LDAP expertise is difficult to consult, now that
LDAP-related WGs have been disbanded.
IMO LDAP, which was chosen early in the 2000s by the IBM prototypes
that were to become FedFS, was ultimately a poor choice for what
eventually became the public FedFS standard. This can be attributed
to changing times.
3. Lack of consensus about how to store junctions
There was never consensus in the Linux community about how to represent
junction objects on disk. NFS wanted something that would look naturally
like an empty directory to NFS versions that did not support referrals.
Samba wanted something that could behave like symbolic links, as it
had been using for its own DFS-style referrals. The filesystem folks
were not interested in creating a new distinct object that could fill
both roles.
As you are probably aware, Bruce, I asked about this every year at
LFS/MM for several years. I was always told "get back to us when you
have users."
Solaris went for reparse points in its implementation. Those are also
supported by FreeBSD. I think RPs would have been a great direction
but sadly these are not being actively pursued in the Linux FS
community.
Lack of user adoption sapped the energy out of the effort to find a
consensus. Though, if FedFS had been widely adopted before a consensus
was reached on junction object representation, we'd have a significant
data conversion problem.
4. Existing implementations are capable enough
This is mostly speculation on my part, but FedFS was a competitive
response to the global namespace capabilities of AFS and SMB, not to
any particular stated need in NFS communities.
I've discussed the use of FedFS with various large enterprises, but
quite often the underlying needs are able to be filled by some form
of pNFS. I think this class of solution is what NetApp and Primary
Data have adopted.
Or put another way, pNFS seems capable of doing most anything that a
referral-based mechanism can do. And in non-NFSv4 environments, the
automounter is good enough for most people.
Certainly we could design something from the ground up that addresses
many of these shortcomings, but I get about one query about FedFS a
year. I just wonder if it would be worth the effort.
> --b.
>
> On Fri, Jun 02, 2017 at 11:01:05AM -0400, Chuck Lever wrote:
>> Upstream fedfs-utils has not been under active development for
>> two years or more, and there is a scant user base. I'd like to
>> propose making 0.10 the final major release of fedfs-utils.
>>
>> The plan:
>>
>> - Since 0.10 is in at least one major enterprise distribution,
>> I will remain available to integrate security fixes and make
>> new minor releases in the 0.10 line, as needed, for one to
>> two more years.
>>
>> - Retire and remove fedfs-utils from upstream mirror distros
>> such as Fedora rawhide.
>>
>> - Transfer utilities such as nfsref into nfs-utils, with
>> support for FedFS junctions removed.
>>
>> - Announcements of the change in status will be made on
>> fedfs-utils-announce and on the wiki.linux-nfs.org site.
--
Chuck Lever