Linus, et al.
I know it's been discussed to death, but I am making a formal request to you
to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
many, many other groups here at Fermilab would be very happy to have this in
the main tree.
Currently the SDSS has ~20TB of XFS filesystems, most of which is in our 14
fileservers and database machines. The D-Zero experiment has ~140 desktops
running XFS and several XFS fileservers. We've been using it since it was
released, and have found it to be very reliable.
I'll even attempt to bribe you with homebrew beer - would that help?? ;-)
Thanks,
Dan
--
Dan Yocum
Sloan Digital Sky Survey, Fermilab 630.840.6509
[email protected], http://www.sdss.org
SDSS. Mapping the Universe.
On Monday 22 April 2002 17:10, Dan Yocum wrote:
> I know it's been discussed to death, but I am making a formal request to you
> to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
> many, many other groups here at Fermilab would be very happy to have this in
> the main tree.
The issue is how XFS's private versions of what would normally be generic vfs
facilities fit with the rest of the kernel. Want to help in the analysis?
Feel free to jump in.
> Currently the SDSS has ~20TB of XFS filesystems, most of which is in our 14
> fileservers and database machines. The D-Zero experiment has ~140 desktops
> running XFS and several XFS fileservers. We've been using it since it was
> released, and have found it to be very reliable.
>
> I'll even attempt to bribe you with homebrew beer - would that help?? ;-)
Programmer cycles would help. Oh, you can offer $$$ to certain kernel hackers
if you want it to go faster. Not to engage in advocacy of course, but to do
the necessary analysis.
--
Daniel
In article <[email protected]>,
Dan Yocum <[email protected]> wrote:
>I know it's been discussed to death, but I am making a formal request to you
>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
>many, many other groups here at Fermilab would be very happy to have this in
>the main tree.
Has XFS been proven to be completely stable and POSIX complient in its
behaviour? The reason I am asking is that XFS seems to be a fairly common
factor for segfault bugreports in dpkg. The problems are rare enough (and
never reproducable) so I can't prove this but it does leave me wondering.
Wichert.
--
_________________________________________________________________
/ Nothing is fool-proof to a sufficiently talented fool \
| [email protected] http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0 2805 3CB8 9250 2FA3 BC2D |
On Mon, 22 Apr 2002, Wichert Akkerman wrote:
> In article <[email protected]>,
> Dan Yocum <[email protected]> wrote:
> >I know it's been discussed to death, but I am making a formal request to you
> >to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
> >many, many other groups here at Fermilab would be very happy to have this in
> >the main tree.
>
> Has XFS been proven to be completely stable and POSIX complient in its
> behaviour? The reason I am asking is that XFS seems to be a fairly common
> factor for segfault bugreports in dpkg. The problems are rare enough (and
> never reproducable) so I can't prove this but it does leave me wondering.
Is there a test suite that checks POSIX (or better yet, SUS v3)
compliance of a file system? That might prove useful, although I'm well
aware it'd probably require some brains (and kernel modules) to check
consistency guarantees. But apart from that, things like "truncate to
zero length does not change the mtime of a file" (fixed in ReiserFS only
some weeks ago) might get caught that way.
On Mon, 2002-04-22 at 18:19, Matthias Andree wrote:
> Is there a test suite that checks POSIX (or better yet, SUS v3)
> compliance of a file system? That might prove useful, although I'm well
> aware it'd probably require some brains (and kernel modules) to check
> consistency guarantees. But apart from that, things like "truncate to
> zero length does not change the mtime of a file" (fixed in ReiserFS only
> some weeks ago) might get caught that way.
ftp://ftp.freestandards.org/pub/lsb/test_suites/
-chris
On 22 Apr 2002 18:55:20 +0200,
[email protected] (Wichert Akkerman) wrote:
>In article <[email protected]>,
>Dan Yocum <[email protected]> wrote:
>>I know it's been discussed to death, but I am making a formal request to you
>>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
>>many, many other groups here at Fermilab would be very happy to have this in
>>the main tree.
>
>Has XFS been proven to be completely stable
As much as any other filesystem. "There are no bugs in filesystem XYZ.
That just means that you have not looked hard enough." :) There is a
daily QA suite that XFS is run through.
>The reason I am asking is that XFS seems to be a fairly common
>factor for segfault bugreports in dpkg.
dpkg uses mmap? There was a bug in XFS and mmapped files where
incorrect blocks were flushed to disk under high load, but that was
fixed around January 30.
--
Not speaking for sgi.
Previously Keith Owens wrote:
> dpkg uses mmap?
To read all its data files, just.
> There was a bug in XFS and mmapped files where incorrect blocks were
> flushed to disk under high load, but that was fixed around January 30.
That would produce corrupt files which does not seem to be the case.
If memory serves me corrrectly one of the problems was that rename(2)
returned an error in rare cases that should not be possible (might have
been ENOENT even though both we have verified in advance that can't be
true).
Wichert
--
_________________________________________________________________
/[email protected] This space intentionally left occupied \
| [email protected] http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0 2805 3CB8 9250 2FA3 BC2D |
On Tue, 23 Apr 2002, Keith Owens wrote:
> On 22 Apr 2002 18:55:20 +0200,
> [email protected] (Wichert Akkerman) wrote:
> >In article <[email protected]>,
> >Dan Yocum <[email protected]> wrote:
> >>I know it's been discussed to death, but I am making a formal request to you
> >>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
> >>many, many other groups here at Fermilab would be very happy to have this in
> >>the main tree.
> >
> >Has XFS been proven to be completely stable
>
> As much as any other filesystem. "There are no bugs in filesystem XYZ.
> That just means that you have not looked hard enough." :) There is a
> daily QA suite that XFS is run through.
In the reality the inclusion on XFS in the 2.5 tree would probably move
more peole to use it, and so also to eventually trigger bugs, to report
them, sometimes to fix them.
This way XFS would improve faster, and of course that would be a
good thing.
That said, it is important to
consider the technical reasons to include XFS in 2.5 or not; if this
inclusion could cause some troubles, if XFS fits the requirements
Linus asks for the inclusion and what impact the inclusion would have on
the kernel (Think to JFS as a good example of an easy inclusion, with low
impact).
Luigi
On Tue, 2002-04-23 at 00:44, Wichert Akkerman wrote:
> If memory serves me corrrectly one of the problems was that rename(2)
> returned an error in rare cases that should not be possible (might have
> been ENOENT even though both we have verified in advance that can't be
> true).
>
That may be related to the accented character handling bug that appeared
for a short period of time, which was fixed a couple of months ago.
-tony
> Re: XFS in the main kernel
>
> From: Luigi Genoni ([email protected])
> On Tue, 23 Apr 2002, Keith Owens wrote:
>
> > On 22 Apr 2002 18:55:20 +0200,
> > [email protected] (Wichert Akkerman) wrote:
> > >In article <[email protected]>,
> > >Dan Yocum <[email protected]> wrote:
> > >>I know it's been discussed to death, but I am making a formal request to you
> > >>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
> > >>many, many other groups here at Fermilab would be very happy to have this in
> > >>the main tree.
> > >
> > >Has XFS been proven to be completely stable
> >
> > As much as any other filesystem. "There are no bugs in filesystem XYZ.
> > That just means that you have not looked hard enough." :) There is a
> > daily QA suite that XFS is run through.
>
> In the reality the inclusion on XFS in the 2.5 tree would probably move
> more peole to use it, and so also to eventually trigger bugs, to report
> them, sometimes to fix them.
> This way XFS would improve faster, and of course that would be a
> good thing.
>
definitely. Unless XFS is in the mainline kernel (marked as
experimantal if necessary) it will not get good exposure.
The most important (only) reason I do not use it (and recommend our
customers against using it) is that at the moment it is impossible to
track both the kernel and XFS at the same time. This is a shame, because
I think that for some application XFS is superior to the other
alternatives (can be said about the other alternatives to :-).
> That said, it is important to
> consider the technical reasons to include XFS in 2.5 or not; if this
> inclusion could cause some troubles, if XFS fits the requirements
> Linus asks for the inclusion and what impact the inclusion would have on
> the kernel (Think to JFS as a good example of an easy inclusion, with low
> impact).
>
so, what were the main obstacles again? The VFS layer?
Martin
--
------------------------------------------------------------------
Martin Knoblauch | email: [email protected]
TeraPort GmbH | Phone: +49-89-510857-309
C+ITS | Fax: +49-89-510857-111
http://www.teraport.de | Mobile: +49-170-4904759
Martin Knoblauch wrote:
>>Re: XFS in the main kernel
>>
>>From: Luigi Genoni ([email protected])
>>On Tue, 23 Apr 2002, Keith Owens wrote:
>>
>>>On 22 Apr 2002 18:55:20 +0200,
>>>[email protected] (Wichert Akkerman) wrote:
>>>
>>>>In article <[email protected]>,
>>>>Dan Yocum <[email protected]> wrote:
>>>>
>>>>>I know it's been discussed to death, but I am making a formal request to you
>>>>>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
>>>>>many, many other groups here at Fermilab would be very happy to have this in
>>>>>the main tree.
>>>>>
>>>>Has XFS been proven to be completely stable
>>>>
>>>As much as any other filesystem. "There are no bugs in filesystem XYZ.
>>>That just means that you have not looked hard enough." :) There is a
>>>daily QA suite that XFS is run through.
>>>
>>In the reality the inclusion on XFS in the 2.5 tree would probably move
>>more peole to use it, and so also to eventually trigger bugs, to report
>>them, sometimes to fix them.
>>This way XFS would improve faster, and of course that would be a
>>good thing.
>>
>
> definitely. Unless XFS is in the mainline kernel (marked as
>experimantal if necessary) it will not get good exposure.
>
> The most important (only) reason I do not use it (and recommend our
>customers against using it) is that at the moment it is impossible to
>track both the kernel and XFS at the same time. This is a shame, because
>I think that for some application XFS is superior to the other
>alternatives (can be said about the other alternatives to :-).
>
You would be surprised about the level of exposure XFS is getting, a lot
more
than you might realize. It is in everything from settop boxes and fiber
channel
switches to NAS boxes, those folks in general do not want to advertise.
Here are
a few larger scale installations out there:
http://oss.sgi.com/projects/xfs/xfs_users.html
Steve
Stephen Lord wrote:
>
> Martin Knoblauch wrote:
>
> >
> > definitely. Unless XFS is in the mainline kernel (marked as
> >experimantal if necessary) it will not get good exposure.
> >
> > The most important (only) reason I do not use it (and recommend our
> >customers against using it) is that at the moment it is impossible to
> >track both the kernel and XFS at the same time. This is a shame, because
> >I think that for some application XFS is superior to the other
> >alternatives (can be said about the other alternatives to :-).
> >
>
> You would be surprised about the level of exposure XFS is getting, a lot
> more
> than you might realize. It is in everything from settop boxes and fiber
> channel
> switches to NAS boxes, those folks in general do not want to advertise.
> Here are
> a few larger scale installations out there:
>
> http://oss.sgi.com/projects/xfs/xfs_users.html
>
> Steve
Steve,
no question that those are seriour users that give you serious
feedback. And if you call that
exposure, I am not going to argue. It is your project, it is your
marketing. (And *I* am not going to argue about SGI marketing :-(
From a mainline point of view XFS on Linux will only be successfull if
it is "in the kernel". Fully maintained and "Linus approved". I am not
sure when SGI started the port (could even go back to the time when I
worked for them, late 1997). Definitely quite some time. By now it
should be in the kernel. Maybe marked "experimental". As I see it now
EXT3, ReiserFS and maybe JFS are just eating the XFS lunch away.
In any case, the Vanderbilt comment is right on.
Martin
--
------------------------------------------------------------------
Martin Knoblauch | email: [email protected]
TeraPort GmbH | Phone: +49-89-510857-309
C+ITS | Fax: +49-89-510857-111
http://www.teraport.de | Mobile: +49-170-4904759
Martin Knoblauch wrote:
>>Re: XFS in the main kernel
>>
>>From: Luigi Genoni ([email protected])
>>On Tue, 23 Apr 2002, Keith Owens wrote:
>>
>>
>>>On 22 Apr 2002 18:55:20 +0200,
>>>[email protected] (Wichert Akkerman) wrote:
>>>
>>>>In article <[email protected]>,
>>>>Dan Yocum <[email protected]> wrote:
>>>>
>>>>>I know it's been discussed to death, but I am making a formal request to you
>>>>>to include XFS in the main kernel. We (The Sloan Digital Sky Survey) and
>>>>>many, many other groups here at Fermilab would be very happy to have this in
>>>>>the main tree.
>>>>>
>>>>Has XFS been proven to be completely stable
>>>>
>>>As much as any other filesystem. "There are no bugs in filesystem XYZ.
>>>That just means that you have not looked hard enough." :) There is a
>>>daily QA suite that XFS is run through.
>>>
>>In the reality the inclusion on XFS in the 2.5 tree would probably move
>>more peole to use it, and so also to eventually trigger bugs, to report
>>them, sometimes to fix them.
>>This way XFS would improve faster, and of course that would be a
>>good thing.
>>
>>
>
> definitely. Unless XFS is in the mainline kernel (marked as
> experimantal if necessary) it will not get good exposure.
>
> The most important (only) reason I do not use it (and recommend our
> customers against using it) is that at the moment it is impossible to
> track both the kernel and XFS at the same time. This is a shame, because
> I think that for some application XFS is superior to the other
> alternatives (can be said about the other alternatives to :-).
>
>
>>That said, it is important to
>>consider the technical reasons to include XFS in 2.5 or not; if this
>>inclusion could cause some troubles, if XFS fits the requirements
>>Linus asks for the inclusion and what impact the inclusion would have on
>>the kernel (Think to JFS as a good example of an easy inclusion, with low
>>impact).
>>
>>
>
> so, what were the main obstacles again? The VFS layer?
>
The VFS and such features like "delayed block allocation". XFS tries
to gather 64K or so before submitting to disk/block layer.
FWIW, SuSE 8 ships with full (but experimental marked) XFS support.
Peter W?chtler wrote:
>
> Martin Knoblauch wrote:
> >>Re: XFS in the main kernel
> >>
> >
> > so, what were the main obstacles again? The VFS layer?
> >
>
> The VFS and such features like "delayed block allocation". XFS tries
> to gather 64K or so before submitting to disk/block layer.
>
> FWIW, SuSE 8 ships with full (but experimental marked) XFS support.
Definitely a step forward. But some people (including myself, I guess)
do not like distribution kernels. Yeah, hard to please - I know :-). I
use SuSE on the desktop, but not because of the kernel. I use RedHat on
servers and compute nodes, but not because of the kernel.
Martin
PS: Thanks for the hint. I didn't realize it when I upgraded to 8.0.
--
------------------------------------------------------------------
Martin Knoblauch | email: [email protected]
TeraPort GmbH | Phone: +49-89-510857-309
C+ITS | Fax: +49-89-510857-111
http://www.teraport.de | Mobile: +49-170-4904759
On 2002.04.23 Martin Knoblauch wrote:
>Stephen Lord wrote:
>>
>> Martin Knoblauch wrote:
>>
>> >
>> > definitely. Unless XFS is in the mainline kernel (marked as
>> >experimantal if necessary) it will not get good exposure.
>> >
[...]
>
> From a mainline point of view XFS on Linux will only be successfull if
>it is "in the kernel". Fully maintained and "Linus approved". I am not
>sure when SGI started the port (could even go back to the time when I
>worked for them, late 1997). Definitely quite some time. By now it
>should be in the kernel. Maybe marked "experimental". As I see it now
>EXT3, ReiserFS and maybe JFS are just eating the XFS lunch away.
>
> In any case, the Vanderbilt comment is right on.
>
If XFS is so good (i do not doubt it), I see some issues (plz correct me
if I'm wrong...):
- XFS needs substantial changes in the VFS layer to work
- This changes are good (or make xfs so good)
- *THE THING* to do is to integrate this changes in mainline tree VFS,
so XFS will stop duplicating half the kernel code.
Why those features are not merged ? Incompatibilities ? Licensing ?
Religious wars about some way of doing things ?
Plz, if SGI splits XFS in small chunks and starts feeding linus with
changes in the VFS, what will happen ? Why that doesn't happen ?
Just some ideas...
--
J.A. Magallon # Let the source be with you...
mailto:[email protected]
Mandrake Linux release 8.3 (Cooker) for i586
Linux werewolf 2.4.19-pre7-jam6 #2 SMP mar abr 23 16:56:56 CEST 2002 i686
On Tue, 23 Apr 2002, Martin Knoblauch wrote:
> > Re: XFS in the main kernel
> >
> definitely. Unless XFS is in the mainline kernel (marked as
> experimantal if necessary) it will not get good exposure.
XFS needs 2.5, not 2.4, because of a lot of reasons.
If I do remember well a strong obiection to XFS is that it introduces a
kernel thread to emulate Irix behavious to talk with pagebuf (a la Irix),
end to have an interface with VM and Block Device layer.
This forces some vincles.
It is a lot of time that I do not try XFS, so maybe things changed, but
XFS has has data block of the same size of memory pages (4 or 8 Kb
depending on architecture), not that I have some remak about that, but I
use XFS on Irix on an Origin 2000 with a couple of TB disk space, it is
another thing in front of Linux ports.
On the other side delayed allocation is quite cool ;)
>
> The most important (only) reason I do not use it (and recommend our
> customers against using it) is that at the moment it is impossible to
> track both the kernel and XFS at the same time. This is a shame, because
> I think that for some application XFS is superior to the other
> alternatives (can be said about the other alternatives to :-).
Every FS has its strenght points and its weackness.
For example on MC^2 I found reiserFS on LVM to have a really good
interaction with the way MC^2 works, expecially because
I have a lot of small|medium sized files, I suppose.
I also tied XFS and JFS on MC^2, and maybe with other file size
and I/O loads they would be better. I just talk for my systems needs.
(all test were with latest 2.4 kernels, a month ago.
I cannot risk corruption on this
storage system, so 2.5 is not for me there :( ).
>
> > That said, it is important to
> > consider the technical reasons to include XFS in 2.5 or not; if this
> > inclusion could cause some troubles, if XFS fits the requirements
> > Linus asks for the inclusion and what impact the inclusion would have on
> > the kernel (Think to JFS as a good example of an easy inclusion, with low
> > impact).
> >
>
> so, what were the main obstacles again? The VFS layer?
>
See my previous comments...
Luigi Genoni wrote:
>
> On Tue, 23 Apr 2002, Martin Knoblauch wrote:
>
> > > Re: XFS in the main kernel
> > >
> > definitely. Unless XFS is in the mainline kernel (marked as
> > experimantal if necessary) it will not get good exposure.
>
> XFS needs 2.5, not 2.4, because of a lot of reasons.
> If I do remember well a strong obiection to XFS is that it introduces a
> kernel thread to emulate Irix behavious to talk with pagebuf (a la Irix),
> end to have an interface with VM and Block Device layer.
>
> This forces some vincles.
>
No problem with XFS going into 2.5 mainline, but not 2.4. But - is that
happening?
Martin
--
------------------------------------------------------------------
Martin Knoblauch | email: [email protected]
TeraPort GmbH | Phone: +49-89-510857-309
C+ITS | Fax: +49-89-510857-111
http://www.teraport.de | Mobile: +49-170-4904759
On Wed, 24 Apr 2002, Martin Knoblauch wrote:
> Luigi Genoni wrote:
> >
> > On Tue, 23 Apr 2002, Martin Knoblauch wrote:
> >
> > > > Re: XFS in the main kernel
> > > >
> > > definitely. Unless XFS is in the mainline kernel (marked as
> > > experimantal if necessary) it will not get good exposure.
> >
> > XFS needs 2.5, not 2.4, because of a lot of reasons.
> > If I do remember well a strong obiection to XFS is that it introduces a
> > kernel thread to emulate Irix behavious to talk with pagebuf (a la Irix),
> > end to have an interface with VM and Block Device layer.
> >
> > This forces some vincles.
> >
>
> No problem with XFS going into 2.5 mainline, but not 2.4. But - is that
> happening?
>
I do not think so. At the end it is Linus the one who should decide, and
XFS should find a way to agree with him. All depends on this.
On Tuesday 23 April 2002 23:37, J.A. Magallon wrote:
> On 2002.04.23 Martin Knoblauch wrote:
> If XFS is so good (i do not doubt it), I see some issues (plz correct me
> if I'm wrong...):
>
> - XFS needs substantial changes in the VFS layer to work
> - This changes are good (or make xfs so good)
> - *THE THING* to do is to integrate this changes in mainline tree VFS,
> so XFS will stop duplicating half the kernel code.
>
> Why those features are not merged ? Incompatibilities ? Licensing ?
> Religious wars about some way of doing things ?
No. It's simply a matter of nobody having done the required analysis to
find a really good way to reconcile XFS's way of doing things with
mainline vfs. This is time-consuming work that requires a good deal of
skill, and right now there are many projects in the same category.
My advice to anyone who wants to make it go faster? Jump in and start
doing the analysis (start with xfs/pagebuf.c). If you are a company who
wants it to go faster, try offering money. Otherwise, it goes at its own
speed, and this work will likely come up to the top of the pile later in
the 2.5 cycle.
--
Daniel
On Tuesday 23 April 2002 23:43, Luigi Genoni wrote:
> If I do remember well a strong obiection to XFS is that it introduces a
> kernel thread to emulate Irix behavious to talk with pagebuf (a la Irix),
> end to have an interface with VM and Block Device layer.
There is nothing wrong with a filesystem having its own management thread -
this seems to be a feature of all the new filesystems. How that thread
interacts with vm and the vfs is an open question, not answered very well
by any filesystem at the moment, let alone XFS. Bottom line: XFS's thread
is not the issue.
--
Daniel
It also depends on what XFS guys want to do.
They are probably confortable with this sort of interface so that XFS
works de facto like if it has to do with Irix.
To go to Linux VFS will be a lot of work for them I think.
On Tue, 23 Apr 2002, Daniel Phillips wrote:
> On Tuesday 23 April 2002 23:37, J.A. Magallon wrote:
> > On 2002.04.23 Martin Knoblauch wrote:
> > If XFS is so good (i do not doubt it), I see some issues (plz correct me
> > if I'm wrong...):
> >
> > - XFS needs substantial changes in the VFS layer to work
> > - This changes are good (or make xfs so good)
> > - *THE THING* to do is to integrate this changes in mainline tree VFS,
> > so XFS will stop duplicating half the kernel code.
> >
> > Why those features are not merged ? Incompatibilities ? Licensing ?
> > Religious wars about some way of doing things ?
>
> No. It's simply a matter of nobody having done the required analysis to
> find a really good way to reconcile XFS's way of doing things with
> mainline vfs. This is time-consuming work that requires a good deal of
> skill, and right now there are many projects in the same category.
>
> My advice to anyone who wants to make it go faster? Jump in and start
> doing the analysis (start with xfs/pagebuf.c). If you are a company who
> wants it to go faster, try offering money. Otherwise, it goes at its own
> speed, and this work will likely come up to the top of the pile later in
> the 2.5 cycle.
>
> --
> Daniel
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>