2009-03-31 08:15:09

by Martin Steigerwald

[permalink] [raw]
Subject: Replacing VFAT as filesystem on removeable media


Hi!

Now as TomTom appears to have surrendered to Microsoft and Microsoft seems
to have accepted this deal probably in order to not find out that their
patents are void, I think replacing VFAT as standard cross platform
filesystem on removeable media would be an even better idea than before
[1].

Granted I believe the patents will be made void one day, but even then I
think it would make sense to replace VFAT for technical reasons.

Such a filesystem IMHO should have the following features:
- cross platform with implementations for Linux, Windows, Mac OS X at
least, also the varios BSD variantes come to my mind
- open source
- probably some support for flash media without or with bad wear leveling
- some kind of journaling or other metadata consistency guarentee

People use ext2 as an alternative, but that lacks journaling.

I could also think of UDF with write support, but I am not sure whether
Windows and Mac OS X has write support.

Or probably even a new filesystem as long as people appear to write one
filesystem after another these days.

Or some kind of FAT *without* compatibility hacks, but I think this would
still be an inferior solution as long as it doesn't provide for metadata
consistency.

Maybe this could become some kind of Linux Foundation or FSF joint effort?
Together with advertising and advocacy of free software users this could
probably really replace VFAT in the long term.

What do you think? What other features would make sense to for such a
filesystem.

I am willing to test such a filesystem and help with documentation as well
as advocacy. But for coding I better start with something easier ;-).

[1]
http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/112964

Ciao,
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7


Attachments:
(No filename) (1.83 kB)
signature.asc (197.00 B)
This is a digitally signed message part.
Download all attachments

2009-03-31 13:23:47

by Mark Williamson

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

Another approach which occurs to me would be to come up with a "filesystem over
USB" protocol so that the actual on-disk format is no longer relevant. With a
standard or reasonably common FSoUSB implementation, a device maker could
choose a disk filesystem that's appropriate to their purpose and hide the
implementation details within the device. For instance, an MP3 player could
use a flash-specific disk filesystem but would be able hide the details of this
from the host PC.

This might not be so desirable for simple devices (like USB card readers,
perhaps) but I imagine it would fairly straightforward on media players,
GPSes, etc that run a "proper" OS. The most pain would arguably be in
*somebody* having to write a filesystem driver for Windows but at least it
would only need writing once ;-)

I've seen some talk of this sort of approach online but nothing too specific.
A Linux USB website suggested that a certain camera protocol might be suitable
as a basis for files-over-USB (there's even a FUSE module for this, I think).
Another possibility that occurs would be to write a USB transport for 9P,
which looked like it might be fairly straightforward...

Just a thought, anyhow.

Cheers,
Mark

On Tuesday 31 March 2009 09:15:28 Martin Steigerwald wrote:
> Hi!
>
> Now as TomTom appears to have surrendered to Microsoft and Microsoft seems
> to have accepted this deal probably in order to not find out that their
> patents are void, I think replacing VFAT as standard cross platform
> filesystem on removeable media would be an even better idea than before
> [1].
>
> Granted I believe the patents will be made void one day, but even then I
> think it would make sense to replace VFAT for technical reasons.
>
> Such a filesystem IMHO should have the following features:
> - cross platform with implementations for Linux, Windows, Mac OS X at
> least, also the varios BSD variantes come to my mind
> - open source
> - probably some support for flash media without or with bad wear leveling
> - some kind of journaling or other metadata consistency guarentee
>
> People use ext2 as an alternative, but that lacks journaling.
>
> I could also think of UDF with write support, but I am not sure whether
> Windows and Mac OS X has write support.
>
> Or probably even a new filesystem as long as people appear to write one
> filesystem after another these days.
>
> Or some kind of FAT *without* compatibility hacks, but I think this would
> still be an inferior solution as long as it doesn't provide for metadata
> consistency.
>
> Maybe this could become some kind of Linux Foundation or FSF joint effort?
> Together with advertising and advocacy of free software users this could
> probably really replace VFAT in the long term.
>
> What do you think? What other features would make sense to for such a
> filesystem.
>
> I am willing to test such a filesystem and help with documentation as well
> as advocacy. But for coding I better start with something easier ;-).
>
> [1]
> http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/
>112964
>
> Ciao,

2009-03-31 13:47:43

by Xavier Bestel

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

On Tue, 2009-03-31 at 13:57 +0100, Mark Williamson wrote:
> Another approach which occurs to me would be to come up with a "filesystem over
> USB" protocol so that the actual on-disk format is no longer relevant.

Look for MTP. It's what you want, made by Microsoft, and enables all
sorts of DRM niceties.

Xav

2009-03-31 15:03:01

by Mark Williamson

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

Hi Xav,

On Tuesday 31 March 2009 14:47:07 Xavier Bestel wrote:
> On Tue, 2009-03-31 at 13:57 +0100, Mark Williamson wrote:
> > Another approach which occurs to me would be to come up with a
> > "filesystem over USB" protocol so that the actual on-disk format is no
> > longer relevant.
>
> Look for MTP. It's what you want, made by Microsoft, and enables all
> sorts of DRM niceties.

Ah, thanks for that. It looks fairly close to what I had in mind, although I
think there are some differences (it transfers whole files at a time and
incorporates device control stuff - neither of which is really necessary in the
scheme I imagined).

According to Wikipedia it is now an official USB device class, which sounds
attractive. I've not heard of Linux providing an MTP responder ("server"),
which I find mildly surprising as it sounds like it would be useful,
particularly given the range of advanced filesystems Linux devices might like
to leverage under the hood. It seems odd if an embedded Linux company hasn't
already coded this up for a device ... ?

I can see how a file-based protocol would be "useful" in enforcing DRM-like
protections in a device implementation but I don't think that such protocols
fundamentally enables any nastiness that couldn't be enforced in other ways by
a motivated pro-DRM device manufacturer. Sounds like MTP is designed to
facilitate DRM if the communications endpoints want it; not something I'm keen
on but also not something that rules it out as a useful standard.

A file-based transfer protocol seems like it would be useful to have for
"sufficiently intelligent" devices, whether it's MTP or something else.

Cheers,
Mark

2009-03-31 15:22:26

by Xavier Bestel

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

On Tue, 2009-03-31 at 16:02 +0100, Mark Williamson wrote:
> I've not heard of Linux providing an MTP responder ("server"),
> which I find mildly surprising as it sounds like it would be useful,
> particularly given the range of advanced filesystems Linux devices might like
> to leverage under the hood. It seems odd if an embedded Linux company hasn't
> already coded this up for a device ... ?

I'm speaking out of my ass here, but seeing how Microsoft managed to
sneak some patents into something as trivial as FAT, I'm pretty sure MTP
is a hell of a minefield.

Xav

2009-03-31 15:53:33

by Mark Williamson

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

On Tuesday 31 March 2009 16:21:37 Xavier Bestel wrote:
> I'm speaking out of my ass here, but seeing how Microsoft managed to
> sneak some patents into something as trivial as FAT, I'm pretty sure MTP
> is a hell of a minefield.

That did occur to me too but I was somewhat hopeful that perhaps this would be
mitigated by the (eventual?) adoption as a usb.org standard.

The spec and an adopters' legal agreement around it is here:
http://www.usb.org/developers/devclass_docs/MTP_1.0.zip

I scanned the agreement and it didn't look like it was obviously evil but I'm
not really qualified to make that judgement ;-) It had some promising words in
it like "zero royalty" but I would be much happier if someone with legal
knowhow (and preferably experience of the usb.org standards procedures)
decoded it for me / us!

Assuming the spec is legally "safe" to implement, I would have thought it
would be generally beneficial for device manufacturers to support both
"initiator" and "responder" endpoints under Linux.

Cheers,
Mark

2009-04-02 17:51:37

by Bryan Henderson

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

> Now as TomTom appears to have surrendered to Microsoft and Microsoft
seems
> to have accepted this deal probably in order to not find out that their
> patents are void, I think replacing VFAT as standard cross platform
> filesystem on removeable media would be an even better idea than before
> [1].

> [1]
>
http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/112964

How do you mean "replace"?

VFAT's standardness today comes from the fact that virtually every
personal computer in the world today and for the foreseeable future is
able to read and write VFAT. Do you envision some other filesystem format
achieving that status?

--
Bryan Henderson IBM Almaden Research Center
San Jose CA Storage Systems

2009-04-03 08:28:40

by Martin Steigerwald

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

Am Donnerstag 02 April 2009 schrieb Bryan Henderson:
> > Now as TomTom appears to have surrendered to Microsoft and Microsoft
>
> seems
>
> > to have accepted this deal probably in order to not find out that
> > their patents are void, I think replacing VFAT as standard cross
> > platform filesystem on removeable media would be an even better idea
> > than before [1].
> >
> > [1]
>
> http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/n
>ews/112964
>
> How do you mean "replace"?
>
> VFAT's standardness today comes from the fact that virtually every
> personal computer in the world today and for the foreseeable future is
> able to read and write VFAT. Do you envision some other filesystem
> format achieving that status?

That was my (possibly crazy) idea yes. ;)

--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7


Attachments:
(No filename) (906.00 B)
signature.asc (197.00 B)
This is a digitally signed message part.
Download all attachments

2009-04-05 06:35:17

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

Martin Steigerwald wrote:
>>
>> How do you mean "replace"?
>>
>> VFAT's standardness today comes from the fact that virtually every
>> personal computer in the world today and for the foreseeable future is
>> able to read and write VFAT. Do you envision some other filesystem
>> format achieving that status?
>
> That was my (possibly crazy) idea yes. ;)
>

Ironically enough, one of the better filesystems for being supported by
many OSes is probably ext2. However, that doesn't mean it is even in
the same rough ballpark as (V)FAT.

-hpa

--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.

Subject: Re: Replacing VFAT as filesystem on removeable media

El Sat, 04 Apr 2009 23:34:50 -0700
"H. Peter Anvin" <[email protected]> escribió:

> Martin Steigerwald wrote:
> >>
> >> How do you mean "replace"?
> >>
> >> VFAT's standardness today comes from the fact that virtually every
> >> personal computer in the world today and for the foreseeable future is
> >> able to read and write VFAT. Do you envision some other filesystem
> >> format achieving that status?
> >
> > That was my (possibly crazy) idea yes. ;)
> >
>
> Ironically enough, one of the better filesystems for being supported by
> many OSes is probably ext2. However, that doesn't mean it is even in
> the same rough ballpark as (V)FAT.

What about UDF ?
>
> -hpa
>

2009-04-05 20:05:27

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

Alejandro Riveira Fernández wrote:
>>>
>> Ironically enough, one of the better filesystems for being supported by
>> many OSes is probably ext2. However, that doesn't mean it is even in
>> the same rough ballpark as (V)FAT.
>
> What about UDF ?

Good point. UDF is probably widely supported, but again, not in the
same ballpark. I don't personally know how complex UDF is to implement
on a small memory device.

-hpa

--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.

2009-04-06 19:52:24

by Nicholas Miell

[permalink] [raw]
Subject: Re: Replacing VFAT as filesystem on removeable media

On Sun, 2009-04-05 at 13:05 -0700, H. Peter Anvin wrote:
> Alejandro Riveira Fernández wrote:
> >>>
> >> Ironically enough, one of the better filesystems for being supported by
> >> many OSes is probably ext2. However, that doesn't mean it is even in
> >> the same rough ballpark as (V)FAT.
> >
> > What about UDF ?
>
> Good point. UDF is probably widely supported, but again, not in the
> same ballpark. I don't personally know how complex UDF is to implement
> on a small memory device.
>
> -hpa
>

DVD players are small memory devices and they don't actually implement
UDF.

(DVD movies have UDF filesystems, but they're constructed such that the
files that the DVD player needs are at specific fixed offsets on the
disc, and the DVD player never actually interprets the UDF filesystem.)

Also, there's several different revisions of UDF, which causes some
level of operating system compatibility problems.

Fortunately, the UDF flavor likely to be used on magnetic/flash media is
also the flavor that's the most compatible, so that makes things easier
for everyone.

--
Nicholas Miell <[email protected]>