This patch adds a set of ioctls to get and set the FAT filesystem native
attribute bits, including the unused bits (6 and 7.)
It also includes some very minor code cleanups; mostly by macroizing a
few idioms.
-hpa
Signed-Off-By: H. Peter Anvin <[email protected]>
On Mon, 2005-01-03 at 12:57 -0800, H. Peter Anvin wrote:
> This patch adds a set of ioctls to get and set the FAT filesystem native
> attribute bits, including the unused bits (6 and 7.)
>
Instead of adding another ioctl, wouldn't an xattr be more appropriate?
For instance, system.fatattrs containing a text representation of the
attribute bits.
i.e.
a = archive
d = directory
h = hidden
r = read only
s = system
v = volume
6 = unused bit 6
7 = unused bit 7
and
bash-3.00$ getfattr -n system.fatattrs dosfile.txt
# file: dosfile.txt
system.fatattrs="ar"
bash-3.00$
--
Nicholas Miell <[email protected]>
Nicholas Miell wrote:
> On Mon, 2005-01-03 at 12:57 -0800, H. Peter Anvin wrote:
>
>>This patch adds a set of ioctls to get and set the FAT filesystem native
>>attribute bits, including the unused bits (6 and 7.)
>>
>
>
> Instead of adding another ioctl, wouldn't an xattr be more appropriate?
> For instance, system.fatattrs containing a text representation of the
> attribute bits.
>
This really worries me, because it's not clear to me that Microsoft
isn't going to add NTFS-style xattrs to FAT in the future. There is a
very specific reason why they might want to do that: since they want to
keep NTFS secret and proprietary, FAT is the published interchange
format that other devices can use to exchange data with MS operating
systems. If we then have overloaded the xattr mechanism, that would be
very ugly.
-hpa
On Mon, 2005-01-03 at 13:35 -0800, H. Peter Anvin wrote:
> Nicholas Miell wrote:
> > Instead of adding another ioctl, wouldn't an xattr be more appropriate?
> > For instance, system.fatattrs containing a text representation of the
> > attribute bits.
> >
>
> This really worries me, because it's not clear to me that Microsoft
> isn't going to add NTFS-style xattrs to FAT in the future. There is a
> very specific reason why they might want to do that: since they want to
> keep NTFS secret and proprietary, FAT is the published interchange
> format that other devices can use to exchange data with MS operating
> systems. If we then have overloaded the xattr mechanism, that would be
> very ugly.
>
> -hpa
That's why I put fatattrs in the system namespace, which is wholly owned
by the Linux kernel. Any theoretical FAT-with-xattrs variant would put
those xattrs in the user namespace.
On another note, NTFS-style xattrs (aka named streams) are unrelated to
Linux xattrs. A named stream is a separate file with a funny name, while
a Linux xattr is a named extension to struct stat.
--
Nicholas Miell <[email protected]>
Nicholas Miell wrote:
>
> That's why I put fatattrs in the system namespace, which is wholly owned
> by the Linux kernel. Any theoretical FAT-with-xattrs variant would put
> those xattrs in the user namespace.
>
> On another note, NTFS-style xattrs (aka named streams) are unrelated to
> Linux xattrs. A named stream is a separate file with a funny name, while
> a Linux xattr is a named extension to struct stat.
>
OK, that does make it more sensible. I do note, however, that ext2/ext3
do not seem to export their attributes (chattr/lsattr) in this way; I do
also note that the xattr code wherever it has been implemented is just
painfully complex.
I'll see if I can weed it down to some kind of sane size.
-hpa
On Mon, 2005-01-03 at 14:02 -0800, H. Peter Anvin wrote:
> Nicholas Miell wrote:
> >
> > That's why I put fatattrs in the system namespace, which is wholly owned
> > by the Linux kernel. Any theoretical FAT-with-xattrs variant would put
> > those xattrs in the user namespace.
> >
> > On another note, NTFS-style xattrs (aka named streams) are unrelated to
> > Linux xattrs. A named stream is a separate file with a funny name, while
> > a Linux xattr is a named extension to struct stat.
> >
>
> OK, that does make it more sensible. I do note, however, that ext2/ext3
> do not seem to export their attributes (chattr/lsattr) in this way; I do
> also note that the xattr code wherever it has been implemented is just
> painfully complex.
>
> I'll see if I can weed it down to some kind of sane size.
>
> -hpa
Yeah, I contemplated adding system.fattattrs, system.ntfsattrs, and
system.linuxattrs (for the ext2 attrs that have popped up in several
other filesystems) a while ago, but xattrs seem to be the red-headed
left-handed stepchild of the Linux VFS and I lost interest in the
project.
Nice to see someone else interested in it, though.
--
Nicholas Miell <[email protected]>
Nicholas Miell wrote:
>
> Yeah, I contemplated adding system.fattattrs, system.ntfsattrs, and
> system.linuxattrs (for the ext2 attrs that have popped up in several
> other filesystems) a while ago, but xattrs seem to be the red-headed
> left-handed stepchild of the Linux VFS and I lost interest in the
> project.
>
> Nice to see someone else interested in it, though.
>
I'm honestly not sure that using an ASCII string in an xattr is the sane
way of doing this. Even a binary byte in an xattr would make more sense
in some ways.
I think the xattr mechanism is ignored largely because it's painfully
complex.
A plus with using xattr is that in theory (but of course not in
practice!) it would let one store a copy of a DOS filesystem on an ext3
(or xfs, or...) filesystem and have it restored, all using standard (but
by necessity, xattr-aware) tools. However, the splitting of xattr into
namespaces may very well make that impossible, since what's a "system"
attribute to one filesystem is a "user" attribute to another. Classic
design flaw, by the way.
Anyway, I'm going to send out something to the various maintainers of
DOS-based filesystems (FAT, CIFS, NTFS) and see what they think.
-hpa
On Mon, 2005-01-03 at 14:25 -0800, H. Peter Anvin wrote:
> I'm honestly not sure that using an ASCII string in an xattr is the sane
> way of doing this. Even a binary byte in an xattr would make more sense
> in some ways.
ASCII strings require no special tools to manipulate from shell scripts.
> I think the xattr mechanism is ignored largely because it's painfully
> complex.
>
> A plus with using xattr is that in theory (but of course not in
> practice!) it would let one store a copy of a DOS filesystem on an ext3
> (or xfs, or...) filesystem and have it restored, all using standard (but
> by necessity, xattr-aware) tools. However, the splitting of xattr into
> namespaces may very well make that impossible, since what's a "system"
> attribute to one filesystem is a "user" attribute to another. Classic
> design flaw, by the way.
The design does allow users to store whatever they want as an xattr
without having to worry about how the kernel chooses to interpret it,
though. (i.e. the user namespace is just a byte array that the kernel
stores for you, while the system/security namespaces are probably
generated and interpreted on demand.)
> Anyway, I'm going to send out something to the various maintainers of
> DOS-based filesystems (FAT, CIFS, NTFS) and see what they think.
>
> -hpa
--
Nicholas Miell <[email protected]>
[Pruning the Cc: list.]
Nicholas Miell wrote:
> On Mon, 2005-01-03 at 14:25 -0800, H. Peter Anvin wrote:
>
>>I'm honestly not sure that using an ASCII string in an xattr is the sane
>>way of doing this. Even a binary byte in an xattr would make more sense
>>in some ways.
>
> ASCII strings require no special tools to manipulate from shell scripts.
>
You need some kind of special tool anyway, i.e. getfattr/setfattr. What
tool you use isn't really important.
The fact that getfattr/setfattr can't deal with attributes that aren't
ASCII strings seem like flaws in these tools.
>
>>I think the xattr mechanism is ignored largely because it's painfully
>>complex.
>>
>>A plus with using xattr is that in theory (but of course not in
>>practice!) it would let one store a copy of a DOS filesystem on an ext3
>>(or xfs, or...) filesystem and have it restored, all using standard (but
>>by necessity, xattr-aware) tools. However, the splitting of xattr into
>>namespaces may very well make that impossible, since what's a "system"
>>attribute to one filesystem is a "user" attribute to another. Classic
>>design flaw, by the way.
>
> The design does allow users to store whatever they want as an xattr
> without having to worry about how the kernel chooses to interpret it,
> though. (i.e. the user namespace is just a byte array that the kernel
> stores for you, while the system/security namespaces are probably
> generated and interpreted on demand.)
>
Exactly, and that's a total screwup. It makes something that would
otherwise be possible -- for some filesystems to have an attribute (call
it "system.dosattrib") which is used, and for others which is stored.
The problem is that with the current design, that won't happen.
Encoding this in the namespace, therefore preventing this kind of
compatiblity, is daft. From the looks of it, the CIFS people were
planning to do the "put everything in user.*" workaround for this design
error.
-hpa
On Mon, 2005-01-03 at 15:22 -0800, H. Peter Anvin wrote:
> [Pruning the Cc: list.]
>
> Nicholas Miell wrote:
> > On Mon, 2005-01-03 at 14:25 -0800, H. Peter Anvin wrote:
> >
> >>I'm honestly not sure that using an ASCII string in an xattr is the sane
> >>way of doing this. Even a binary byte in an xattr would make more sense
> >>in some ways.
> >
> > ASCII strings require no special tools to manipulate from shell scripts.
> >
>
> You need some kind of special tool anyway, i.e. getfattr/setfattr. What
> tool you use isn't really important.
I was talking about getdosattr and setdosattr (and the corresponding
pair for every other filesystem with it's own set of special
attributes).
getfattr and setfattr are standard tools already provided with the
distro.
> The fact that getfattr/setfattr can't deal with attributes that aren't
> ASCII strings seem like flaws in these tools.
They can. Non-ASCII xattrs are either base64 encoded or octal escaped.
Try
getfattr -n system.posix_acl_access some_file_with_an_acl
getfattr -e text -n system.posix_acl_access some_file_with_an_acl
for a quick example.
> > The design does allow users to store whatever they want as an xattr
> > without having to worry about how the kernel chooses to interpret it,
> > though. (i.e. the user namespace is just a byte array that the kernel
> > stores for you, while the system/security namespaces are probably
> > generated and interpreted on demand.)
> >
>
> Exactly, and that's a total screwup. It makes something that would
> otherwise be possible -- for some filesystems to have an attribute (call
> it "system.dosattrib") which is used, and for others which is stored.
> The problem is that with the current design, that won't happen.
>
I responded to this in the other thread already.
> Encoding this in the namespace, therefore preventing this kind of
> compatiblity, is daft. From the looks of it, the CIFS people were
> planning to do the "put everything in user.*" workaround for this design
> error.
>
> -hpa
--
Nicholas Miell <[email protected]>
"H. Peter Anvin" <[email protected]> writes:
> - inode->i_ctime.tv_nsec = de->ctime_ms * 1000000;
> - MSDOS_I(inode)->i_ctime_ms = de->ctime_ms;
> + inode->i_ctime.tv_nsec = de->ctime_ms * 1000000U;
Actually, the ->ctime_ms is not mili seconds. The valid range is 0-199 (*10ms).
(And ->ctime is started from 2 seconds)
> - raw_entry->ctime_ms = MSDOS_I(inode)->i_ctime_ms; /* use i_ctime.tv_nsec? */
> + raw_entry->ctime_ms = inode->i_ctime.tv_nsec / 1000000U;
Ditto
BTW, do you already have any plan to use this ioctls?
Thanks.
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi wrote:
> "H. Peter Anvin" <[email protected]> writes:
>
>
>>- inode->i_ctime.tv_nsec = de->ctime_ms * 1000000;
>>- MSDOS_I(inode)->i_ctime_ms = de->ctime_ms;
>>+ inode->i_ctime.tv_nsec = de->ctime_ms * 1000000U;
>
> Actually, the ->ctime_ms is not mili seconds. The valid range is 0-199 (*10ms).
> (And ->ctime is started from 2 seconds)
>
D'oh! They probably should be renamed _cs (centiseconds ;)
>
>>- raw_entry->ctime_ms = MSDOS_I(inode)->i_ctime_ms; /* use i_ctime.tv_nsec? */
>>+ raw_entry->ctime_ms = inode->i_ctime.tv_nsec / 1000000U;
>
>
> BTW, do you already have any plan to use this ioctls?
>
Yes, I wanted to use them for the syslinux installer.
-hpa
> Nicholas Miell wrote:
> > On another note, NTFS-style xattrs (aka named streams) are unrelated to
> > Linux xattrs. A named stream is a separate file with a funny name, while
> > a Linux xattr is a named extension to struct stat.
This is incorrect. NTFS has two different beasts:
- Extended Attributes (EAs) which are just like Linux xattrs and in
Windows a very similar API exists as in Linux for accessing them (on
NTFS volumes only - VFAT does not support EAs in Windows).
- Named streams for which you are pretty much correct that they are like
a file with a funny name but note that named streams share the same
inode as their parent (unnamed) stream. And this is not just the inode
number, it is the same on-disk inode. This means that the ACLs, dos
attributes, etc, are all valid for both the unnamed and all named
streams. So you cannot for example allow user A to access the unnamed
stream but not some named stream and vice versa you cannot allow user B
to access some named stream but not the unnamed stream. In Windows
these beasts are accessed as filename::namedstreamname and in Linux
there is no existing API to access named streams at all that I am aware
off. Again, as EAs, in Windows named streams only are supported on NTFS
volumes - VFAT does not support them.
The major difference is that access to named streams is much faster on a
per-stream basis as they are separate entities, i.e. stream A and stream
B share the same inode but are otherwise accessed independently and are
stored independently on disk (and as such can be read/written
simultaneously). EAs on the other hand are all linked together and are
in fact just one large blob of data. Resizing one EA modifies
potentially all of them since all EAs are stored in a single byte
stream, not very dissimilar to a named stream in fact. So EAs are more
suited for storage of constant length structures while named streams are
better suited for variable length data (e.g. icons for executables, name
of document author, etc).
One interesting bit of trivia is that Windows uses named streams very
extensively while it _never_ uses EAs. In fact I have never seen a
Windows OS or application that uses EAs. They were added to be
compatible with OS/2 EAs when it came out but since OS/2 died they now
just seem like old baggage/backwards compatibility in Windows that is no
longer used. (If anyone knows of a Windows application that uses EAs
please let me know. I would be most interested in knowing about it!)
Hope this clears things up a bit as far as NTFS is concerned...
I don't know what API would be best for accessing named streams on NTFS
but an xattrs like interface is not suitable IMO. You really want to be
able to open them and access them like normal files. An interface
similar to the Solaris openat() system call (see
http://docs.sun.com/app/docs/doc/816-0212/6m6nd4nc7?a=view) that has
been discussed on LKML before seems like a good way to deal with this
but I am more interested in getting normal write support into NTFS at
present than I am in fancy features like EAs and named streams...
Best regards,
Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
Nicholas Miell wrote:
> Instead of adding another ioctl, wouldn't an xattr be more appropriate?
> For instance, system.fatattrs containing a text representation of the
> attribute bits.
>
> i.e.
> a = archive
Should be the "dump" attribute
> d = directory
It's the file type. Redundant.
> h = hidden
> r = read only
Should be reflected by the write bits. (Maybe there should be an option to
set the file mode for "read only" files to something different than
$rw_mode and not 0222.)
> s = system
Should be made "immutable", IMO
> v = volume
The volume bit of a file will never be set, since no file is a volume name.
Setting a volume attribute should be disallowed, too, since it would cause
fs corruption. This leaves us with no need to know or set 'v'.
You should rather make the volume name be similar to the e2fs volume label.
Followup to: <[email protected]>
By author: Bodo Eggert <[email protected]>
In newsgroup: linux.dev.kernel
>
> > a = archive
>
> Should be the "dump" attribute
>
What dump attribute?
> > h = hidden
> > r = read only
>
> Should be reflected by the write bits. (Maybe there should be an option to
> set the file mode for "read only" files to something different than
> $rw_mode and not 0222.)
It is.
> > s = system
>
> Should be made "immutable", IMO
This is a filesystem mount option, but it's really unpleasant to set
it. It's one of those things that seems to make sense, but really
doesn't.
-hpa
On Tue, 2005-01-04 at 10:09 +0000, Anton Altaparmakov wrote:
> > Nicholas Miell wrote:
> > > On another note, NTFS-style xattrs (aka named streams) are unrelated to
> > > Linux xattrs. A named stream is a separate file with a funny name, while
> > > a Linux xattr is a named extension to struct stat.
>
> This is incorrect. NTFS has two different beasts:
>
[ why this is incorrect omitted for brevity ]
>
> One interesting bit of trivia is that Windows uses named streams very
> extensively while it _never_ uses EAs. In fact I have never seen a
> Windows OS or application that uses EAs. They were added to be
> compatible with OS/2 EAs when it came out but since OS/2 died they now
> just seem like old baggage/backwards compatibility in Windows that is no
> longer used. (If anyone knows of a Windows application that uses EAs
> please let me know. I would be most interested in knowing about it!)
>
This would probably explain why I've never heard of them. In my brief
perusal of the MSDN Library in search of more information about these
beasts, the only hint I could find related to their existence is in
parameters to ZwCreateFile/NtCreateFile (which are themselves mostly
undocumented). The high level file API certainly doesn't support their
use, AFAICT.
I think it's reasonably safe to assume that hpa's worry that FAT might
get EAs in the future is unfounded. (Named streams is still a
possibility, but I don't think Microsoft is all that interested in
making improvements to a filesystem that people use so that they don't
have to license NTFS.)
> Hope this clears things up a bit as far as NTFS is concerned...
>
> I don't know what API would be best for accessing named streams on NTFS
> but an xattrs like interface is not suitable IMO. You really want to be
> able to open them and access them like normal files. An interface
> similar to the Solaris openat() system call (see
> http://docs.sun.com/app/docs/doc/816-0212/6m6nd4nc7?a=view) that has
> been discussed on LKML before seems like a good way to deal with this
> but I am more interested in getting normal write support into NTFS at
> present than I am in fancy features like EAs and named streams...
>
Good luck with that.
> Best regards,
>
> Anton
--
Nicholas Miell <[email protected]>