Andreas Dilger <[email protected]> wrote:

> > STATX_INFO_ENCRYPTED File is encrypted
>
> This flag overlaps with FS_ENCRYPT_FL that is encoded in the FS_IOC_GETFLAGS
> attributes. Are the FS_* flags expected to be translated into STATX_INFO_*
> flags by each filesystem, or will they be partly duplicated in a separate
> "st_attrs" field added in the future?

I think that most of the FS_IOC_GETFLAGS flags are sufficiently specialised
that they aren't something the ordinary user would necessarily find to be of
interest, so I'm not sure that mapping all of them to STATX_INFO_* flags is
necessary.

That said, I think STATX_INFO_ENCRYPTED *is* usefully deployed here to tell
the user that the file or directory is encrypted and that the user will have
to unlock or provide a key to access it.

I'm also thinking that a STATX_INFO_NEED_AUTHENTICATION flag may be needed to
indicate that the user must authenticate in some way (probably only applicable
to network files) to be able to access the file.

David

2016-05-03 20:23:47

by David Howells

[permalink] [raw]

Subject: Re: [PATCH 5/6] statx: Make windows attributes available for CIFS, NTFS and FAT to use

Andreas Dilger <[email protected]> wrote:

> > __u32 st_win_attrs;
>
> It seems some of these flags are duplicated with the st_information field,
> and some are duplicate with FS_IOC_GETFLAGS values, and returning the same
> information in multiple ways is confusing.
>
> If these flags are part of the CIFS protocol,

They're part of the Windows API. CIFS/SMB and NTFS have them at the same
values.

> and are directly usable by Samba then I can understand we wouldn't want to
> change them once in the kernel and then convert them back in userspace, but
> I'm a bit reluctant to have flags only for CIFS/NTFS/FAT that might also be
> useful for other filesystems.

Where said "other filesystems" have some support for Windows or OS/2 and have
bits defined that map directly in semantics, if not in value, to these bits.

JFS, for example, has IREADONLY, IHIDDEN, ISYSTEM, IDIRECTORY, IARCHIVE and
ISPARSE.

> Would we want to be able to get translated
> st_win_attrs flags in ext4 attrs when it is being exported by Samba?

Actually, I was thinking of proposing an ext4 patch that stored windows flags
in a separate word in the inode on disk. There isn't sufficient bit space to
map all the windows flags to FS_IOC_GETFLAGS. However, as you say, there is a
certain amount of overlap...

David

2016-05-03 20:24:42

by David Howells

[permalink] [raw]

Subject: Re: [PATCH 3/6] statx: Ext4: Return enhanced file attributes

Andreas Dilger <[email protected]> wrote:

> > (3) FS_xxx_FL flags are returned as for ioctl(FS_IOC_GETFLAGS), setting
> > STATX_IOC_FLAGS.
>
> I don't see where this is implemented in this patch, and it should be
> removed from the commit message to avoid confusion.

Oops - I forgot to remove it from the other patches.

David

2016-05-04 13:46:36

by Arnd Bergmann

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/6] Enhanced file stat system call

On Friday 29 April 2016 13:57:36 David Howells wrote:
> struct statx *buffer);
>
> This is an enhanced file stat function that provides a number of useful
> features, in summary:
>
> (1) More information: creation time, data version number, inode generation
> number and flags. A subset of these is available through a number of
> filesystems (such as CIFS, NFS, AFS, Ext4 and BTRFS).
>

I have a question about birthtime/creationtime: As we are gaining a way
to read this, should we also provide a way to update it using a new variant
of the utimensat syscall in order to have 'cp -a' create an identical copy,
or is the idea that this is defined as the time that is particular copy
of the inode was created?

I've discussed this with Deepa in the past, as she is driving the
convertion of the inode timestamps to timespec64 now, and we will
need a new version of utimensat for her work as well. I can see good
reasons either way (allowing updates of btime or disallowing them).

This should not hold up the statx syscall from getting merged as
soon as we can, but I'd like to know what everyone feels about that
question now that they are looking at the interface already.

Arnd

2016-05-04 22:56:19

by Dave Chinner

[permalink] [raw]

Subject: Re: [PATCH 1/6] statx: Add a system call to make enhanced file info available

On Fri, Apr 29, 2016 at 01:57:43PM +0100, David Howells wrote:
> (4) File creation time (st_btime*), data version (st_version), inode
> generation number (st_gen).
>
> These will be returned if available whether the caller asked for them or
> not. The corresponding bits in st_mask will be set or cleared as
> appropriate to indicate a valid value.

IMO, exposing the inode generation number to anyone is a potential
security problem because they are used in file handles.

Most file handles provided by filesystems are simply an encoding of
the inode number + generation number, plus maybe the ino+gen of the
parent dir if the NFS server is configured to do this. This makes it
trivial for an attacker to guess what the likely generation numbers
are going to be for inode numbers surrounding any given inode, hence
greatly reducing the search space for guessing valid file handles.

We've known this to be a problem for a long time - file handles are
not cryptographically secure, so exposing information like this by
default make guessing handles successfully almost trivial for many
filesystems.

In the latest XFS filesystem format, we randomise the generation
value during every inode allocation to make it hard to guess the
handle of adjacent inodes from an existing ino+gen pair, or even
from life time to life time of the same inode. We don't use a secure
random number generator (prandom_u32()) so it's still possible to
guess with enough trial and observation. However, it makes it
several orders of magnitude harder to guess and requires knowledge
of inode allocation order to guess correctly once the random number
sequence has been deduced and so makes brute force the only real
option for guessing a valid handle for an inode.

However, this is definitely a problem for the older format where
each cluster of inodes was initialised with the same seed at cluster
allocation time and the generation number was simply incremented for
each life time. Most filesystems use a similar method for seeding
and incrementing generation numbers, so once the generation numbers
are exposed it makes handles trivial to calculate successfully.

> If the caller didn't ask for them, then they may be approximated. For
> example, NFS won't waste any time updating them from the server, unless
> as a byproduct of updating something requested.

I would suggest that exposing them from the NFS server is something
we most definitely don't want to do because they are the only thing
that keeps remote users from guessing filehandles with ease....

Cheers,

Dave.
--
Dave Chinner
[email protected]

2016-05-04 23:57:17

by NeilBrown

[permalink] [raw]

Subject: Re: [PATCH 1/6] statx: Add a system call to make enhanced file info available

On Fri, Apr 29 2016, David Howells wrote:

> Add a system call to make extended file information available, including
> file creation time, inode version and data version where available through
> the underlying filesystem.
>
>
> ========
> OVERVIEW
> ========

I think all this documentation is invaluable - thanks.
I would really like to see much of it in
Documentation/filesystems/something.txt
rather than just in the commit log.

>
> The defined bits in the st_information field give local system data on a
> file, how it is accessed, where it is and what it does:

These bits form a channel for communication between the filesystem
developer and the application writer. As such we should be sure that
channel actually communicates meaning...

>
> STATX_INFO_ENCRYPTED File is encrypted
> STATX_INFO_TEMPORARY File is temporary

What is "temporary"? Is it a statement about quality of storage
technology (will be destroyed by reboot) or intention of creator
(created with O_TMPFILE) or something else?

> STATX_INFO_FABRICATED File was made up by filesystem
> STATX_INFO_KERNEL_API File is kernel API (eg: procfs/sysfs)

What is the difference between these two? Both are synthesized by the
kernel.
Maybe the "KERNEL_API" is declared never to change its meaning, while the
fabricated one doesn't make a "stable API" promise?

What is the difference between fabricating a file from a bunch of blocks
spread over a storage device, and fabricating a file from a single field
in the super-block?

> STATX_INFO_REMOTE File is remote

How far is "remote"? Does Infiniband count? Fibre channel? iSCSI?
Is a file on a loop-back mounted NFS filesystem more remote than a
fibre-channel connection to the next town?

Or is this relative? Within a filesystem there are "remote" files and
"non-remote" files and the distinction is filesystem-dependant??

> STATX_INFO_AUTOMOUNT Dir is automount trigger
> STATX_INFO_AUTODIR Dir provides unlisted automounts

I think this last one means that there are names in the directory which
may not appear in "readdir" but will respond to "stat". I would prefer
the description to match the behavior without necessarily implying that
those names will be automounts. e.g
STATX_INFO_INCOMPLETE_READDIR getdents may not report all
names that respond to stat

> STATX_INFO_NONSYSTEM_OWNERSHIP File has non-system ownership details

This probably is a well defined meaning that I just don't have the
context to understand. For me, more words would help here.

I don't object to any of these flag. I just want to be sure that I
understand them.
I am generally in favour this functionality going in promptly.

Thanks,
NeilBrown

Attachments:

signature.asc (818.00 B)

2016-05-05 00:10:03

On Mon, May 2, 2016 at 5:52 PM, Andreas Dilger <[email protected]> wrote:
> On Apr 29, 2016, at 6:58 AM, David Howells <[email protected]> wrote:
>>
>> Make windows attributes available for CIFS, NTFS and FAT to use in the
>> statx struct. The attribute flags map directly by value to those in the
>> CIFS PDU flags. Some of these bits can also be used by JFS, UFS and HPFS.
>>
>> The statx struct acquires:
>>
>> __u32 st_win_attrs;
>
> It seems some of these flags are duplicated with the st_information field,
> and some are duplicate with FS_IOC_GETFLAGS values, and returning the same
> information in multiple ways is confusing.
>
> If these flags are part of the CIFS protocol, and are directly usable by
> Samba then I can understand we wouldn't want to change them once in the
> kernel and then convert them back in userspace, but I'm a bit reluctant
> to have flags only for CIFS/NTFS/FAT that might also be useful for other filesystems. Would we want to be able to get translated st_win_attrs
> flags in ext4 attrs when it is being exported by Samba?

It could be useful to have them in statx in an fs neutral way (and
could be implemented by CIFS/NTFS/FAT and probably NFS), but I am ok
with doing this via file system specific xattrs as we do now: NTFS has
an ntfs specific pseudo-xattr for these and for birth time (and now
cifs with recent patches can return the SMB3 inode attributes,
including the older DOS attributes noted below). Samba server also
stores them in an xattr (although they ndr encode them so it is not
useful to us in the kernel) in local file systems. NFS v4 (and later)
can also return (in theory) some of these flags but they are optional.

>> The value in this is present if STATX_WIN_ATTRS is set.
>>
>> The defined flags in this are:
>>
>> STATX_WIN_ATTR_READONLY
>> STATX_WIN_ATTR_HIDDEN
>> STATX_WIN_ATTR_SYSTEM
>> STATX_WIN_ATTR_DIRECTORY
>
> How does this differ from (st_mode & S_IFMT) == S_IFDIR)?
>
>> STATX_WIN_ATTR_ARCHIVE
>> STATX_WIN_ATTR_NORMAL
>
> How does this differ from (st_mode & S_IFMT) == S_IFREG)?
>
>> STATX_WIN_ATTR_TEMPORARY
>
> How does this differ from STATX_INFO_TEMPORARY?
>
>> STATX_WIN_ATTR_SPARSE_FILE
>> STATX_WIN_ATTR_REPARSE_POINT
>> STATX_WIN_ATTR_COMPRESSED
>> STATX_WIN_ATTR_OFFLINE
>> STATX_WIN_ATTR_NOT_CONTENT_INDEXED
>> STATX_WIN_ATTR_ENCRYPTED
>
> How does this differ from STATX_INFO_ENCRYPTED?

As you noted, directory, compressed, encrypted overlap with existing
posix file type (directory vs. file) and FS_IOC_FLAGS (although
cifs.ko does not allow you to set encrypted yet as we do with
compressed through the IOC flags, I could add a patch for setting the
encrypted flag since it should be trivial as it does not affect
reads/writes/open/close over the network fs since the file would be
only encrypted at rest on the server fs)

--
Thanks,

Steve