On Wed, Dec 31, 2003 at 05:42:45PM +0100, R?diger Klaehn wrote:
> Hi everybody.
>
> This is my first post to lkml, so please be patient with me
>
> I have been wondering for some time why there is no decent file change
> notification mechanism in linux. Is there some deep philosophical reason
> for this, or is it just that nobody has found the time to implement it?
> If it is the latter, I am willing to implement it as long there is a
> chance to get this accepted into the mainstream kernel.
>
> Is there already somebody working on this problem? I know a few efforts
> that try to do something similar, but they all work by intercepting
> every single system call that has to do with files, and they are thus
> not very fast. See for example
> <http://www.bangstate.com/software.html#changedfiles>
> <http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/openxdsm/openxdsm/eventmodule/>
>
>
> The dnotify mechanism is very limited since it requires a file handle,
> it is not working for whole directory trees and it does not report much
> useful information. For example to watch for changes in the /home tree
> you would have to open every single directory in the tree, which would
> probably not even work since it would require more than the maximum
> number of file handles. If you have a directory with many files in it,
> the only thing dnotify tells you is that something has changed in the
> directory, so you have to rescan the whole directory to find out which
> file has changed. Kind of defeats the purpose of change notification...
>
> What I would like to have would be some way to watch for certain changes
> anywhere in a whole tree or even the whole file system. This new
> mechanism should have no measurable performance impact and should log
> all information that is readily available. The amount of new code in
> kernel space should be as small as possible. So complicated stuff like
> pattern matching would have to happen in user space.
>
> I wrote some experimental mechanism yesterday. Whenever a file is
> accessed or changed, I write all easily available information to a ring
> buffer which is presented to user space as a device. The information
> that is easily available is the inode number of the file or directory
> that has changed, the inode number of the directory in which the change
> took place, and in most cases the name of the dentry of the file that
> has changed.
hmm...
#ln tree1/sub/dir/file tree2/sub/dir/file
#watch_tree tree1 &
#do_something_to tree2/sub/dir/file
A dnotify can potentially know about open, chown, chmod,
utimes and possibly link of the files by watching the paths
and cwd; meaning it won't know about alternate paths. How
is it to know about read, write, fchown, fchmod and
truncate?
Perhaps someone else has a more fertile imagination but
short of looking up all the file inode numbers of the tree
in question and watching that whole list this sounds futile.
jw schultz wrote:
[snip]
>hmm...
>
>
>#ln tree1/sub/dir/file tree2/sub/dir/file
>#watch_tree tree1 &
>#do_something_to tree2/sub/dir/file
>
>A dnotify can potentially know about open, chown, chmod,
>utimes and possibly link of the files by watching the paths
>and cwd; meaning it won't know about alternate paths. How
>is it to know about read, write, fchown, fchmod and
>truncate?
>
>
>
Take a look at fs/read_write.c. There are calls to dnotify_parent in all
file operation functions. There is a comment in fs/dnotify.c which says
that dnotify_parent is "hopelessly wrong, but unfixable without API
changes". Another good reason for a new file change notification api...
The only thing I am not so sure about is mmap. I think a mmapped file
will not create change notifications.
>Perhaps someone else has a more fertile imagination but
>short of looking up all the file inode numbers of the tree
>in question and watching that whole list this sounds futile.
>
>
Whats wrong with that? You would just have to know the inode numbers of
all directories in the subtree you are interested in. Then you can do a
really fast inode->name translation using a hashtable or something. At
least it is much more lightweight than having to open all directories :-)
I think that it is much cleaner and faster to report the inode numbers
of the changed files since inode numbers are unique per filesystem and
they are immediately available. The complicated mapping of inodes to
path names should happen in user space only for the files the userspace
process is interested in.
On Thu, Jan 01, 2004 at 01:44:01PM +0100, R?diger Klaehn wrote:
> I think that it is much cleaner and faster to report the inode numbers
> of the changed files since inode numbers are unique per filesystem and
> they are immediately available. The complicated mapping of inodes to
Inode number are not necessarily unique per filesystem. Any filesystem
that uses iget4 can have several objects that have the same inode
number. For instance, Coda uses 128-bit file-identifiers and the i_ino
number is a simple hash that is 'typically' unique. There are also
filesystems that invent inode numbers whenever inodes are brought into
the cache, but which have no persistency when the inode_cache is pruned.
So the next time you see the same object, it could have a different
(unique) inode number.
Returning paths is probably also not quite that easy when you only have
the inode. Because of hardlinks we can have several paths (aliases) that
lead to the same object, but the kernel does not necessarily have all of
those paths available in the dcache.
Jan