2005-02-10 18:06:58

by John McCutchan

[permalink] [raw]
Subject: [ANNOUNCE] inotify 0.19

Announcing release 0.19.0 of inotify.

Changes in this release include:

- Rename event.filename to event.name (breaks api)
- Rename request.dirname to request.name (breaks api)
- Add Mike's patch to fix unmount race
- Remove watch_mask and all the associated code.
- Don't check the mask in inotify_dev_queue_event()
- Adding Robert Love to Authors list
- Update Copyright to 2005
- Check the mask in the one place we actually need to.
- Break ignore_helper up, adding a lockless version
- Remove the event parameter from ignore_helper, handle that
one case as needed, separately
- Rename ignore_helper to remove_watch
- More comments!

Attached is a patch for 2.6.10
Attached is a patch for gamin CVS
Attached is an updated inotify-utils

Or,
http://tentacle.dhs.org/inotify/

And for even more patches,
ftp://ftp.kernel.org/pub/linux/kernel/people/rml/inotify

John McCutchan

Inotify documentation:

--HOWTO USE--
Inotify is a character device that when opened offers 2 IOCTL's.

INOTIFY_WATCH:
Which takes a path and event mask and returns a unique
(to the instance of the driver) integer (wd [watch descriptor]
from here on) that is a 1:1 mapping to the path passed.
What happens is inotify refs the inode for the path and
adds a inotify_watch structure to the inode. Each
inotify device instance can only watch an inode once.

INOTIFY_IGNORE:
Which takes an integer (that you got from INOTIFY_WATCH)
representing a wd that you are not interested in watching
anymore. This will:

send an IGNORE event to the device
remove the inotify_watch structure from the device and
from the inode and unref the inode.

After you are watching 1 or more paths, you can read from the fd
and get events. The events are struct inotify_event. If you are
watching a directory and something happens to a file in the directory
the event will contain the filename (just the filename not the full
path).

-- EVENTS --
IN_ACCESS - Sent when file is accessed.
IN_MODIFY - Sent when file is modified.
IN_ATTRIB - Sent when file is chmod'ed.
IN_CLOSE_WRITE - Sent when file is closed and was opened for writing.
IN_CLOSE_NOWRITE - Sent when file is closed but was not opened for
writing.
IN_CLOSE - either of the two above events.
IN_OPEN - Sent when file is opened.
IN_MOVED_FROM - Sent to the source folder of a move.
IN_MOVED_TO - Sent to the destination folder of a move.
IN_DELETE_SUBDIR - Sent when a sub directory is deleted.(When watching
parent)
IN_DELETE_FILE - Sent when a file is deleted. (When watching parent)
IN_CREATE_SUBDIR - Sent when a sub directory is created.(When watching
parent)
IN_CREATE_FILE - Sent when a file is created. (When watching parent)
IN_DELETE_SELF - Sent when file is deleted.
IN_UNMOUNT - Sent when the filesystem is being unmounted.
IN_Q_OVERFLOW - Sent when your event queue has over flowed.

The MOVED_FROM/MOVED_TO events are always sent in pairs.
MOVED_FROM/MOVED_TO
is also sent when a file is renamed. The cookie field in the event
structure
allows you to pair MOVED_FROM/MOVED_TO events. These two events are not
guaranteed to be successive in the event stream. You must rely on the
cookie to pair them up.

If you aren't watching the source and destination folders of a
move/rename
You will only get MOVED_TO or MOVED_FROM. In this case, MOVED_TO
is equivelent to a CREATE and MOVED_FROM is equivelent to a DELETE.

--Why Not dnotify and Why inotify (By Robert Love)--

Everyone seems quick to deride the blunder known as "dnotify" and
applaud a
replacement, any replacement, man anything but that current mess, but in
the
name of fairness I present my treatise on why dnotify is what one might
call
not good:

* dnotify requires the opening of one fd per each directory that you
intend to
watch.
o The file descriptor pins the directory, disallowing the
backing device to be unmounted, which absolutely wrecks
havoc with removable media.
o Watching many directories results in many open file
descriptors, possibly hitting a per-process fd limit.
* dnotify is directory-based. You only learn about changes to
directories. Sure, a change to a file in a directory affects the
directory, but you are then forced to keep a cache of stat structures
around to compare things in order to find out which file.
* dnotify's interface to user-space is awful.
o dnotify uses signals to communicate with user-space.
o Specifically, dnotify uses SIGIO.
o But then you can pick a different signal! So by "signals," I really
meant you need to use real-time signals if you want to queue the
events.
* dnotify basically ignores any problems that would arise in the VFS
from hard
links.
* Rumor is that the "d" in "dnotify" does not stand for "directory"
but for "suck."

A suitable replacement is "inotify." And now, my tract on what inotify
brings
to the table:

* inotify's interface is a device node, not SIGIO.
o You open only a single fd, to the device node. No more pinning
directories or opening a million file descriptors.
o Usage is nice: open the device, issue simple commands via ioctl(),
and then block on the device. It returns events when, well, there are
events to be returned.
o You can select() on the device node and so it integrates with main
loops like coffee mixed with vanilla milkshake.
* inotify has an event that says "the filesystem that the item you were
watching is on was unmounted" (this is particularly cool).
* inotify can watch directories or files.
* The "i" in inotify does not stand for "suck" but for "inode" -- the
logical
choice since inotify is inode-based.

--COMPLEXITY--

I have been asked what the complexity of inotify is. Inotify has
2 path codes where complexity could be an issue:

Adding a watch to a device
This code has to check if the inode is already being watched
by the device, this is O(N) where N is the number of devices
watching the inode

Removing a watch from a device
This code
This code has to do a search of all watches on the device to
find the watch descriptor that is being asked to remove.
This involves a linear search, but should not really be an issue
because it is limited to 8192 entries. If this does turn in to
a concern, I would replace the list of watches on the device
with a sorted binary tree, so that the search could be done
very quickly.

In the near future this will be based on idr, and the complexity
will be O(logn)

The calls to inotify from the VFS code has a complexity of O(1).

--MEMORY USAGE--

The inotify data structures are light weight:

**NOTE** The memory analysis is out of date as of dynamic filename
lengths

inotify watch is 40 bytes
inotify device is 68 bytes
inotify event is 272 bytes

So assuming a device has 8192 watches, the structures are only going
to consume 320KB of memory. With a maximum number of 8 devices allowed
to exist at a time, this is still only 2.5 MB

Each device can also have 256 events queued at a time, which sums to
68KB per device. And only .5 MB if all devices are opened and have
a full event queue.

So approximately 3 MB of memory are used in the rare case of
everything open and full.

Each inotify watch pins the inode of a directory/file in memory,
the size of an inode is different per file system but lets assume
that it is 512 byes.

So assuming the maximum number of global watches are active, this would
pin down 32 MB of inodes in the inode cache. Again not a problem
on a modern system.

On smaller systems, the maximum watches / events could be lowered
to provide a smaller foot print.

Keep in mind that this is an absolute worst case memory analysis.
In reality it will most likely cost approximately 5MB.

--KERNEL CHANGES--
inotify char device driver.

Adding calls to inotify_inode_queue_event and
inotify_dentry_parent_queue_event from VFS operations.
Dnotify has the same function calls.

Adding a call to inotify_super_block_umount from
generic_shutdown_superblock

inotify_super_block_umount:
find all of the inodes that are on the super block being shut down
foreach of these inodes:
remove the watchers on them
send UNMOUNT and IGNORED events
unref the inode


Attachments:
inotify-0.19.patch (53.87 kB)
gamin-inotify-0.19.patch (4.26 kB)
inotify-utils-0.19.tar.gz (3.41 kB)
Download all attachments