Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752229AbaDDHf7 (ORCPT ); Fri, 4 Apr 2014 03:35:59 -0400 Received: from mail-bk0-f44.google.com ([209.85.214.44]:60225 "EHLO mail-bk0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751382AbaDDHfz (ORCPT ); Fri, 4 Apr 2014 03:35:55 -0400 Message-ID: <533E60D6.2000704@gmail.com> Date: Fri, 04 Apr 2014 09:35:50 +0200 From: "Michael Kerrisk (man-pages)" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Jan Kara CC: mtk.manpages@gmail.com, John McCutchan , Robert Love , Eric Paris , Lennart Poettering , radu.voicilas@gmail.com, daniel@veillard.com, Christoph Hellwig , Vegard Nossum , "linux-fsdevel@vger.kernel.org" , linux-man , gamin-list@gnome.org, lkml , inotify-tools-general@lists.sourceforge.net Subject: Re: Things I wish I'd known about Inotify References: <20140403205236.GE14107@quack.suse.cz> In-Reply-To: <20140403205236.GE14107@quack.suse.cz> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/03/2014 10:52 PM, Jan Kara wrote: > On Thu 03-04-14 08:34:44, Michael Kerrisk (man-pages) wrote: >> Limitations and caveats >> The inotify API provides no information about the user or process >> that triggered the inotify event. In particular, there is no >> easy way for a process that is monitoring events via inotify to >> distinguish events that it triggers itself from those that are >> triggered by other processes. >> >> The inotify API identifies affected files by filename. However, >> by the time an application processes an inotify event, the file‐ >> name may already have been deleted or renamed. >> >> The inotify API identifies events via watch descriptors. It is >> the application's responsibility to cache a mapping (if one is >> needed) between watch descriptors and pathnames. Be aware that >> directory renamings may affect multiple cached pathnames. >> >> Inotify monitoring of directories is not recursive: to monitor >> subdirectories under a directory, additional watches must be cre‐ >> ated. This can take a significant amount time for large direc‐ >> tory trees. > And also there's a problem with the limit on the number of watches a user > can have. What is the problem exactly (given that the limit is configurable)? >> If monitoring an entire directory subtree, and a new subdirectory >> is created in that tree or an existing directory is renamed into >> that tree, be aware that by the time you create a watch for the >> new subdirectory, new files (and subdirectories) may already >> exist inside the subdirectory. Therefore, you may want to scan >> the contents of the subdirectory immediately after adding the >> watch (and, if desired, recursively add watches for any subdirec‐ >> tories that it contains). >> >> Note that the event queue can overflow. In this case, events are >> lost. Robust applications should handle the possibility of lost >> events gracefully. For example, it may be necessary to rebuild >> part or all of the application cache. (One simple, but possibly >> expensive, approach is to close the inotify file descriptor, >> empty the cache, create a new inotify file descriptor, and then >> re-create watches and cache entries for the objects to be moni‐ >> tored.) >> >> Dealing with rename() events >> The IN_MOVED_FROM and IN_MOVED_TO events that are generated by >> rename(2) are usually available as consecutive events when read‐ >> ing from the inotify file descriptor. However, this is not guar‐ >> anteed. If multiple processes are triggering events for moni‐ >> tored objects, then (on rare occasions) an arbitrary number of >> other events may appear between the IN_MOVED_FROM and IN_MOVED_TO >> events. >> >> Matching up the IN_MOVED_FROM and IN_MOVED_TO event pair gener‐ >> ated by rename(2) is thus inherently racy. (Don't forget that if >> an object is renamed outside of a monitored directory, there may >> not even be an IN_MOVED_TO event.) Heuristic approaches (e.g., >> assume the events are always consecutive) can be used to ensure a >> match in most cases, but will inevitably miss some cases, causing >> the application to perceive the IN_MOVED_FROM and IN_MOVED_TO >> events as being unrelated. If watch descriptors are destroyed >> and re-created as a result, then those watch descriptors will be >> inconsistent with the watch descriptors in any pending events. >> (Re-creating the inotify file descriptor and rebuilding the cache >> may be useful to deal with this scenario.) > Well, but there's 'cookie' value meant exactly for matching up > IN_MOVED_FROM and IN_MOVED_TO events. And 'cookie' is guaranteed to be > unique at least within the inotify instance (in fact currently it is unique > within the whole system but I don't think we want to give that promise). Yes, that's already assumed by my discussion above (its described elsewhere in the page). But your comment makes me think I should add a few words to remind the reader of that fact. I'll do that. But, the point is that even with the cookie, matching the events is nontrivial, since: * There may not even be an IN_MOVED_FROM event * There may be an arbitrary number of other events in between the IN_MOVED_FROM and the IN_MOVED_TO. Therefore, one has to use heuristic approaches such as "allow at least N millisconds" or "check the next N events" to see if there is an IN_MOVED_FROM that matches the IN_MOVED_TO. I can't see any way around that being inherently racy. (It's unfortunate that the kernel can't provide a guarantee that the two events are always consecutive, since that would simply user space's life considerably.) Cheers, Michael >> Applications should also allow for the possibility that the >> IN_MOVED_FROM event was the last event that could fit in the buf‐ >> fer returned by the current call to read(2), and the accompanying >> IN_MOVED_TO event might be fetched only on the next read(2). > > Honza > -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/