On Wed, May 20, 2009 at 11:05:27AM +0200, Miklos Szeredi wrote:
> On Tue, 19 May 2009, Valerie Aurora wrote:
> > As Jan said, readdir() of read-only unioned file systems works with a
> > tmpfs top layer. If you think about it, this is the exact equivalent
> > of the version of union mounts which used the in-kernel caching
> > approach - except that it's better, because it reuses existing code
> > and caches between readdir() calls. Cool, huh?
>
> Yeah... OTOH tmpfs is probably a way too heavyweight solution for
> cases where memory is short, and union mounts would typically be used
> on such systems.
(Sorry for the delay - I've been on vacation.)
Hm, my intuition is that a tmpfs mount would be fairly lightweight in
terms of memory - the main overhead over the barebones solution would
be one superblock and vfsmount struct per mount. What am I missing?
> The big reason why kernel impementation of readdir is hard is that
> unswappable kernel memory needs to be used for caching directory
> contents while the directory is open. Well, tmpfs does the same,
> dentries and inodes are _not_ swappable, and they gobble up memory.
That's a good point. It seemed to me that it wouldn't be too
difficult to make those entries evictable - drop a reference count and
set the ->d_release to mark the directory as needing rebuilding. What
do you think?
> So where's the advantage over implementing a thin deduplicating and
> caching layer for union mounts?
>
> Thanks,
> Miklos
-VAL
On Mon, 8 Jun 2009, Valerie Aurora wrote:
> On Wed, May 20, 2009 at 11:05:27AM +0200, Miklos Szeredi wrote:
> > The big reason why kernel impementation of readdir is hard is that
> > unswappable kernel memory needs to be used for caching directory
> > contents while the directory is open. Well, tmpfs does the same,
> > dentries and inodes are _not_ swappable, and they gobble up memory.
>
> That's a good point. It seemed to me that it wouldn't be too
> difficult to make those entries evictable - drop a reference count and
> set the ->d_release to mark the directory as needing rebuilding. What
> do you think?
AFAICS, there are nontrivial problems to deal with:
If directory is still open, child dentries must not go away.
If directory is closed, and at least one child is evicted, then the
whole directory is unusable and needs to be rebuilt on next readdir.
If we can solve those in a non-racy way than it might work. I suspect
however, that some additional code in union-mounts that adds all this
functionality without reusing tmpfs would actually be simpler to
implement.
Thanks,
Miklos