Hi!
> | You do need extra tools anyway, placing them in the kernel is cheating
> (and
> | absolutely pointless, IMHO).
>
> Repeat after me: plugins in kernel does NOT equal tools in kernel.
>
> It's not hard to, for instance, imagine a generic plugin for archive
> manipulation which talks to a userspace daemon/library. The kernel
> doesn't know anything other than (maybe) a list of extensions that are
> archives. All else is handled in userspace -- the idea that "this is a
> zipfile" and "zipfiles can be extracted with the 'zip' command" are all
> in userspace.
>
> It's not about the kernel, it's about the interface. And see my other mail:
> cat foo.zip/README
> less foo.zip/contents/bar.c
> is a lot easier than
> lynx http://google.com/search?q=zip
> emerge zip
> man zip
> unzip foo.zip
> cat bar.c
> which already assumes quite a lot of expertise.
If you only want nice user interface, you can have that today. Its
done using coda, and hosted at uservfs.sf.net.
Pavel
--
When do you have heart between your knees?
Pavel Machek wrote:
> > It's not about the kernel, it's about the interface. And see my other mail:
> > cat foo.zip/README
> > less foo.zip/contents/bar.c
> > is a lot easier than
> > lynx http://google.com/search?q=zip
> > emerge zip
> > man zip
> > unzip foo.zip
> > cat bar.c
> > which already assumes quite a lot of expertise.
>
> If you only want nice user interface, you can have that today. Its
> done using coda, and hosted at uservfs.sf.net.
Can I do these already using uservfs?
cd
less foo.zip/contents/bar.c
less /usr/portage/distfiles/perl-5.8.5.tar.gz/contents/toke.c
grep -R obscure_label ~/RPMS
No I can't.
Even using the right "#" names for uservfs ('foo.zip#zip' etc.), I can
only do the above with convenient paths, completion etc. if I mount my
home directory and distfiles over uservfs, so that all I/O in those
directories including ordinary accesses is through uservfs, and have
the _real_ storage elsewhere. That's quite illogical, cumbersome to
set up, and slows down normal I/O significantly.
I can do some of those things in a similar way if I do them inside
Midnight Commander, Gnome, KDE, or Emacs -- but even then a path that
works in one of those doesn't work in another, and the extensions
don't even work in all programs, plugins or scripts for those systems.
As we're adding file-as-directories to VFS, I propose it would be good
to have *hooks* in place, and non-metadata namespace reserved inside
the directories, so that uservfs-like userspace filesystems can
be auto-mounted at those places and synchronised with the file.
In other words, something that makes the next version of uservfs a
whole lot more useful.
The good news is that a lot of the hooks are falling into place already.
Kernel support is needed to handle the interface, cacheing to trigger
deletions, page cacheing so that I/O is a normal speed and mmap()
works, moveable mounts so that renames work and the cached data
followed them, and coherency so that if the underlying archive is
changed, it invalidates cached data with the appropriate degree of
synchronisation. Note that the current dnotify cannot provide the
latter with any strength; it's too weak an interface (so is inotify).
(You can omit a lot of these things, but then you get a flaky interface).
The FUSE project looks like it will provide an efficient interface to
the page cache, avoiding round trips where possible and supporting
mmap. If it doesn't, it should. (Similar VFS facilities are useful
for distributed filesystems). The presently discussed file-as-directory
semantics will provide a nice view, and force the necessary minimal
tool changes, so _all_ tools will be fine with the new uservfs.
So all I am asking for is a facility to auto-mount with
file-as-directory, and the ability for a userspace daemon to be
notified of regular file modifications synchronously. Both can be
added later, once file-as-directory and moveable mounts are
established. (fcntl leases and dnotify _almost_ provide this, but
they don't. Looks like incoherent hacks keep getting added all over
the place for Samba... :).
That's all. Kernel space needs to provide very little that it doesn't
already provide, for this rather nice and universally usable feature to work.
-- Jamie
Pavel Machek wrote:
> > Can I do these already using uservfs?
> >
> > cd
> > less foo.zip/contents/bar.c
>
> less foo.zip#uzip/contents/bar.c
>
> > less /usr/portage/distfiles/perl-5.8.5.tar.gz/contents/toke.c
>
> less /usr/portage/distfiles/perl-5.8.5.tar.gz#utar/contents/toke.c
>
> > grep -R obscure_label ~/RPMS
>
> I do not think you'd want this. How would you tell grep obscure label
> without entering archives, then?
Granted, this is where we need "grep --recurse-into-files" or, more
generally useful, a find option.
> > No I can't.
>
> > Even using the right "#" names for uservfs ('foo.zip#zip' etc.), I can
> > only do the above with convenient paths, completion etc. if I mount
> > my
>
> Well, you can... or at least I had version somewhere that
> automagically prepended /overlay when file was not found. That way,
> normal requests were fast and uservfs were used whenever it was
> needed... At the price of little uglyness and slowing down -ENOENTs.
There you go... a kernel hook to prepend /overlay works :)
These hook I keep mentioning, they are really very simple :)
> > So all I am asking for is a facility to auto-mount with
> > file-as-directory, and the ability for a userspace daemon to be
> > notified of regular file modifications synchronously. Both can be
> > added later, once file-as-directory and moveable mounts are
>
> If the userspace daemon is synchronously notified of file
> modification... will not that lead to very ugly deadlocks?
Not if it's written properly.
-- Jamie
On Wed, 2004-09-01 at 17:59, Jamie Lokier wrote:
> Pavel Machek wrote:
> > > Can I do these already using uservfs?
> > >
> > > cd
> > > less foo.zip/contents/bar.c
> >
> > less foo.zip#uzip/contents/bar.c
> >
> > > less /usr/portage/distfiles/perl-5.8.5.tar.gz/contents/toke.c
> >
> > less /usr/portage/distfiles/perl-5.8.5.tar.gz#utar/contents/toke.c
> >
> > > grep -R obscure_label ~/RPMS
> >
> > I do not think you'd want this. How would you tell grep obscure label
> > without entering archives, then?
>
> Granted, this is where we need "grep --recurse-into-files" or, more
> generally useful, a find option.
>
FWIW, this is how Windows does it now. As of XP, 'Find files' has an
option, enabled by default, to look inside archives. If you tell it to
look for a driver in a given directory it will also look inside .cab
and .zip files. It's extremely useful, I would imagine someone who uses
XP a lot will come to expect this feature.
Of course, no idea how it's implemented, but users like it.
Lee
Hi!
> > > It's not about the kernel, it's about the interface. And see my other mail:
> > > cat foo.zip/README
> > > less foo.zip/contents/bar.c
> > > is a lot easier than
> > > lynx http://google.com/search?q=zip
> > > emerge zip
> > > man zip
> > > unzip foo.zip
> > > cat bar.c
> > > which already assumes quite a lot of expertise.
> >
> > If you only want nice user interface, you can have that today. Its
> > done using coda, and hosted at uservfs.sf.net.
>
> Can I do these already using uservfs?
>
> cd
> less foo.zip/contents/bar.c
less foo.zip#uzip/contents/bar.c
> less /usr/portage/distfiles/perl-5.8.5.tar.gz/contents/toke.c
less /usr/portage/distfiles/perl-5.8.5.tar.gz#utar/contents/toke.c
> grep -R obscure_label ~/RPMS
I do not think you'd want this. How would you tell grep obscure label
without entering archives, then?
> No I can't.
> Even using the right "#" names for uservfs ('foo.zip#zip' etc.), I can
> only do the above with convenient paths, completion etc. if I mount
> my
Well, you can... or at least I had version somewhere that
automagically prepended /overlay when file was not found. That way,
normal requests were fast and uservfs were used whenever it was
needed... At the price of little uglyness and slowing down -ENOENTs.
> As we're adding file-as-directories to VFS, I propose it would be good
> to have *hooks* in place, and non-metadata namespace reserved inside
> the directories, so that uservfs-like userspace filesystems can
> be auto-mounted at those places and synchronised with the file.
>
> In other words, something that makes the next version of uservfs a
> whole lot more useful.
Yep, something less hacky would be nice.
> So all I am asking for is a facility to auto-mount with
> file-as-directory, and the ability for a userspace daemon to be
> notified of regular file modifications synchronously. Both can be
> added later, once file-as-directory and moveable mounts are
If the userspace daemon is synchronously notified of file
modification... will not that lead to very ugly deadlocks?
Pavel
--
Horseback riding is like software...
...vgf orggre jura vgf serr.
On Wed, 2004-09-01 at 17:14 +0100, Jamie Lokier wrote:
> So all I am asking for is a facility to auto-mount with
> file-as-directory, and the ability for a userspace daemon to be
> notified of regular file modifications synchronously. Both can be
> added later, once file-as-directory and moveable mounts are
> established. (fcntl leases and dnotify _almost_ provide this, but
> they don't. Looks like incoherent hacks keep getting added all over
> the place for Samba... :).
How will synchronous file modification notifications be handled?
Programs listening for the notification will want to see the changes
when they read the file but the reason this is synchronous is to prevent
other applications seeing the changes before they are reflected in these
indexing programs etc.. right?
--
// Gianni Tedesco (gianni at scaramanga dot co dot uk)
lynx --source http://www.scaramanga.co.uk/scaramanga.asc | gpg --import
8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D
Salut,
On Wed, Sep 01, 2004 at 06:51:12PM -0400, Lee Revell wrote:
> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> option, enabled by default, to look inside archives. If you tell it to
> look for a driver in a given directory it will also look inside .cab
> and .zip files. It's extremely useful, I would imagine someone who uses
> XP a lot will come to expect this feature.
>
> Of course, no idea how it's implemented, but users like it.
The trick is that their tools are aware of it, and that the library
the programs use supports on-the-fly decompression.
Same applies to Spotlight for Tiger.
Tonnerre