Lee Revell <[email protected]> said:
[...]
> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> option, enabled by default, to look inside archives. If you tell it to
> look for a driver in a given directory it will also look inside .cab
> and .zip files. It's extremely useful, I would imagine someone who uses
> XP a lot will come to expect this feature.
It is trivial to implement this by looking inside the files. I.e., the way
mc has done this for ages.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Thu, 2004-09-02 at 10:25, Horst von Brand wrote:
> Lee Revell <[email protected]> said:
>
> [...]
>
> > FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> > option, enabled by default, to look inside archives. If you tell it to
> > look for a driver in a given directory it will also look inside .cab
> > and .zip files. It's extremely useful, I would imagine someone who uses
> > XP a lot will come to expect this feature.
>
> It is trivial to implement this by looking inside the files. I.e., the way
> mc has done this for ages.
This requires a separate, MC-specific namespace. The point is to unify
the namespace, not fragment it.
If Hans had a comprehensible web page, maybe more people would
understand this aspect of his argument. Would you put digressions about
chaos theory and King Arthur in a man page?
Lee
> Lee Revell <[email protected]> said:
> [...]
>> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
>> option, enabled by default, to look inside archives. If you tell it to
>> look for a driver in a given directory it will also look inside .cab
>> and .zip files. It's extremely useful, I would imagine someone who uses
>> XP a lot will come to expect this feature.
> It is trivial to implement this by looking inside the files. I.e., the way
> mc has done this for ages.
Difference is that you can't do "locate" or "find" or "Search".. You
would have to open the files in an archive-supporting application
such as mc.
Hi!
> >> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> >> option, enabled by default, to look inside archives. If you tell it to
> >> look for a driver in a given directory it will also look inside .cab
> >> and .zip files. It's extremely useful, I would imagine someone who uses
> >> XP a lot will come to expect this feature.
>
> > It is trivial to implement this by looking inside the files. I.e., the way
> > mc has done this for ages.
>
> Difference is that you can't do "locate" or "find" or "Search".. You
> would have to open the files in an archive-supporting application
> such as mc.
You really need archive support in find. At the very least you need
option "enter archives" vs. "do not enter archives". Entering archives
automagically is seriously wrong.
Pavel
On Iau, 2004-09-02 at 20:41, Spam wrote:
> > It is trivial to implement this by looking inside the files. I.e., the way
> > mc has done this for ages.
>
> Difference is that you can't do "locate" or "find" or "Search".. You
> would have to open the files in an archive-supporting application
> such as mc.
And would you rather that logic was running swappable in shared library
space or privileged and unswappable in kernel ?
Alan
On Thu, 2004-09-02 at 15:49, Pavel Machek wrote:
> Hi!
>
> > >> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> > >> option, enabled by default, to look inside archives. If you tell it to
> > >> look for a driver in a given directory it will also look inside .cab
> > >> and .zip files. It's extremely useful, I would imagine someone who uses
> > >> XP a lot will come to expect this feature.
> >
> > > It is trivial to implement this by looking inside the files. I.e., the way
> > > mc has done this for ages.
> >
> > Difference is that you can't do "locate" or "find" or "Search".. You
> > would have to open the files in an archive-supporting application
> > such as mc.
>
> You really need archive support in find. At the very least you need
> option "enter archives" vs. "do not enter archives". Entering archives
> automagically is seriously wrong.
But is it efficient to make every application that reads files have to
know how to get inside a tar file, just to read its contents? That
seems like a massive duplication of effort. Better to have the contents
accessible via a separate stream, in the same namespace. Fix it once in
the kernel vs. fix it in umpteen apps.
The key point here is that the expressive power of the system is greatly
reduced by having a fragmented namespace. Of course there are any
number of ways for an app to find out what is in a tar file, but
exporting all of that information in a unified namespace is nontrivial
and much more interesting.
Lee
On Thu, Sep 02, 2004 at 04:01:17PM -0400, Lee Revell wrote:
> Better to have the contents accessible via a separate stream, in the
> same namespace. Fix it once in the kernel vs. fix it in umpteen
> apps.
This is ridiculous. We have shared libraries, 99% of applications
manage to use libc for example --- this isn't that different.
--cw
> On Iau, 2004-09-02 at 20:41, Spam wrote:
>> > It is trivial to implement this by looking inside the files. I.e., the way
>> > mc has done this for ages.
>>
>> Difference is that you can't do "locate" or "find" or "Search".. You
>> would have to open the files in an archive-supporting application
>> such as mc.
> And would you rather that logic was running swappable in shared library
> space or privileged and unswappable in kernel ?
I would rather have it as a filesystem/vfs plugin that would allow
all my programs to use the features the plugin gives, even if it
would mean being totally in kernel space.
> Alan
> On Thu, Sep 02, 2004 at 04:01:17PM -0400, Lee Revell wrote:
>> Better to have the contents accessible via a separate stream, in the
>> same namespace. Fix it once in the kernel vs. fix it in umpteen
>> apps.
> This is ridiculous. We have shared libraries, 99% of applications
> manage to use libc for example --- this isn't that different.
Yes if that was the case then programs wouldn't brake if libc and
its API was fixed. Sadly, I think it isn't.
~S
> --cw
> On Iau, 2004-09-02 at 21:07, Spam wrote:
>> > And would you rather that logic was running swappable in shared library
>> > space or privileged and unswappable in kernel ?
>>
>> I would rather have it as a filesystem/vfs plugin that would allow
>> all my programs to use the features the plugin gives, even if it
>> would mean being totally in kernel space.
> Thats not what I asked.
I failed to understand your question then. :(
~S
Hi!
> > > >> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> > > >> option, enabled by default, to look inside archives. If you tell it to
> > > >> look for a driver in a given directory it will also look inside .cab
> > > >> and .zip files. It's extremely useful, I would imagine someone who uses
> > > >> XP a lot will come to expect this feature.
> > >
> > > > It is trivial to implement this by looking inside the files. I.e., the way
> > > > mc has done this for ages.
> > >
> > > Difference is that you can't do "locate" or "find" or "Search".. You
> > > would have to open the files in an archive-supporting application
> > > such as mc.
> >
> > You really need archive support in find. At the very least you need
> > option "enter archives" vs. "do not enter archives". Entering archives
> > automagically is seriously wrong.
>
> But is it efficient to make every application that reads files have to
> know how to get inside a tar file, just to read its contents? That
Application does not have to know how to handle tar/zip/etc, but it
has to make distinction between "enter archives" and "do not enter
archives". See uservfs.sf.net.
Pavel
On Thu, 2004-09-02 at 16:43, Pavel Machek wrote:
> Hi!
>
> > > > >> FWIW, this is how Windows does it now. As of XP, 'Find files' has an
> > > > >> option, enabled by default, to look inside archives. If you tell it to
> > > > >> look for a driver in a given directory it will also look inside .cab
> > > > >> and .zip files. It's extremely useful, I would imagine someone who uses
> > > > >> XP a lot will come to expect this feature.
> > > >
> > > > > It is trivial to implement this by looking inside the files. I.e., the way
> > > > > mc has done this for ages.
> > > >
> > > > Difference is that you can't do "locate" or "find" or "Search".. You
> > > > would have to open the files in an archive-supporting application
> > > > such as mc.
> > >
> > > You really need archive support in find. At the very least you need
> > > option "enter archives" vs. "do not enter archives". Entering archives
> > > automagically is seriously wrong.
> >
> > But is it efficient to make every application that reads files have to
> > know how to get inside a tar file, just to read its contents? That
>
> Application does not have to know how to handle tar/zip/etc, but it
> has to make distinction between "enter archives" and "do not enter
> archives". See uservfs.sf.net.
But how do you cache the information you had to look in the archive for
in a way that other apps can use it? How do you synchronize access to
the cache and maintain cache coherency in userspace?
Lee
On Thu, Sep 02, 2004 at 04:47:40PM -0400, Lee Revell wrote:
> But how do you cache the information you had to look in the archive
> for in a way that other apps can use it?
~/.object-cache/ or whatever
> How do you synchronize access to the cache and maintain cache
> coherency in userspace?
coherency doesn't exist for normal files for the most part anyhow
Hi!
> > > It is trivial to implement this by looking inside the files. I.e., the way
> > > mc has done this for ages.
> >
> > Difference is that you can't do "locate" or "find" or "Search".. You
> > would have to open the files in an archive-supporting application
> > such as mc.
>
> And would you rather that logic was running swappable in shared library
> space or privileged and unswappable in kernel ?
Well, having it in kernel mean that cache gets actually shared between
different processes -- and that's usefull. With coda and similar tools
you can put most of the logic into userspace (kernel impact is 20
lines). You *will* want it to run priviledged, because otherwise
caches are useless.
Pavel
On Thu, 2004-09-02 at 16:49, Chris Wedgwood wrote:
> On Thu, Sep 02, 2004 at 04:47:40PM -0400, Lee Revell wrote:
>
> > But how do you cache the information you had to look in the archive
> > for in a way that other apps can use it?
>
> ~/.object-cache/ or whatever
>
How are permissions handled? If root lists the contents of a tar file
that is world readable, then joeuser comes along and does the same, can
joeuser sees the cached listing? Would it depend on root's umask?
Lee
On Thu, Sep 02, 2004 at 04:57:06PM -0400, Lee Revell wrote:
> How are permissions handled? If root lists the contents of a tar
> file that is world readable, then joeuser comes along and does the
> same, can joeuser sees the cached listing?
everyone has their own cache i guess, works well enough elsewhere
also, am i the only person scared by the code complexity when it comes
to root and setuid code here? sounds like a disaster waiting to
happen
Hi!
> > Application does not have to know how to handle tar/zip/etc, but it
> > has to make distinction between "enter archives" and "do not enter
> > archives". See uservfs.sf.net.
>
> But how do you cache the information you had to look in the archive for
> in a way that other apps can use it? How do you synchronize access to
> the cache and maintain cache coherency in userspace?
Uservfs.sf.net.
Unlike alan, I do not think that "do it all in library" is good
idea. I put it in the userspace as "codafs" server, and let
applications see it as a regular filesystem.
Pavel
On Iau, 2004-09-02 at 21:58, Pavel Machek wrote:
> Uservfs.sf.net.
>
> Unlike alan, I do not think that "do it all in library" is good
> idea. I put it in the userspace as "codafs" server, and let
> applications see it as a regular filesystem.
That works for me too, providing someone has fixed all the user mode fs
deadlocks with paging
On Thu, 2004-09-02 at 23:33 +0100, Alan Cox wrote:
> On Iau, 2004-09-02 at 21:58, Pavel Machek wrote:
> > Uservfs.sf.net.
> >
> > Unlike alan, I do not think that "do it all in library" is good
> > idea. I put it in the userspace as "codafs" server, and let
> > applications see it as a regular filesystem.
>
> That works for me too, providing someone has fixed all the user mode fs
> deadlocks with paging
Aren't the deadlock scenarios only applicable for read/write mounted
filesystems ?
--
// Gianni Tedesco (gianni at scaramanga dot co dot uk)
lynx --source http://www.scaramanga.co.uk/scaramanga.asc | gpg --import
8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D
On Iau, 2004-09-02 at 21:01, Lee Revell wrote:
> But is it efficient to make every application that reads files have to
> know how to get inside a tar file, just to read its contents? That
> seems like a massive duplication of effort. Better to have the contents
> accessible via a separate stream, in the same namespace. Fix it once in
> the kernel vs. fix it in umpteen apps.
Thats how you get yourself a non useful OS. Fix it in a library and
share it between the apps that care. Like say.. gnome-vfs2
On Iau, 2004-09-02 at 21:07, Spam wrote:
> > And would you rather that logic was running swappable in shared library
> > space or privileged and unswappable in kernel ?
>
> I would rather have it as a filesystem/vfs plugin that would allow
> all my programs to use the features the plugin gives, even if it
> would mean being totally in kernel space.
Thats not what I asked.
Il Fri, 03 Sep 2004 04:22:48 +0100, Gianni Tedesco <[email protected]> ha scritto:
> On Thu, 2004-09-02 at 23:33 +0100, Alan Cox wrote:
> > On Iau, 2004-09-02 at 21:58, Pavel Machek wrote:
> > > Uservfs.sf.net.
> > >
> > > Unlike alan, I do not think that "do it all in library" is good
> > > idea. I put it in the userspace as "codafs" server, and let
> > > applications see it as a regular filesystem.
> >
> > That works for me too, providing someone has fixed all the user mode fs
> > deadlocks with paging
>
> Aren't the deadlock scenarios only applicable for read/write mounted
> filesystems ?
>
AFAIK deadlock arises when kernel manages buffers:
it has to free a buffer ==> choose a dirty one ==> if cleaning requires to make a call to
network server and this last is waiting for a buffer (cleaning accomplished) ==>
==> deadlock.
I think buffers are involved even in reading file system access.
I think also that NFS is threatened by deadlock, not CodaFS.
The problem with CodaFS is that it doesn't support random file access:
you have to download a whole file in coda cache directory even
to read its first byte.
I'm writing my laurea thesis about userspace file system,
and two Interesting papers i found about this, are:
http://a5.repetae.net/~frederik/avfs/frederik_surf_paper.pdf by Frederik Eaton
and the more generic UFO paper
http://www.usenix.org/publications/library/proceedings/ana97/alexandrov.html
I think that LUFS (Linux User File System) and FUSE (Filesystem in USErspace)
kernel modules are solutions to be considered for adding to Linux develop branch.
They are similar, but FUSE is still maintained and improved, and it permit to create
a custom daemon for each user filesystem (unlinke lufsd daemon which loads a
shared library). Communication kernel-user space is done through a special file descriptor,
unlike LUFS socket.
The only doubt is that FUSE is tuned for local file system performance, but I don't know
if it is possible to develop network file system with this framework.
LUFS is slower but more generic in this sense.
http://sourceforge.net/projects/avf
There is more documentation in the tarball than in the home page.
Ciao
Luca
Hi!
> > Uservfs.sf.net.
> >
> > Unlike alan, I do not think that "do it all in library" is good
> > idea. I put it in the userspace as "codafs" server, and let
> > applications see it as a regular filesystem.
>
> That works for me too, providing someone has fixed all the user mode fs
> deadlocks with paging
Its okay: coda works on whole file granularity, and files are backed
by real disk, so there are no deadlocks with paging. We only block
open() and similar syscalls, and those are not in paging paths. It
should be safe.
Pavel
On Fri, 2004-09-03 at 11:24 +0200, Luca Ferroni wrote:
> Il Fri, 03 Sep 2004 04:22:48 +0100, Gianni Tedesco <[email protected]> ha scritto:
>
> > On Thu, 2004-09-02 at 23:33 +0100, Alan Cox wrote:
> > > On Iau, 2004-09-02 at 21:58, Pavel Machek wrote:
> > > > Uservfs.sf.net.
> > > >
> > > > Unlike alan, I do not think that "do it all in library" is good
> > > > idea. I put it in the userspace as "codafs" server, and let
> > > > applications see it as a regular filesystem.
> > >
> > > That works for me too, providing someone has fixed all the user mode fs
> > > deadlocks with paging
> >
> > Aren't the deadlock scenarios only applicable for read/write mounted
> > filesystems ?
> >
>
> AFAIK deadlock arises when kernel manages buffers:
> it has to free a buffer ==> choose a dirty one ==> if cleaning
> requires to make a call to
> network server and this last is waiting for a buffer (cleaning
> accomplished) ==>
> ==> deadlock.
so during page launder, we need to write to the filesystem, but if thats
in userspace there is the possibility the page launder could have been
caused in order that the filesystem daemon may run. AFAICS this problem
only arises when file is being written.
The only deadlock I can think of for read-only filesystems is if the
demon inadvertantly accesses one of the files that it is handling. That
could be avoided quite simply by preventing the demon from doing that in
the kernel.
I'm sure I'm missing something, I'd just like to know what ;)
--
// Gianni Tedesco (gianni at scaramanga dot co dot uk)
lynx --source http://www.scaramanga.co.uk/scaramanga.asc | gpg --import
8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D
Salut,
On Thu, Sep 02, 2004 at 08:16:35PM +0100, Alan Cox wrote:
> Thats how you get yourself a non useful OS. Fix it in a library and
> share it between the apps that care. Like say.. gnome-vfs2
Even KIOslave has it. They even support sftp and stuff just by using
shared files in /tmp in reality. That's a much saner interface than
doing it all in the kernel.
I mean, the kernel is supposed to support access to the disk
drives. Who says that it's got to be the uppermost VFS level? You can
be perfectly happy to build your own VFS on top of it (or use other's
implementations, that is.)
I can already see people moving to FreeBSD if this gets implemented
into the kernel...
Tonnerre
> Salut,
> On Thu, Sep 02, 2004 at 08:16:35PM +0100, Alan Cox wrote:
>> Thats how you get yourself a non useful OS. Fix it in a library and
>> share it between the apps that care. Like say.. gnome-vfs2
> Even KIOslave has it. They even support sftp and stuff just by using
> shared files in /tmp in reality. That's a much saner interface than
> doing it all in the kernel.
> I mean, the kernel is supposed to support access to the disk
> drives. Who says that it's got to be the uppermost VFS level? You can
> be perfectly happy to build your own VFS on top of it (or use other's
> implementations, that is.)
Yes, we have several VFS versions. KDE and Gnome has their own, mc
another, and so on. None is compatible with the other and all data
is unavailable if you do not use a program specifically coded to use
the particular VFS you stored the data with.
> I can already see people moving to FreeBSD if this gets implemented
> into the kernel...
Why? No one has to use it. you do not have to load FS/VFS plugins so
you can extend the functionality of your filesystem with stuff like
compression, encryption, sorting, filtering, etc etc..
> Tonnerre
Hi!
> > Thats how you get yourself a non useful OS. Fix it in a library and
> > share it between the apps that care. Like say.. gnome-vfs2
>
> Even KIOslave has it. They even support sftp and stuff just by using
> shared files in /tmp in reality. That's a much saner interface than
> doing it all in the kernel.
>
> I mean, the kernel is supposed to support access to the disk
> drives. Who says that it's got to be the uppermost VFS level? You can
> be perfectly happy to build your own VFS on top of it (or use other's
> implementations, that is.)
You can not reasonably do caching when you are in shared library. And
you can not do caching across users at all.
Pavel