Salut,
On Wed, Sep 01, 2004 at 01:02:24AM +0200, Christer Weinigel wrote:
> I can see the argument for having the equivalent of Content-type or
> Content-transfer-encoding as a named stream though.
Why having them as named streams if we can get them as xattrs for no
additional pain? (Since fileutils would have to be changed anyway, we
can even make cp copy and emacs update xattrs.)
Tonnerre
On Sun, 5 Sep 2004 13:17:43 +0200, Tonnerre <[email protected]> wrote:
> Salut,
>
> On Wed, Sep 01, 2004 at 01:02:24AM +0200, Christer Weinigel wrote:
> > I can see the argument for having the equivalent of Content-type or
> > Content-transfer-encoding as a named stream though.
>
> Why having them as named streams if we can get them as xattrs for no
> additional pain? (Since fileutils would have to be changed anyway, we
> can even make cp copy and emacs update xattrs.)
Because some of as requested to be able to open tar,iso, and few other
formats this way. So I can simply access it with cat (as a dir/file).
--
GJ
> Salut,
> On Wed, Sep 01, 2004 at 01:02:24AM +0200, Christer Weinigel wrote:
>> I can see the argument for having the equivalent of Content-type or
>> Content-transfer-encoding as a named stream though.
> Why having them as named streams if we can get them as xattrs for no
> additional pain? (Since fileutils would have to be changed anyway, we
> can even make cp copy and emacs update xattrs.)
What if I do not use emacs, but vim, mcedit, gedit, or some other
editor? It doesn't seem logical to have to patch every application
that uses files.
~S
> Tonnerre
Salut,
On Sun, Sep 05, 2004 at 01:57:49PM +0200, Spam wrote:
> What if I do not use emacs, but vim, mcedit, gedit, or some other
> editor? It doesn't seem logical to have to patch every application
> that uses files.
We would have to do that in either case, so let's patch them to do it
in a nonintrusive way. And as to reading and writing inside tar files,
write and/or use a really nice userspace library to do it. (As does
MacOS/X, as does KDE, etc.)
Tonnerre
> Salut,
> On Sun, Sep 05, 2004 at 01:57:49PM +0200, Spam wrote:
>> What if I do not use emacs, but vim, mcedit, gedit, or some other
>> editor? It doesn't seem logical to have to patch every application
>> that uses files.
> We would have to do that in either case, so let's patch them to do it
> in a nonintrusive way. And as to reading and writing inside tar files,
> write and/or use a really nice userspace library to do it. (As does
> MacOS/X, as does KDE, etc.)
No, not with the current setup with file-as-dir. It also works in
Gnome/Natilus. Just tested:
mkdir test
chmod +x test
cd test
dir metas
echo 0777>rwx
In Nautilus I just typed in in the address bar /test/metas/
I could open meta-data and change with GEdit too.
I do not see that I have had to patch any programs to get this to
work.
~S
> Tonnerre
> Salut,
> On Sun, Sep 05, 2004 at 01:57:49PM +0200, Spam wrote:
>> What if I do not use emacs, but vim, mcedit, gedit, or some other
>> editor? It doesn't seem logical to have to patch every application
>> that uses files.
> We would have to do that in either case, so let's patch them to do it
> in a nonintrusive way. And as to reading and writing inside tar files,
> write and/or use a really nice userspace library to do it. (As does
> MacOS/X, as does KDE, etc.)
The problem with the userspace library is standardization. What
would be needed is a userspace library that has a extensible plugin
interface that is standardized. Otherwise we would need lots of
different libraries, and I seriously doubt that 1) this will happen
and 2) we get all Linux programs to be patched to use it.
~S
> Tonnerre
Hi!
> >> What if I do not use emacs, but vim, mcedit, gedit, or some other
> >> editor? It doesn't seem logical to have to patch every application
> >> that uses files.
>
> > We would have to do that in either case, so let's patch them to do it
> > in a nonintrusive way. And as to reading and writing inside tar files,
> > write and/or use a really nice userspace library to do it. (As does
> > MacOS/X, as does KDE, etc.)
>
> The problem with the userspace library is standardization. What
> would be needed is a userspace library that has a extensible plugin
> interface that is standardized. Otherwise we would need lots of
> different libraries, and I seriously doubt that 1) this will happen
> and 2) we get all Linux programs to be patched to use it.
libvfs from midnight commander (and anything build on the top of it)
already is very extensible.
Pavel
> Hi!
>> >> What if I do not use emacs, but vim, mcedit, gedit, or some other
>> >> editor? It doesn't seem logical to have to patch every application
>> >> that uses files.
>>
>> > We would have to do that in either case, so let's patch them to do it
>> > in a nonintrusive way. And as to reading and writing inside tar files,
>> > write and/or use a really nice userspace library to do it. (As does
>> > MacOS/X, as does KDE, etc.)
>>
>> The problem with the userspace library is standardization. What
>> would be needed is a userspace library that has a extensible plugin
>> interface that is standardized. Otherwise we would need lots of
>> different libraries, and I seriously doubt that 1) this will happen
>> and 2) we get all Linux programs to be patched to use it.
> libvfs from midnight commander (and anything build on the top of it)
> already is very extensible.
This wasn't my point. My point was about all applications using it.
Most aren't.
> Pavel
Spam <[email protected]> writes:
> >> The problem with the userspace library is standardization. What
> >> would be needed is a userspace library that has a extensible plugin
> >> interface that is standardized. Otherwise we would need lots of
> >> different libraries, and I seriously doubt that 1) this will happen
> >> and 2) we get all Linux programs to be patched to use it.
>
> > libvfs from midnight commander (and anything build on the top of it)
> > already is very extensible.
>
> This wasn't my point. My point was about all applications using it.
> Most aren't.
So why do you think that just because you put an interface into the
everyone will automatically start using it and use it without any
conflicts (i.e. is foo/icon a windows icon, a gif, png, or jpeg file,
what size is it, 16x16, 48x48...).
When you add a new shared namespace, userspace must be taught about it
anyways. So start with a userspace library, convince people to use
it, and when you know what people want, _then_ put it into the kernel
to increase performance or security. Don't start with "wouldn't it be
nice if" without any thought behind it, it will just mean that we get
another useless and misdesigned interface in the kernel that we have
to live with for years.
Do you know how long it took to get rid of /dev/cua0 and the locking
mess associated with it? (Deity! I just checked on my FC2 system,
it's still there. I thought it would be dead by now.)
/Christer
--
"Just how much can I get away with and still go to heaven?"
Freelance consultant specializing in device driver programming for Linux
Christer Weinigel <[email protected]> http://www.weinigel.se
> Spam <[email protected]> writes:
>> >> The problem with the userspace library is standardization. What
>> >> would be needed is a userspace library that has a extensible plugin
>> >> interface that is standardized. Otherwise we would need lots of
>> >> different libraries, and I seriously doubt that 1) this will happen
>> >> and 2) we get all Linux programs to be patched to use it.
>>
>> > libvfs from midnight commander (and anything build on the top of it)
>> > already is very extensible.
>>
>> This wasn't my point. My point was about all applications using it.
>> Most aren't.
> So why do you think that just because you put an interface into the
> everyone will automatically start using it and use it without any
> conflicts (i.e. is foo/icon a windows icon, a gif, png, or jpeg file,
> what size is it, 16x16, 48x48...).
> When you add a new shared namespace, userspace must be taught about it
> anyways. So start with a userspace library, convince people to use
> it, and when you know what people want, _then_ put it into the kernel
> to increase performance or security.
Several aspects of this;
1) Programs will need to actually be coded against this shared
library which probably will be more advanced that just usning normal
file access code.
2) Then we redo it all again to fit in the kernel, as a kernel
plugin or user-level plugin.
> Don't start with "wouldn't it be
> nice if" without any thought behind it, it will just mean that we get
> another useless and misdesigned interface in the kernel that we have
> to live with for years.
I am not saying that there should not be any thought behind
implementing something. But from many hear I just hear a big no,
without any thought about it either.
Named streams and meta files are just one example of a general
access to extra data that belongs to files. It seem to me as a much
simpler way for program to use those than to link in several various
libraries to do the same thing.
Plugins were just another thing that could extend the functionality
of these streams and meta data. Reiser4 has a plugin architecture,
although not yet any run-time support for loading them. Is this so
bad that we have to prevent it?
No one has to use them. maybe at some point it would be possible to
specify which classes of plugins that could be loaded by a user and
his files.
What are the complaints about. The technical challenge to make
things like this secure, or the concept to have this functionality
at all. I do not think that because something is hard to do, it
should be dismissed.
~S
> Do you know how long it took to get rid of /dev/cua0 and the locking
> mess associated with it? (Deity! I just checked on my FC2 system,
> it's still there. I thought it would be dead by now.)
> /Christer
Spam <[email protected]> writes:
> > When you add a new shared namespace, userspace must be taught about it
> > anyways. So start with a userspace library, convince people to use
> > it, and when you know what people want, _then_ put it into the kernel
> > to increase performance or security.
>
> Several aspects of this;
>
> 1) Programs will need to actually be coded against this shared
> library which probably will be more advanced that just usning normal
> file access code.
Not at all. As many others have already shown, trying to use files as
directories won't work since then you can't have named streams or
attributes for real directories. What does foo/icon refer to when foo
is a directory? There must be some way of differentiating between
them, which in turn means that we need a new way of referring to the
stream. So the simple, hackish, solution really won't fly.
Besides, it will probably break a lot of security critical assumptions
in todays software. What if you have a filter in your web server that
excludes executable files or files that end with .php, can you imagine
the mess if you could just open "foo.php/main-stream" instead?
> 2) Then we redo it all again to fit in the kernel, as a kernel
> plugin or user-level plugin.
No, we use the same user space library, but let the user space library
use helper functions in the kernel.
We could for example add a function that opens a named stream based on
a filename which uses a .xattr directory in the same directory as the
file, or in database in our home directory. If we add a solaris-like
openat(3) syscall we modify the library to take advantage of it.
> Plugins were just another thing that could extend the functionality
> of these streams and meta data. Reiser4 has a plugin architecture,
> although not yet any run-time support for loading them. Is this so
> bad that we have to prevent it?
Take an example: "expose a tar-file as streams below the file" which
as been suggested here is IMNSHO plain silly. I'm not saying anything
about mounting a tar file via userfs somewhere else in the file
system, that is quite ok, but trying to mount it on top of the same
file which suddenly and automagically turns into a sort-of-directory
breaks too many thing. Let your file manager do the choice instead,
based on the users preferences. For example, with a tar.gz-file, I'd
like to have a choice of "open file as a seekable file" which would
do:
mount -t userfs -o driver=gunzip foo.tar.gz /tmp/xyzzy
Then I can work with /tmp/xyzzy as a normal file (maybe even with
write access if the userfs driver can handle this). Another choice in
the same file manager would be "open file as a directory" which would
mount the same file in _another place_ as a directory, and I can even
have different views of the same file mounted at the same time. With
the named streams junk that have been suggested here, having two
different views would be impossible.
Sure, we could say that we add another level of indirection to the
named streams, so that we specify the driver as the first component of
te magic file-as-directory, i.e. foo.tar.gz/ungzipped would refer to
the ungzipped stream and foo.tar.gz/ungzipped-and-untarred would show
the tar file as a directory, but really, this isn't any more useful
than doing a userfs mount. The userfs mount does not break existing
semantics (anymore than mount -o loop does today), and it will work
with the existing infrastructure in the linux kernel. The only
advantage of files-as-directories with magic plugins in the kernel is
that one can look at it and say "look, how neat, the filenames look
almost the same".
/Christer
--
"Just how much can I get away with and still go to heaven?"
Freelance consultant specializing in device driver programming for Linux
Christer Weinigel <[email protected]> http://www.weinigel.se
Hi!
> Sure, we could say that we add another level of indirection to the
> named streams, so that we specify the driver as the first component of
> te magic file-as-directory, i.e. foo.tar.gz/ungzipped would refer to
> the ungzipped stream and foo.tar.gz/ungzipped-and-untarred would show
> the tar file as a directory, but really, this isn't any more useful
> than doing a userfs mount. The userfs mount does not break existing
> semantics (anymore than mount -o loop does today), and it will work
> with the existing infrastructure in the linux kernel. The only
> advantage of files-as-directories with magic plugins in the kernel is
> that one can look at it and say "look, how neat, the filenames look
> almost the same".
Who is going to umount it when application crashes, etc? Plus mount
required root priviledges last time I checked.
Pavel
Pavel Machek <[email protected]> writes:
> Who is going to umount it when application crashes, etc?
Who removes temporary files left behind when an application crashes?
I'd guess a daemon such as autofs could do a very good job here.
>Plus mount required root priviledges last time I checked.
bash$ ls -l /bin/mount
-rwsr-xr-x 1 root root 78504 May 4 23:34 /bin/mount
with proper policies in userspace it allows users to perform mounts.
I'm not suggesting that the kernel should be unchanged, but really,
some of the proposals here, to put a hell of a lot of complexity into
the kernel it just wet dreams with not much thought of how it affects
the kernel. What happened to the philosophy of putting complexity and
policy in userspace? Look at khttpd and tux, they were hacks in the
kernel to try things out. But what ended up in the kernel is generic
infrastructure that is useful for a lot more applications than just a
web server. That is the right way to do things.
/Christer
--
"Just how much can I get away with and still go to heaven?"
Freelance consultant specializing in device driver programming for Linux
Christer Weinigel <[email protected]> http://www.weinigel.se
>> Plugins were just another thing that could extend the functionality
>> of these streams and meta data. Reiser4 has a plugin architecture,
>> although not yet any run-time support for loading them. Is this so
>> bad that we have to prevent it?
> Take an example: "expose a tar-file as streams below the file" which
> as been suggested here is IMNSHO plain silly. I'm not saying anything
> about mounting a tar file via userfs somewhere else in the file
> system, that is quite ok, but trying to mount it on top of the same
> file which suddenly and automagically turns into a sort-of-directory
> breaks too many thing. Let your file manager do the choice instead,
> based on the users preferences. For example, with a tar.gz-file, I'd
> like to have a choice of "open file as a seekable file" which would
> do:
> mount -t userfs -o driver=gunzip foo.tar.gz /tmp/xyzzy
Where is the difference? Both would be handled by a specific driver
or module and export the same semantics (files+dirs) with permissions
etc to the user. With your idea you still would need the userfs
kernel module, and with "magic plugins", as you said, you will need a
vfs/reiser4 kernel plugin.
> Then I can work with /tmp/xyzzy as a normal file (maybe even with
> write access if the userfs driver can handle this). Another choice in
> the same file manager would be "open file as a directory" which would
> mount the same file in _another place_ as a directory, and I can even
> have different views of the same file mounted at the same time. With
> the named streams junk that have been suggested here, having two
> different views would be impossible.
> Sure, we could say that we add another level of indirection to the
> named streams, so that we specify the driver as the first component of
> te magic file-as-directory, i.e. foo.tar.gz/ungzipped would refer to
> the ungzipped stream and foo.tar.gz/ungzipped-and-untarred would show
> the tar file as a directory, but really, this isn't any more useful
> than doing a userfs mount. The userfs mount does not break existing
> semantics (anymore than mount -o loop does today), and it will work
> with the existing infrastructure in the linux kernel. The only
> advantage of files-as-directories with magic plugins in the kernel is
> that one can look at it and say "look, how neat, the filenames look
> almost the same".
No there are _usability_ differences. I cirtanly do not want to
mount lots of files with mount -t userfs, just to see extra
meta-data that I want to quickly be able to use. And it also
wouldn't work generically (searchable) with tools like find, grep,
etc either.
~S
> /Christer
Hi!
> >Plus mount required root priviledges last time I checked.
>
> bash$ ls -l /bin/mount
> -rwsr-xr-x 1 root root 78504 May 4 23:34 /bin/mount
>
> with proper policies in userspace it allows users to perform mounts.
You want to do exec() each time you enter a archive? I do not think
that is good idea.
> I'm not suggesting that the kernel should be unchanged, but really,
> some of the proposals here, to put a hell of a lot of complexity into
> the kernel it just wet dreams with not much thought of how it affects
> the kernel. What happened to the philosophy of putting complexity and
> policy in userspace? Look at khttpd and tux, they were hacks in the
> kernel to try things out. But what ended up in the kernel is generic
> infrastructure that is useful for a lot more applications than just a
> web server. That is the right way to do things.
Well, generic infrastructure for uservfs would be enough for me... And
it is few lines in kernel.
Pavel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Christer Weinigel wrote:
| Spam <[email protected]> writes:
|
|
|>>When you add a new shared namespace, userspace must be taught about it
|>>anyways. So start with a userspace library, convince people to use
|>>it, and when you know what people want, _then_ put it into the kernel
|>>to increase performance or security.
|>
|> Several aspects of this;
|>
|> 1) Programs will need to actually be coded against this shared
|> library which probably will be more advanced that just usning normal
|> file access code.
|
|
| Not at all. As many others have already shown, trying to use files as
| directories won't work since then you can't have named streams or
| attributes for real directories. What does foo/icon refer to when foo
Oh yes, you can. That's what "metas" is for. I wanted to call it ...
| is a directory? There must be some way of differentiating between
When foo is a directory, foo/icon is a normal file. But foo/metas/icon
(or foo/.../icon if you like) is the metadata.
This already works. I can use it right now for file permissions:
echo 755 > /bin/bash/metas/rwx
echo 755 > ./metas/rwx
| Besides, it will probably break a lot of security critical assumptions
| in todays software. What if you have a filter in your web server that
| excludes executable files or files that end with .php, can you imagine
| the mess if you could just open "foo.php/main-stream" instead?
Why would you have a foo.php/main-stream?
And what would be so hard about allowing read access to foo, but not
access to foo's metas? This is essentially the same as having mode 6 on
a directory.
[...]
|> Plugins were just another thing that could extend the functionality
|> of these streams and meta data. Reiser4 has a plugin architecture,
|> although not yet any run-time support for loading them. Is this so
|> bad that we have to prevent it?
|
|
| Take an example: "expose a tar-file as streams below the file" which
| as been suggested here is IMNSHO plain silly. I'm not saying anything
| about mounting a tar file via userfs somewhere else in the file
| system, that is quite ok, but trying to mount it on top of the same
| file which suddenly and automagically turns into a sort-of-directory
| breaks too many thing. Let your file manager do the choice instead,
What things? It's just dangerously different thinking. I don't see it
breaking anything on my system right now. If anything, the nvidia
binary drivers are a bigger issue...
| based on the users preferences. For example, with a tar.gz-file, I'd
| like to have a choice of "open file as a seekable file" which would
| do:
|
| mount -t userfs -o driver=gunzip foo.tar.gz /tmp/xyzzy
|
| Then I can work with /tmp/xyzzy as a normal file (maybe even with
| write access if the userfs driver can handle this). Another choice in
| the same file manager would be "open file as a directory" which would
| mount the same file in _another place_ as a directory, and I can even
| have different views of the same file mounted at the same time. With
| the named streams junk that have been suggested here, having two
| different views would be impossible.
|
| Sure, we could say that we add another level of indirection to the
| named streams, so that we specify the driver as the first component of
| te magic file-as-directory, i.e. foo.tar.gz/ungzipped would refer to
| the ungzipped stream and foo.tar.gz/ungzipped-and-untarred would show
| the tar file as a directory, but really, this isn't any more useful
| than doing a userfs mount. The userfs mount does not break existing
Yes, it is more useful than doing a userfs mount. The userfs mount has
to be done manually or by a file manager. I do not use a file manager.
~ I am not alone here. Unless you're going to have the entire FS mounted
userfs and have that be the file manager, which wouldn't be nearly as
fast as plugins.
I've heard autofs mentioned -- which only works on specific files which
it's been preconfigured for. Not good here.
| semantics (anymore than mount -o loop does today), and it will work
| with the existing infrastructure in the linux kernel. The only
That's true -- it's easier to implement if you ignore the number of apps
which need patching.
There's also the fact that most people here won't be patching apps,
they'll just be breathing a sigh of relief that they didn't have to
patch the kernel.
Just because someone else has to do it doesn't make it any less work.
| advantage of files-as-directories with magic plugins in the kernel is
| that one can look at it and say "look, how neat, the filenames look
| almost the same".
And this is a usability issue. Not all newbies use Nautilus.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iQIVAwUBQTy3MngHNmZLgCUhAQJ/lQ//UmQOJLZAmsFFD8TJehD8WitMrImrcrBC
SJ/mCkIaPq5JNo9R/ovd1UoGlf47JUvaaT5G1d/yOMA+/0kPNqjJL49Km9A3Ze+C
00Uq12yP8yp6NjBzq9HKAjvyHVghl1A5ZllIdiwuc8Fywbl6UKUHlN1D4PpUp1uA
znKPjmGpBOsTzgNWJYbAfjns60C7DHZoD/S+ldt3af2PsJKOpx0v4fusV8Y2+XRG
kWybZLl1YPwPLfJLCaa3KpbY1KkSuk6TSeut90+zRV6s1WJ8LE98NpDBGUDt/+cm
AXcp/zyrUmbpvQtZBaUFYwZ8imizspiSHYnBwdhVm9tfERFOFI9diBidvanHk/s0
D+FSJhnQSeo/hMnbHlfv4E6p4k1jDvJk06XJ/U45sEym4hCp9roDPB0tUic7bOnE
mxNxzogp92KHfsVPuqkidq255ztAjRbzdnt8zsBOPImqTgfsO3xVfT3svIixNxWv
+qaAlrR7dAri9bi3wNLCjzRhp3id0fspetLJrrCHI/8hMlu7a6nsIHLocuvG069W
OzdXtTPRYsV8YlElsdL5d/3grZSL3CWNpQxoRb9gzj0hKHHPjrXw9ACcQthiknNc
DkjjcLxV2gPqhKq3WgMTKXHhLDWEfJpFVQ8PccJHsKaaUgGtDUg0ETBT54h8Mo8E
I0bkcYOLtHk=
=o3vQ
-----END PGP SIGNATURE-----
Salut,
On Mon, Sep 06, 2004 at 05:54:56PM +0200, Pavel Machek wrote:
> Who is going to umount it when application crashes, etc?
This has been discussed along with the HAL people a while
ago. Actually, file systems can introduce a refcount, where we need a
decrement function which automatically unmounts the filesystem if we
decrement the use count to zero. Kind of an automatic umount.
How do you tell which file systems shall be autoumounted? HAL
suggestion: introduce a no-op mount option. For some classes of fs'es
we might as well make that obligatory, such as tar.
Tonnerre
On Mer, 2004-09-08 at 07:11, Tonnerre wrote:
> This has been discussed along with the HAL people a while
> ago. Actually, file systems can introduce a refcount, where we need a
> decrement function which automatically unmounts the filesystem if we
> decrement the use count to zero. Kind of an automatic umount.
We've supported file system garbage collection when they become unused
since umm about 2.4.10 I think.