Quick question, which I suspect has a long answer.
I would like to write a multiplexing filesystem. The idea is as follows:
The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
ext2. Most operations would just be passed to the native fs call. However, for
some files, selectable at run time by some control singal, would actually reside
on another file system. The other filesystem would have to be mounted.
The idea is for a version controlling filesystem. The server would be a network
server (hence the desire to wrap nfs) which presents a 'view' of the source
code. When the user reserves a file for editing, the file is copied to the
local disk. From that point on, the local file is referred to until the user
commits the change or unreserves the file. Ideally, the local copy of the file
could be on any file system, not one that is necessarily local. And this has to
be totally transparent to the user, except for the step where the user
'reserves' the file.
I've thought about two ways to do this. One is to wrap the 'versioning' file
system with a multiplexor that checks fs calls to see if they are referring to a
file that is on a different fs. The other approach is to intercept calls to the
VFS to do the same trick.
I'm new to the whole filesystem-coding thing, so bear with me if what i've just
said makes no sense. So, my question (I guess it wasn't quick after all) is:
Can it be done, and are either of my two approaches feasible? Any suggestions
or tips?
Thanks,
Mark Richards
PS please CC me if possible.
You may want to look at FS filter drivers in Windows 2000 as a good model
for something like this. Intermezzo also seems to do something a lot
like this, but not to the Windows 2000 degree. W2K has something
called a filter driver interface, which is super handy for
encryption, and going remote.
The filter driver idea is pretty cool, and MS has been able to
implement some pretty cool stuff with it, like snap in virus
detection software across different file systems. They also
did something called a "mini-port" file system, which was a
filter driver on top of their very complex IFS interface which
made writing remote file systems, like network clients, as lot
easier and faster.
The disadvantage to their approach was that several new
and nasty "viruses" showed up as filter drivers and used
this interface to trash file systems.
Food for thought.
:-)
Jeff
On Mon, Nov 26, 2001 at 10:59:48PM -0500, Mark Richards wrote:
> Quick question, which I suspect has a long answer.
>
> I would like to write a multiplexing filesystem. The idea is as follows:
>
> The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
> ext2. Most operations would just be passed to the native fs call. However, for
> some files, selectable at run time by some control singal, would actually reside
> on another file system. The other filesystem would have to be mounted.
>
> The idea is for a version controlling filesystem. The server would be a network
> server (hence the desire to wrap nfs) which presents a 'view' of the source
> code. When the user reserves a file for editing, the file is copied to the
> local disk. From that point on, the local file is referred to until the user
> commits the change or unreserves the file. Ideally, the local copy of the file
> could be on any file system, not one that is necessarily local. And this has to
> be totally transparent to the user, except for the step where the user
> 'reserves' the file.
>
> I've thought about two ways to do this. One is to wrap the 'versioning' file
> system with a multiplexor that checks fs calls to see if they are referring to a
> file that is on a different fs. The other approach is to intercept calls to the
> VFS to do the same trick.
>
> I'm new to the whole filesystem-coding thing, so bear with me if what i've just
> said makes no sense. So, my question (I guess it wasn't quick after all) is:
> Can it be done, and are either of my two approaches feasible? Any suggestions
> or tips?
>
> Thanks,
> Mark Richards
>
> PS please CC me if possible.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Oh yeah. The MS Distributed File System in W2K is actually
a filter driver on top of their existings FS's. So is
the mail slot interface.
Jeff
On Mon, Nov 26, 2001 at 09:34:38PM -0700, Jeff V. Merkey wrote:
>
> You may want to look at FS filter drivers in Windows 2000 as a good model
> for something like this. Intermezzo also seems to do something a lot
> like this, but not to the Windows 2000 degree. W2K has something
> called a filter driver interface, which is super handy for
> encryption, and going remote.
>
> The filter driver idea is pretty cool, and MS has been able to
> implement some pretty cool stuff with it, like snap in virus
> detection software across different file systems. They also
> did something called a "mini-port" file system, which was a
> filter driver on top of their very complex IFS interface which
> made writing remote file systems, like network clients, as lot
> easier and faster.
>
> The disadvantage to their approach was that several new
> and nasty "viruses" showed up as filter drivers and used
> this interface to trash file systems.
>
> Food for thought.
>
> :-)
>
> Jeff
>
>
>
> On Mon, Nov 26, 2001 at 10:59:48PM -0500, Mark Richards wrote:
> > Quick question, which I suspect has a long answer.
> >
> > I would like to write a multiplexing filesystem. The idea is as follows:
> >
> > The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
> > ext2. Most operations would just be passed to the native fs call. However, for
> > some files, selectable at run time by some control singal, would actually reside
> > on another file system. The other filesystem would have to be mounted.
> >
> > The idea is for a version controlling filesystem. The server would be a network
> > server (hence the desire to wrap nfs) which presents a 'view' of the source
> > code. When the user reserves a file for editing, the file is copied to the
> > local disk. From that point on, the local file is referred to until the user
> > commits the change or unreserves the file. Ideally, the local copy of the file
> > could be on any file system, not one that is necessarily local. And this has to
> > be totally transparent to the user, except for the step where the user
> > 'reserves' the file.
> >
> > I've thought about two ways to do this. One is to wrap the 'versioning' file
> > system with a multiplexor that checks fs calls to see if they are referring to a
> > file that is on a different fs. The other approach is to intercept calls to the
> > VFS to do the same trick.
> >
> > I'm new to the whole filesystem-coding thing, so bear with me if what i've just
> > said makes no sense. So, my question (I guess it wasn't quick after all) is:
> > Can it be done, and are either of my two approaches feasible? Any suggestions
> > or tips?
> >
> > Thanks,
> > Mark Richards
> >
> > PS please CC me if possible.
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
What about implimenting something like KDE's IOSlaves at kernel level? you
could then create a symlink to any recognized URL. This would also have
plenty of other uses.
On Tue, 27 Nov 2001 13:59, Mark Richards wrote:
> Quick question, which I suspect has a long answer.
>
> I would like to write a multiplexing filesystem. The idea is as follows:
>
> The filesystem would ideally wrap another filesystem, such as nfs or smbfs
> or ext2. Most operations would just be passed to the native fs call.
> However, for some files, selectable at run time by some control singal,
> would actually reside on another file system. The other filesystem would
> have to be mounted.
>
> The idea is for a version controlling filesystem. The server would be a
> network server (hence the desire to wrap nfs) which presents a 'view' of
> the source code. When the user reserves a file for editing, the file is
> copied to the local disk. From that point on, the local file is referred
> to until the user commits the change or unreserves the file. Ideally, the
> local copy of the file could be on any file system, not one that is
> necessarily local. And this has to be totally transparent to the user,
> except for the step where the user 'reserves' the file.
>
> I've thought about two ways to do this. One is to wrap the 'versioning'
> file system with a multiplexor that checks fs calls to see if they are
> referring to a file that is on a different fs. The other approach is to
> intercept calls to the VFS to do the same trick.
>
> I'm new to the whole filesystem-coding thing, so bear with me if what i've
> just said makes no sense. So, my question (I guess it wasn't quick after
> all) is: Can it be done, and are either of my two approaches feasible? Any
> suggestions or tips?
>
> Thanks,
> Mark Richards
>
> PS please CC me if possible.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
Mark Richards wrote:
>
> Quick question, which I suspect has a long answer.
>
> I would like to write a multiplexing filesystem. The idea is as follows:
>
> The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
> ext2. Most operations would just be passed to the native fs call. However, for
> some files, selectable at run time by some control singal, would actually reside
> on another file system. The other filesystem would have to be mounted.
>
> The idea is for a version controlling filesystem. The server would be a network
> server (hence the desire to wrap nfs) which presents a 'view' of the source
> code. When the user reserves a file for editing, the file is copied to the
> local disk. From that point on, the local file is referred to until the user
> commits the change or unreserves the file. Ideally, the local copy of the file
> could be on any file system, not one that is necessarily local. And this has to
> be totally transparent to the user, except for the step where the user
> 'reserves' the file.
Coda already do what you want:
Files are kept on a server, and copied to your local disk when
you use it. You may even disconnect when working on the local
copy - your changes will be propagated back to the server
whenever you reconnect to the network.
The copying is indeed completely transparent.
If you need reservation - use the permission system.
A suid program simply makes _you_ owner, and only
the owner gets write permission. This is your check-out
program. Check-in consists of changing ownership
back to root (or some userid allocated to the versioning system)
Helge Hafting
Helge Hafting wrote:
>
> Coda already do what you want:
> Files are kept on a server, and copied to your local disk when
> you use it. You may even disconnect when working on the local
> copy - your changes will be propagated back to the server
> whenever you reconnect to the network.
> The copying is indeed completely transparent.
>
> If you need reservation - use the permission system.
> A suid program simply makes _you_ owner, and only
> the owner gets write permission. This is your check-out
> program. Check-in consists of changing ownership
> back to root (or some userid allocated to the versioning system)
>
> Helge Hafting
I'll look into Coda, but ideally I wouldn't have to copy each file to the local
workstation when I use it, only when it is reserved for editing. Also, I'd like to
be able to store the local copy anywhere on the filesystem, if possible.
Mark
Mark Richards wrote:
>
> Quick question, which I suspect has a long answer.
>
> I would like to write a multiplexing filesystem. The idea is as follows:
>
> The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
> ext2. Most operations would just be passed to the native fs call. However, for
> some files, selectable at run time by some control singal, would actually reside
> on another file system. The other filesystem would have to be mounted.
>
> The idea is for a version controlling filesystem. The server would be a network
> server (hence the desire to wrap nfs) which presents a 'view' of the source
> code. When the user reserves a file for editing, the file is copied to the
> local disk. From that point on, the local file is referred to until the user
> commits the change or unreserves the file. Ideally, the local copy of the file
> could be on any file system, not one that is necessarily local. And this has to
> be totally transparent to the user, except for the step where the user
> 'reserves' the file.
>
> I've thought about two ways to do this. One is to wrap the 'versioning' file
> system with a multiplexor that checks fs calls to see if they are referring to a
> file that is on a different fs. The other approach is to intercept calls to the
> VFS to do the same trick.
>
> I'm new to the whole filesystem-coding thing, so bear with me if what i've just
> said makes no sense. So, my question (I guess it wasn't quick after all) is:
> Can it be done, and are either of my two approaches feasible? Any suggestions
> or tips?
I've heard of 'transparent' or 'stackable' filesystems that let you
mount a set of filesystems on top of each other, and changes are made to
the top level filesystem. (I think the BSDs might have (had) that...)
If you were to use something like that, the 'reserve' step would be
transparent to the user, but the 'commit' (to the lower layer fs) would
not be. (Mount the lower layer in some other place as well and write to
it for your commit.)
You might want to consider that approach since I think you would find a
number of other people would have other uses for the same filesystem.
(And thus you might get more help, and have a chance of getting it into
(one of) the standard kernel(s).)
Hmm... it just occurred to me... multisession CDs do something
similar... That might have some benefits for those who do multisession
cd mastering--and that gives you more potential contributors.
Just some thoughts... have fun!
Eli
--------------------. Real Users find the one combination of bizarre
Eli Carter \ input values that shuts down the system for days.
eli.carter(a)inet.com `-------------------------------------------------
On Tue, Nov 27, 2001 at 09:45:22AM -0500, Mark Richards wrote:
> Helge Hafting wrote:
>
> >
> > Coda already do what you want:
> > Files are kept on a server, and copied to your local disk when
> > you use it. You may even disconnect when working on the local
> > copy - your changes will be propagated back to the server
> > whenever you reconnect to the network.
> > The copying is indeed completely transparent.
> >
> > If you need reservation - use the permission system.
> > A suid program simply makes _you_ owner, and only
> > the owner gets write permission. This is your check-out
> > program. Check-in consists of changing ownership
> > back to root (or some userid allocated to the versioning system)
> >
> > Helge Hafting
>
> I'll look into Coda, but ideally I wouldn't have to copy each file to the local
> workstation when I use it, only when it is reserved for editing. Also, I'd like to
> be able to store the local copy anywhere on the filesystem, if possible.
>
Don't know much about coda, but...
Helge has basically said that the files *are* copied to your system
transparently (probably to a configurable location) and once you are
finished it will be synced with the coda server.
If you need *exclusive* access, just check out (change permissions to you as
owner, and chmod 0600), and back when you're done. This is probably done
with root suid tools to be able to chown.
MF
You may want to look into utilizing CVS for the check-in/check-out
mechanism and just virtualizing that layer on top of whatever FS is
being used....
This may be a good place to start looking.
http://sourceforge.net/projects/cvsfs/
If you don't need all the sophistication of CVS perhaps RCS would do.
The way Coda does it is by "hoarding" files, you specify what files you
want to hoard, and it refers to those as local only, when you "un-hoard"
them it syncs them with the server (I think this is the process, it's
been a while since I played with it.)
I think intermezzo has something similar, but neither does any revision
control.
On Mon, 2001-11-26 at 20:59, Mark Richards wrote:
> Quick question, which I suspect has a long answer.
>
> I would like to write a multiplexing filesystem. The idea is as follows:
>
> The filesystem would ideally wrap another filesystem, such as nfs or smbfs or
> ext2. Most operations would just be passed to the native fs call. However, for
> some files, selectable at run time by some control singal, would actually reside
> on another file system. The other filesystem would have to be mounted.
>
> The idea is for a version controlling filesystem. The server would be a network
> server (hence the desire to wrap nfs) which presents a 'view' of the source
> code. When the user reserves a file for editing, the file is copied to the
> local disk. From that point on, the local file is referred to until the user
> commits the change or unreserves the file. Ideally, the local copy of the file
> could be on any file system, not one that is necessarily local. And this has to
> be totally transparent to the user, except for the step where the user
> 'reserves' the file.
>
> I've thought about two ways to do this. One is to wrap the 'versioning' file
> system with a multiplexor that checks fs calls to see if they are referring to a
> file that is on a different fs. The other approach is to intercept calls to the
> VFS to do the same trick.
>
> I'm new to the whole filesystem-coding thing, so bear with me if what i've just
> said makes no sense. So, my question (I guess it wasn't quick after all) is:
> Can it be done, and are either of my two approaches feasible? Any suggestions
> or tips?
>
> Thanks,
> Mark Richards
>
> PS please CC me if possible.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Blake Barnett (bdb) <[email protected]>
Sr. Unix Administrator
DevelopOnline.com office: 480-377-6816
"Do, or do not. There is no try." --Yoda
Mark Richards wrote:
[...]
> I'll look into Coda, but ideally I wouldn't have to copy each file to the local
> workstation when I use it, only when it is reserved for editing. Also, I'd like to
> be able to store the local copy anywhere on the filesystem, if possible.
Worried your drive will fill up? The files copied to your
drive is merely copied as a "caching" operation. They
still seem to reside at the server - this is totally transparent.
And of course you can limit this caching - if too many files
is cached some is simply thrown away. (Or sent back
if they were changed.) They will be re-
loaded automatically if you ever need them again.
If you really want to store them where you want instead of
transparent access, why bother with a new FS at all?
(I believe coda lets you specify where the caching
will happen, if you have several partitions/drives)
Simply run a script that reserves the file (by using
the permission system) and *copy* it to
where you want. Check-in will consist of copying
the altered file back, and restore normal permissions.
You might even want to run a system like CVS, unless
there is some special reason for not doing that.
Helge Hafting
Helge Hafting wrote:
> Mark Richards wrote:
> [...]
> > I'll look into Coda, but ideally I wouldn't have to copy each file to the local
> > workstation when I use it, only when it is reserved for editing. Also, I'd like to
> > be able to store the local copy anywhere on the filesystem, if possible.
>
> Worried your drive will fill up? The files copied to your
> drive is merely copied as a "caching" operation. They
> still seem to reside at the server - this is totally transparent.
> And of course you can limit this caching - if too many files
> is cached some is simply thrown away. (Or sent back
> if they were changed.) They will be re-
> loaded automatically if you ever need them again.
>
> If you really want to store them where you want instead of
> transparent access, why bother with a new FS at all?
> (I believe coda lets you specify where the caching
> will happen, if you have several partitions/drives)
>
> Simply run a script that reserves the file (by using
> the permission system) and *copy* it to
> where you want. Check-in will consist of copying
> the altered file back, and restore normal permissions.
>
> You might even want to run a system like CVS, unless
> there is some special reason for not doing that.
Heh, it's funny that you mention that, because one of the goals of this project is to
access a CVS-like repository as if it were a filesystem. I suppose that i'm not opposed
to on-disk caching of files; this may improve performance in some cases. However, the
specific feature I need right now is that a file being modified by one person is
explicitly not seen by other people (they see the original version). The parts that
need to be transparent to the user are the copying/moving of the file from one place
(network) to another (somewhere else). Also, files created locally should not be
propagated back to the network unless the users explicitly specifies that they should.
So, this system is not totally transparent to the user (because we can't read the user's
mind to figure out what files (s)he would like to edit or 'check in'), but the
filesystem part of it should be. Plus, it's important that local files and network
files appear in the same directory side by side, as if they were in the same directory
on the server.
I still haven't had a chance to look at Coda; maybe it does solve my problem, but
probably not quite.
Mark
>
>
> Helge Hafting
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Hi!
> What about implimenting something like KDE's IOSlaves at kernel level? you
> could then create a symlink to any recognized URL. This would also have
> plenty of other uses.
uservfs.sf.net
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.