Hi,
I'm working on a FUSE filesystem, which is part of a construction like
GoboLinux. It offers the user, when her/his session begins an
environment like:
/Computer
/Home
/Internet Services
/Mounts
/Network
/Shared
/System
The normal directories like /bin, /etc, /usr etcetera are present but
hidden ( by the FUSE fs).
See:
http://linux.bononline.nl/wiki/index.php/Mount.md5key
(especially the changes and the screenshots..)
To make this environment more consistent and usefull, it has to
support (i)notify.
Earlier I've posted a message about that on the linux fsdevel mailist:
http://marc.info/?l=linux-fsdevel&m=130052497829681&w=2
Mentioned there - and I agree - that when developing support for
notify on mounted filesystems you need to have one base (in VFS), and
a part which is handled by the (mounted) fs self.
So here my question, how does NFS handle notify?? I'm not that
familiar with NFS, so maybe I'm asking something really stupid here...
Thanks in advance,
Stef
2011/5/2 Trond Myklebust <[email protected]>:
>>
>> So, what's your opinion?
>
> Your proposal requires NFS and CIFS protocol support and I just told you
> that there is none in NFS...
Hi,
yes I've read that. I also have read that there is possibly support
for in newer versions in NFS.
I imagined that you (not only you but I mean all who are working on
NFS) are interested to find a good way to do this.
I want to work on this, but when I'm trying to achieve something in
the VFS, I'm interested in what other developers with simular projects
(most important cifs and nfs) look at it.
Stef
>
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> [email protected]
> http://www.netapp.com
>
>
On Mon, 2011-05-02 at 16:29 +0200, Stef Bon wrote:
> Ok,
>
> so as I understand NFS does not support it yet. I thought it does.
>
> Are you interested when I set up a project to work on the VFS to make
> it possible to
> make the mounted fs notified (by the VFS) that a notify watch is set
> (or removed)?
> I guess the developers of cifs are interested too, and me, as
> developer of a FUSE fs.
http://people.gnome.org/~veillard/gamin/
Cheers
Trond
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
Ok,
so as I understand NFS does not support it yet. I thought it does.
Are you interested when I set up a project to work on the VFS to make
it possible to
make the mounted fs notified (by the VFS) that a notify watch is set
(or removed)?
I guess the developers of cifs are interested too, and me, as
developer of a FUSE fs.
Stef
2011/5/2 J. Bruce Fields <[email protected]>:
> On Sun, May 01, 2011 at 11:33:32AM +0200, Stef Bon wrote:
>> Mentioned there - and I agree - that when developing support for
>> notify on mounted filesystems you need to have one base (in VFS), and
>> a part which is handled by the (mounted) fs self.
>>
>> So here my question, how does NFS handle notify??
>
> No. It might be possible with new features in the 4.1 version of the
> protocol (http://tools.ietf.org/html//rfc5661#section-10.9).
>
> --b.
>
Sorry,
Gamin is in the first place completly useless in my case. Futher I've
tried to contact the developer of gamin, but no response at all, from
different addresses.
And also with other fs's like cifs (and I think nfs as well), what I
try to achieve is that the fs self is responsible for the handling of
the inotify request.
How to do this? First possible via VFS, clients will set a watch on
(mounted) fs directly, VFS handles this call and forwards it to the
mounted fs. I write here forward, I mean sends a signal to the mounted
fs about a notify watch being set on a inode. After that (and being
successfull) the VFS opens a fd the mounted fs can send events to when
something happens on the inode (what is significant).
Next via a daemon like Gamin. Yes, possible, but is this smart? Isn't
it just possible through the VFS like above described.
Stef
2011/5/2 Trond Myklebust <[email protected]>:
> On Mon, 2011-05-02 at 16:29 +0200, Stef Bon wrote:
>> Ok,
>>
>> so as I understand NFS does not support it yet. I thought it does.
>>
>> Are you interested when I set up a project to work on the VFS to make
>> it possible to
>> make the mounted fs notified (by the VFS) that a notify watch is set
>> (or removed)?
>> I guess the developers of cifs are interested too, and me, as
>> developer of a FUSE fs.
>
> http://people.gnome.org/~veillard/gamin/
>
> Cheers
> Trond
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> [email protected]
> http://www.netapp.com
>
>
On Mon, 2011-05-02 at 22:16 +0200, Stef Bon wrote:
> 2011/5/2 Trond Myklebust <[email protected]>:
> >> Next via a daemon like Gamin. Yes, possible, but is this smart? Isn't
> >> it just possible through the VFS like above described.
> >
> > All filesystems should already support notification for files that are
> > created through the VFS.
>
> Exact, only I like to say here that the filesystems do not have to do
> anything for that.
> The VFS combines various things here. The filesystem self does not do
> anything here.
>
> >
> > The missing bit is to support notification for files that are created on
> > the _server_ and that never go through the client VFS. That is precisely
> > the problem that projects such as FAM and gamin are supposed to solve
> > using generic userland daemons that can re-export the notifications from
> > the NFS/CIFS/... server to any interested listeners.
>
> Yes that's also true. If you are taking nfs, cifs or a FUSE fs, where
> the server is for example the localhost self, thus more like backend.
> I 'm not very convinced Gamin is the way to go.
>
> Let me explain:
>
> for example a simple overlay FUSE fs, fusexmp-bind.
> It's behaving like a mount bind, making the contents of the directory
> to bind available under the mountpoint.
>
> Now user a mounts it like:
>
> mkdir ~/mount
> fusexmp-bind --bind-directory=/tmp ~/mount
>
> This makes the directory /tmp available under ~/mount for user a.
> As you say, when the user does:
>
> touch ~/mount/testfile1
>
> any client watching the ~/mount directory (through notify) will be
> notfied of this change, while
> fusexmp-bind does not support inotify at all. It the VFS which does all that.
>
> Now if any other process is doing something in /tmp,
> the same client watching ~/mount will not get notified, until a hard
> refresh command is given.
> The kernel (VFS) does not understand (and does not have to) that the
> directories /tmp and
> ~/mount are related.
>
> A sollution is that the VFS makes the filesystem (in this case thus
> fusexmp-bind) aware of a client
> watching directory ~/mount.
>
> This filesystem "knows" as the only one what to do with it, namenly
> set a inotify watch (with the same mask) on /tmp, read any events
> there and forward them back to the original inotify watch (the one on
> ~/mount).
>
> I'm suggesting here the use of VFS, making the filesystem aware of an
> inotify watch.
>
> But what about gamin?? Is is possible to make gamin take care of the
> changes, like with the simple example of above. Well, possible, but
> with extra configuration.
> And this is only a simple example. With much more complicated setups,
> where the files and directories a client "sees", which may depend on
> the specific context of that connection, Gamin cannot be aware af that
> all.
>
> So my suggestion is to make everything go through the filesystem. As I
> mentioned above, the VFS should make the filesystem aware of a notify
> watch being added, the filesystem should take the right action (like
> sending a SMB2 CHANGE_NOTIFY Request to the server and wait for
> anything to come back from the server with cifs, or setting another
> intofy watch on the backend with the simple FUSE fs above, and with
> NFS doing something simular like cifs) and giving any received event
> back to the VFS.
>
> There is another advantage here, which for my fs is interesting.
> "Knowing" that there is a watch set on a certain inode is interesting.
> My fs handles various resources, like local harddrives, usb sticks,
> cdroms but also SMB shares mounted with cifs, and especially with
> removable resources, like USB sticks and cdroms, it's very usefull
> that someone is watching it. This prevents it from unmounting it. But
> this counts for my fs.
> Ans also, it helps my fs to stay upto date. I say here helps, is not a
> complete garantee.
>
> So, what's your opinion?
Your proposal requires NFS and CIFS protocol support and I just told you
that there is none in NFS...
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
2011/5/2 Trond Myklebust <[email protected]>:
>> Next via a daemon like Gamin. Yes, possible, but is this smart? Isn't
>> it just possible through the VFS like above described.
>
> All filesystems should already support notification for files that are
> created through the VFS.
Exact, only I like to say here that the filesystems do not have to do
anything for that.
The VFS combines various things here. The filesystem self does not do
anything here.
>
> The missing bit is to support notification for files that are created on
> the _server_ and that never go through the client VFS. That is precisely
> the problem that projects such as FAM and gamin are supposed to solve
> using generic userland daemons that can re-export the notifications from
> the NFS/CIFS/... server to any interested listeners.
Yes that's also true. If you are taking nfs, cifs or a FUSE fs, where
the server is for example the localhost self, thus more like backend.
I 'm not very convinced Gamin is the way to go.
Let me explain:
for example a simple overlay FUSE fs, fusexmp-bind.
It's behaving like a mount bind, making the contents of the directory
to bind available under the mountpoint.
Now user a mounts it like:
mkdir ~/mount
fusexmp-bind --bind-directory=/tmp ~/mount
This makes the directory /tmp available under ~/mount for user a.
As you say, when the user does:
touch ~/mount/testfile1
any client watching the ~/mount directory (through notify) will be
notfied of this change, while
fusexmp-bind does not support inotify at all. It the VFS which does all that.
Now if any other process is doing something in /tmp,
the same client watching ~/mount will not get notified, until a hard
refresh command is given.
The kernel (VFS) does not understand (and does not have to) that the
directories /tmp and
~/mount are related.
A sollution is that the VFS makes the filesystem (in this case thus
fusexmp-bind) aware of a client
watching directory ~/mount.
This filesystem "knows" as the only one what to do with it, namenly
set a inotify watch (with the same mask) on /tmp, read any events
there and forward them back to the original inotify watch (the one on
~/mount).
I'm suggesting here the use of VFS, making the filesystem aware of an
inotify watch.
But what about gamin?? Is is possible to make gamin take care of the
changes, like with the simple example of above. Well, possible, but
with extra configuration.
And this is only a simple example. With much more complicated setups,
where the files and directories a client "sees", which may depend on
the specific context of that connection, Gamin cannot be aware af that
all.
So my suggestion is to make everything go through the filesystem. As I
mentioned above, the VFS should make the filesystem aware of a notify
watch being added, the filesystem should take the right action (like
sending a SMB2 CHANGE_NOTIFY Request to the server and wait for
anything to come back from the server with cifs, or setting another
intofy watch on the backend with the simple FUSE fs above, and with
NFS doing something simular like cifs) and giving any received event
back to the VFS.
There is another advantage here, which for my fs is interesting.
"Knowing" that there is a watch set on a certain inode is interesting.
My fs handles various resources, like local harddrives, usb sticks,
cdroms but also SMB shares mounted with cifs, and especially with
removable resources, like USB sticks and cdroms, it's very usefull
that someone is watching it. This prevents it from unmounting it. But
this counts for my fs.
Ans also, it helps my fs to stay upto date. I say here helps, is not a
complete garantee.
So, what's your opinion?
Stef
On Mon, 2011-05-02 at 16:56 +0200, Stef Bon wrote:
> Sorry,
>
> Gamin is in the first place completly useless in my case. Futher I've
> tried to contact the developer of gamin, but no response at all, from
> different addresses.
>
> And also with other fs's like cifs (and I think nfs as well), what I
> try to achieve is that the fs self is responsible for the handling of
> the inotify request.
>
> How to do this? First possible via VFS, clients will set a watch on
> (mounted) fs directly, VFS handles this call and forwards it to the
> mounted fs. I write here forward, I mean sends a signal to the mounted
> fs about a notify watch being set on a inode. After that (and being
> successfull) the VFS opens a fd the mounted fs can send events to when
> something happens on the inode (what is significant).
>
> Next via a daemon like Gamin. Yes, possible, but is this smart? Isn't
> it just possible through the VFS like above described.
All filesystems should already support notification for files that are
created through the VFS.
The missing bit is to support notification for files that are created on
the _server_ and that never go through the client VFS. That is precisely
the problem that projects such as FAM and gamin are supposed to solve
using generic userland daemons that can re-export the notifications from
the NFS/CIFS/... server to any interested listeners.
Note that there is no NFS protocol support for notifications in v2, v3
or v4. NFSv4.1 does have some limited protocol support for notifications
(dnotify only), but we haven't implemented them on Linux.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
On Sun, May 01, 2011 at 11:33:32AM +0200, Stef Bon wrote:
> Mentioned there - and I agree - that when developing support for
> notify on mounted filesystems you need to have one base (in VFS), and
> a part which is handled by the (mounted) fs self.
>
> So here my question, how does NFS handle notify??
No. It might be possible with new features in the 4.1 version of the
protocol (http://tools.ietf.org/html//rfc5661#section-10.9).
--b.
2011/5/3 Trond Myklebust <[email protected]>:
> You've proposed a 'solution', but I'm still not seeing what the problem
> is.
>
> 1. Why do we need to develop kernel support for dnotify+inotify+...
> in every networked filesystem + fuse?
Well yes that's a problem indeed. I see also there are more different
types, which do not make life easier.
> 2. What are the killer app use cases that we need to support and
> that cannot be supported with existing methods?
>
I do not have one, only the idea that Linux in general will benefit,
when notify is supported for users using cifs, nfs and fuse fs's like
mine.
> Notifications on large distributed filesystems are a scalability
> nightmare (particularly so for inotify), so they need careful
> justification and need to be limited in scope. Just pulling out a
> solution from your magic hat isn't sufficient.
I do not have much experience with large distributed filesystems (can
you name an example?) but isn't it possible to just turn notify off
for such systems?
Stef
On Mon, 2011-05-02 at 22:28 +0200, Stef Bon wrote:
> 2011/5/2 Trond Myklebust <[email protected]>:
>
> >>
> >> So, what's your opinion?
> >
> > Your proposal requires NFS and CIFS protocol support and I just told you
> > that there is none in NFS...
>
> Hi,
>
> yes I've read that. I also have read that there is possibly support
> for in newer versions in NFS.
As I said, there is support for dnotify-like functionality in NFSv4.1,
but no support for inotify, and certainly not for fanotify.
> I imagined that you (not only you but I mean all who are working on
> NFS) are interested to find a good way to do this.
You've proposed a 'solution', but I'm still not seeing what the problem
is.
1. Why do we need to develop kernel support for dnotify+inotify+...
in every networked filesystem + fuse?
2. What are the killer app use cases that we need to support and
that cannot be supported with existing methods?
Notifications on large distributed filesystems are a scalability
nightmare (particularly so for inotify), so they need careful
justification and need to be limited in scope. Just pulling out a
solution from your magic hat isn't sufficient.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com