2012-11-29 09:32:35

by Stef Bon

[permalink] [raw]
Subject: Possible to make nfs aware of a inotify watch has been set.

Hi,

as far as I know individual filesystems are not aware an inotify watch
has been set on a
inode managed by that filesystem.

For local only filesystems like ext2/3/4 etcetera this is no problem.
But with network filesystems
like nfs and cifs (and also fuse types) this is not ideal, since
changes made by others (since the backend
is shared with others) are never watched.

I'm working on a filesystem change watcher notifyfs, and puzzling
already for a long time howto enable
the watching of fs events on these filesystems.

I've some idea's:

a. notifyfs is working on the local (A) and the remote host (B), and
is doing it's thing without using the filesystem. When a client
contacts notifyfs to set a watch on a nfs (or cifs) share, notifyfs
detects it a nfs share. It finds out the remote host by analyzing the
mount, and gets the details like remote address and directory. With
nfs that's easy, since the mount is like remotehost:/directory, with
cifs this is a bit more difficult, but doable.
After getting the details, it can forward the watch to the remote host
(B), on which also notifyfs is running, and listning to a tcp port.
This notifyfs process on his turn watches the desired fs object using
the abilities on that host, and sends anything back to the local host
(A).

I've experimented with this, and got it working when using sshfs.

This is very easy to setup, and simple to understand, but some drawbacks:

1. while the filesystems are using credentails or tickets to get
access to a remote resource, this is a bit difficult for notifyfs.
Notifyfs bypasses that. Maybe this leads to permissions/abuse I cannot
see directly.

2. the filesystem still does not know that something has changed (on
B). It should be made aware as fast as possible to make the changes
effective on localhost A. Maybe notifyfs should just trigger that by
doing a lookup on an entry when it has been removed on the remote host
for example.

b. notifyfs forwards the watch to nfs (and cifs) using netlink. Since
nfs and cifs are in kernelspace, and notifyfs is in userspace, netlink
is the way to go to communicate.This is possible, and I've thought
this is the best way to go for a long time, but complicated when
compared to the last method:

c. make filesystems like nfs and cifs aware a inotify watch has been
set. Untill now the filesystems self are totally not notified a watch
has been set (and of course also changed/removed). This is by design.
For local filesystems it's not required, it will not get any extra,
but nfs and cifs and fuse fs's can use this info to set a watch on the
backend using the methods/api specific to the filesystem. For example
cifs will send a SMB2_CHANGE_NOTIFY to the server, and waiting for
results.

What do you think, is the latest option possible??

Stef Bon


2012-11-29 17:06:23

by Myklebust, Trond

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

T24gVGh1LCAyMDEyLTExLTI5IGF0IDEwOjI4IC0wNjAwLCBTdGV2ZSBGcmVuY2ggd3JvdGU6DQo+
IE9uIFRodSwgTm92IDI5LCAyMDEyIGF0IDk6MzMgQU0sIE15a2xlYnVzdCwgVHJvbmQNCj4gPFRy
b25kLk15a2xlYnVzdEBuZXRhcHAuY29tPiB3cm90ZToNCj4gPiBPbiBUaHUsIDIwMTItMTEtMjkg
YXQgMTA6MjIgLTA1MDAsIHNpbW8gd3JvdGU6DQo+ID4+IE9uIFRodSwgMjAxMi0xMS0yOSBhdCAx
NTo0OSArMDEwMCwgU3RlZiBCb24gd3JvdGU6DQo+ID4+ID4gMjAxMi8xMS8yOSBNeWtsZWJ1c3Qs
IFRyb25kIDxUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbT46DQo+ID4+ID4gPj4gLS0tLS1Pcmln
aW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gPiA+Pg0KPiA+PiA+ID4+IDEuIHdoaWxlIHRoZSBmaWxl
c3lzdGVtcyBhcmUgdXNpbmcgY3JlZGVudGFpbHMgb3IgdGlja2V0cyB0byBnZXQgYWNjZXNzIHRv
IGENCj4gPj4gPiA+PiByZW1vdGUgcmVzb3VyY2UsIHRoaXMgaXMgYSBiaXQgZGlmZmljdWx0IGZv
ciBub3RpZnlmcy4NCj4gPj4gPiA+PiBOb3RpZnlmcyBieXBhc3NlcyB0aGF0LiBNYXliZSB0aGlz
IGxlYWRzIHRvIHBlcm1pc3Npb25zL2FidXNlIEkgY2Fubm90IHNlZQ0KPiA+PiA+ID4+IGRpcmVj
dGx5Lg0KPiA+PiA+ID4NCj4gPj4gPiA+IExhY2sgb2Ygc2VjdXJpdHkgaXMgYSBzaG93c3RvcHBl
ci4gVGhlcmUgYXJlIGdvb2QgcmVhc29ucyB3aHkgaW5vdGlmeSB3b24ndCBhbGxvdyB5b3UgdG8g
bW9uaXRvciBmaWxlcyBmb3Igd2hpY2ggeW91IGRvbid0IGhhdmUgYWNjZXNzIHBlcm1pc3Npb25z
Lg0KPiA+PiA+ID4NCj4gPj4gPg0KPiA+PiA+IExldCBtZSBleHBsYWluLCBJIHRoaW5rIHlvdSBu
b3QgdW5kZXJzdGFuZCBmdWxseS4NCj4gPj4gPg0KPiA+PiA+IE5vdGlmeWZzIGRvZXMgbm90IGFs
bG93IHVzZXJzL2NsaWVudHMgdG8gc2V0IGEgd2F0Y2ggaWYgdGhlcmUgYXJlIG5vDQo+ID4+ID4g
cmVhZCBwZXJtaXNzaW9ucyAodGhlIG9iamVjdCBhbmQgYWNjZXNzIGZvciB0aGUgd2hvbGUgcGF0
aCB0byBpdCksIHNvDQo+ID4+ID4gdGhlcmUgYXJlIG5vIHNlY3VyaXR5IGlzc3VlcyB0aGVyZS4N
Cj4gPj4gPg0KPiA+PiA+IFdoYXQgSSBtZWFuIGlzIHRoYXQgYW55IHByb2dyYW0gY2FuIGNvbnRh
Y3QgdGhlIHJlbW90ZSBub3RpZnlmcw0KPiA+PiA+IHNlcnZlciwgYW5kIHRoaXMgcmVtb3RlIG5v
dGlmeWZzIHNlcnZlciBjYW5ub3QgZmlndXJlIG91dCBpdCdzIGEgdmFsaWQNCj4gPj4gPiByZXF1
ZXN0IGZyb20gYW5vdGhlciBub3RpZnlmcyBzZXJ2ZXIsIG9yIGEgcHJvZ3JhbSBmYWtpbmcgdGhh
dC4NCj4gPj4gPiBJbiB0aGUgY29uc3RydWN0aW9uIEkgZGVzY3JpYmUgaXQgZG9lcyBub3QgY2hl
Y2sgdGhhdCAoeWV0KS4NCj4gPj4gPg0KPiA+PiA+ID4+DQo+ID4+ID4gPj4gV2hhdCBkbyB5b3Ug
dGhpbmssIGlzIHRoZSBsYXRlc3Qgb3B0aW9uIHBvc3NpYmxlPz8NCj4gPj4gPiA+DQo+ID4+ID4g
PiBTbyB3aGF0IGlzIHRoZSBraWxsZXIgYXBwIGZvciBpbm90aWZ5IG9uIE5GUy9DSUZTL0ZVU0U/
IFdoYXQgcHJvZ3JhbXMgZG8geW91IG5lZWQgdG8gcnVuIG9uIGEgTkZTL0NJRlMvRlVTRSBjbGll
bnQgdGhhdCB1c2UgaW5vdGlmeSBhbmQgdGhhdCB3b3VsZG4ndCBiZSBiZXR0ZXIgb2ZmIHJ1bm5p
bmcgb24gdGhlIHNlcnZlciBpbnN0ZWFkPw0KPiA+PiA+ID4NCj4gPj4gPg0KPiA+PiA+IFdoYXQg
ZG8geW91IG1lYW4gd2l0aCAiYmV0dGVyIG9mZiBydW5uaW5nIG9uIHRoZSBzZXJ2ZXIgaW5zdGVh
ZCI/DQo+ID4+ID4gVGhlcmUgYXJlIGEgbG90IG9mIHByb2dyYW1zIGludGVyZXN0ZWQgaW4gZnMg
Y2hhbmdlcywgbGlrZSBhIHNpbXBsZQ0KPiA+PiA+IGZpbGUgbWFuYWdlci4gSSB0aGluayBpdCdz
IGEgdmVyeSBuaWNlIGZlYXR1cmUgdG8gc2VlIGNoYW5nZXMgcmlnaHQNCj4gPj4gPiBhd2F5IGlu
IHRoZSB2aWV3Lg0KPiA+PiA+IEl0J3Mgbm90IGEga2lsbGVyIGFwcCwgYnV0IGEgdGhpbmsgdGhl
IHdob2xlIHVzZXIgZXhwZXJpZW5jZSBpcw0KPiA+PiA+IGltcHJvdmluZyB3aGVuIHlvdXIgc3lz
dGVtIGlzIGFibGUgdG8ga2VlcCBhIHZpZXcgKGxpa2UgYSB2aWV3IGluIHRoZQ0KPiA+PiA+IGZp
bGUgbWFuYWdlcikgdXAgdG8gZGF0ZS4NCj4gPj4gPg0KPiA+PiA+ID4gSU9XOiB3aG9zZSBwcm9i
bGVtIGFyZSB5b3UgdHJ5aW5nIHRvIHNvbHZlPw0KPiA+PiA+DQo+ID4+ID4gSSB0aGluayB0aGF0
IGVuYWJsaW5nIGZzIG5vdGlmeSBvbiBuZXR3b3JrIGZpbGVzeXN0ZW1zIGxpa2UgbmZzLCBjaWZz
DQo+ID4+ID4gYW5kIGZ1c2UgaXMgYSBnb29kIHRoaW5nIChzZWUgYWJvdmUpLiBPbiBzeXN0ZW1z
IGxpa2UgV2luZG93cyBhbmQgaU9TDQo+ID4+ID4gc2luY2UgbG9uZyB0aW1lIHRoaXMgd29ya3Mu
DQo+ID4+DQo+ID4+IENJRlMgaGFzIG5vdGlmaWNhdGlvbiBjYXBhYmlsaXRpZXMgYnVpbHQgaW4g
KG9wbG9ja3MpLCBhcyBkb2VzIE5GUw0KPiA+PiAobGVhc2VzKSwgaXMgdGhpcyBub3Qgc3VmZmlj
aWVudCA/DQo+ID4NCj4gPiBORlN2NC4xIGFjdHVhbGx5IGhhcyBkaXJlY3Rvcnkgbm90aWZpY2F0
aW9ucyB3aGljaCBkdXBsaWNhdGUgbW9zdCBvZiB0aGUNCj4gPiBmdW5jdGlvbmFsaXR5IG9mIGRu
b3RpZnkuIFRoZSBxdWVzdGlvbiBJJ20gYXNraW5nIGlzICJXaHkgc2hvdWxkIHdlIGRvDQo+ID4g
aXQ/Iiwgbm90ICJjYW4gd2UgZG8gaXQ/Ii4NCj4gPg0KPiA+IEFuc3dlcnMgbGlrZSAid2VsbCBX
aW5kb3dzIGFuZCBpT1MgZG8gaXQiIGFyZW4ndCBoZWxwZnVsIHVubGVzcyB0aGV5DQo+ID4gaW5j
bHVkZSBhIGRlc2NyaXB0aW9uIG9mIHdoYXQgV2luZG93cyBhbmQgaU9TIGFwcHMgdXNlIGl0IGZv
ciB0aGF0IHdlDQo+ID4gY2FuJ3QgYWxyZWFkeSBkbyBvbiBMaW51eC4NCj4gPiBJZiB0aGUgb25s
eSBhcHBsaWNhdGlvbiBpcyBiZWFnbGUsIHRoZW4gTGludXggaGFzIHRoaXMgcmVhbGx5IGhlbHBm
dWwNCj4gPiB1dGlsaXR5IGNhbGxlZCAic3NoIiwgd2hpY2ggYWxsb3dzIHlvdSB0byBhdm9pZCB3
YXN0aW5nIGEgbG9hZCBvZg0KPiA+IG5ldHdvcmsgYmFuZHdpZHRoLi4uDQo+IA0KPiBPZiBjb3Vy
c2Ugb25lIHJlYXNvbiB0aGF0IGZpbGUgbWFuYWdlcnMgbGlrZSBkaXJlY3Rvcnkgbm90aWZpY2F0
aW9ucyBpcyB0aGF0DQo+IGl0IGF2b2lkcyB3YXN0aW5nIGEgbG9hZCBvZiBuZXR3b3JrIGJhbmR3
aWR0aCAtIGFzIHRoZSBhbHRlcm5hdGl2ZSB0bw0KPiBkaXJlY3RvcnkgY2hhbmdlDQo+IG5vdGlm
aWNhdGlvbnMgaXMgcG9sbGluZyB0aGUgZGlyZWN0b3JpZXMgZm9yIGV2ZXJ5IG9wZW4gd2luZG93
LiAgIEFsc28gbm90ZSB0aGF0DQo+IG5ldHdvcmsgZmlsZSBzeXN0ZW1zIHdoZXJlIHBvc3NpYmxl
IHNob3VsZCBpbXBsZW1lbnQgc2ltaWxhciBzZW1hbnRpY3MgYW5kDQo+IGludGVyZmFjZXMgdG8g
dGhlIGxvY2FsIGZpbGUgc3lzdGVtcywgYW5kIGlub3RpZnkvZG5vdGlmeSBhcmUgbWFwcGFibGUN
Cj4gdG8gbmV0d29yaw0KPiAnZmlsZSBzeXN0ZW1zIChwcmVzdW1hYmx5IG5vdCBqdXN0IGNpZnMv
c21iMi9zbWIzKS4gICBTdXBwb3J0IGZvcg0KPiBrZWVwaW5nIGZpbGUgbWFuYWdlcg0KPiB2aWV3
cyByZWFzb25hYmx5IHVwIHRvIGRhdGUgd291bGQgYmUgYSBnb29kIGVub3VnaCByZWFzb24gZm9y
IHNvbWV0aGluZyBsaWtlDQo+IGlub3RpZnkvZG5vdGlmeSAoYW5kIGl0IGhhcyBiZWVuIHVzZWQg
YnkgZmlsZSBtYW5hZ2VycyBmb3Igb3ZlciAyMA0KPiB5ZWFycywgZXZlbiBiZWZvcmUNCj4gV2lu
ZG93cyBhbmQgaU9TKSAtIGFuZCB3ZSB3YW50IHRvIGdldCB0byB0aGUgcG9pbnQgd2hlcmUgaG9t
ZSBkaXJlY3Rvcmllcw0KPiB3b3JrIGFzIHdlbGwgb24gbmV0d29yayBtb3VudHMgYXMgb24gbG9j
YWwgc3RvcmFnZS4NCg0KZ2FtaW4gYW5kIHRoZSBGQU0gcHJvdG9jb2wgYWxyZWFkeSBmaWxscyB0
aGUgZmlsZSBtYW5hZ2VyL2Jyb3dzZXIgZ2FwIGFzDQpmYXIgYXMgSSBrbm93LiBOb2JvZHkgaGFz
IGNvbWUga25vY2tpbmcgb24gbXkgZG9vciBmcm9tIHRoZSBHTk9NRSBvciBLREUNCnByb2plY3Rz
IGFza2luZyBmb3IgbW9yZSBmdW5jdGlvbmFsaXR5IGZyb20gdGhlIE5GUyBjbGllbnQuDQoNCj4g
RGlyZWN0b3J5IG9wbG9ja3MgKGluIHNtYjMpIG1heSBiZSBoZWxwZnVsIGluIHJlZHVjaW5nIHRo
ZSBudW1iZXIgb2YgcmVtb3RlDQo+IGRpcmVjdG9yeSBjaGFuZ2Ugbm90aWZpY2F0aW9ucyBmdXJ0
aGVyLg0KDQpXaGVuIENJVEkgdHJpZWQgdG8gYWN0dWFsbHkgbWVhc3VyZSB0aGUgZWZmZWN0IG9m
IE5GU3Y0LjEgZGlyZWN0b3J5DQpkZWxlZ2F0aW9ucyBvbiBuZXR3b3JrIGJhbmR3aWR0aCwgdGhl
eSBmb3VuZCBub3RoaW5nLiBXZSdyZSBhbHJlYWR5DQpiZWluZyBlZmZpY2llbnQgZm9yIHRoZSBj
YXNlIHdoZXJlIHRoZSBkaXJlY3RvcnkgaXMgbm90IGJlaW5nIGNoYW5nZWQgYnkNCmFueW9uZSBv
dGhlciB0aGFuIHVzLg0KQlRXOiBpbm90aWZ5IGFscmVhZHkgd29ya3MgZm9yIHRoYXQgY2FzZSBv
biBib3RoIENJRlMgYW5kIE5GUy4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBj
bGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tDQp3
d3cubmV0YXBwLmNvbQ0K

2012-11-29 14:03:29

by Myklebust, Trond

[permalink] [raw]
Subject: RE: Possible to make nfs aware of a inotify watch has been set.

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBsaW51eC1uZnMtb3duZXJAdmdl
ci5rZXJuZWwub3JnIFttYWlsdG86bGludXgtbmZzLQ0KPiBvd25lckB2Z2VyLmtlcm5lbC5vcmdd
IE9uIEJlaGFsZiBPZiBTdGVmIEJvbg0KPiBTZW50OiBUaHVyc2RheSwgTm92ZW1iZXIgMjksIDIw
MTIgNDozMyBBTQ0KPiBUbzogbGludXgtbmZzQHZnZXIua2VybmVsLm9yZw0KPiBDYzogbGludXgt
Y2lmcw0KPiBTdWJqZWN0OiBQb3NzaWJsZSB0byBtYWtlIG5mcyBhd2FyZSBvZiBhIGlub3RpZnkg
d2F0Y2ggaGFzIGJlZW4gc2V0Lg0KPiANCj4gSGksDQo+IA0KPiBhcyBmYXIgYXMgSSBrbm93IGlu
ZGl2aWR1YWwgZmlsZXN5c3RlbXMgYXJlIG5vdCBhd2FyZSBhbiBpbm90aWZ5IHdhdGNoIGhhcw0K
PiBiZWVuIHNldCBvbiBhIGlub2RlIG1hbmFnZWQgYnkgdGhhdCBmaWxlc3lzdGVtLg0KPiANCj4g
Rm9yIGxvY2FsIG9ubHkgZmlsZXN5c3RlbXMgbGlrZSBleHQyLzMvNCBldGNldGVyYSB0aGlzIGlz
IG5vIHByb2JsZW0uDQo+IEJ1dCB3aXRoIG5ldHdvcmsgZmlsZXN5c3RlbXMNCj4gbGlrZSBuZnMg
YW5kIGNpZnMgKGFuZCBhbHNvIGZ1c2UgdHlwZXMpIHRoaXMgaXMgbm90IGlkZWFsLCBzaW5jZSBj
aGFuZ2VzIG1hZGUgYnkNCj4gb3RoZXJzIChzaW5jZSB0aGUgYmFja2VuZCBpcyBzaGFyZWQgd2l0
aCBvdGhlcnMpIGFyZSBuZXZlciB3YXRjaGVkLg0KPiANCj4gSSdtIHdvcmtpbmcgb24gYSBmaWxl
c3lzdGVtIGNoYW5nZSB3YXRjaGVyIG5vdGlmeWZzLCBhbmQgcHV6emxpbmcgYWxyZWFkeSBmb3IN
Cj4gYSBsb25nIHRpbWUgaG93dG8gZW5hYmxlIHRoZSB3YXRjaGluZyBvZiBmcyBldmVudHMgb24g
dGhlc2UgZmlsZXN5c3RlbXMuDQo+IA0KPiBJJ3ZlIHNvbWUgaWRlYSdzOg0KPiANCj4gYS4gbm90
aWZ5ZnMgaXMgd29ya2luZyBvbiB0aGUgbG9jYWwgKEEpIGFuZCB0aGUgcmVtb3RlIGhvc3QgKEIp
LCBhbmQgaXMgZG9pbmcgaXQncw0KPiB0aGluZyB3aXRob3V0IHVzaW5nIHRoZSBmaWxlc3lzdGVt
LiBXaGVuIGEgY2xpZW50IGNvbnRhY3RzIG5vdGlmeWZzIHRvIHNldCBhDQo+IHdhdGNoIG9uIGEg
bmZzIChvciBjaWZzKSBzaGFyZSwgbm90aWZ5ZnMgZGV0ZWN0cyBpdCBhIG5mcyBzaGFyZS4gSXQg
ZmluZHMgb3V0IHRoZQ0KPiByZW1vdGUgaG9zdCBieSBhbmFseXppbmcgdGhlIG1vdW50LCBhbmQg
Z2V0cyB0aGUgZGV0YWlscyBsaWtlIHJlbW90ZQ0KPiBhZGRyZXNzIGFuZCBkaXJlY3RvcnkuIFdp
dGggbmZzIHRoYXQncyBlYXN5LCBzaW5jZSB0aGUgbW91bnQgaXMgbGlrZQ0KPiByZW1vdGVob3N0
Oi9kaXJlY3RvcnksIHdpdGggY2lmcyB0aGlzIGlzIGEgYml0IG1vcmUgZGlmZmljdWx0LCBidXQg
ZG9hYmxlLg0KPiBBZnRlciBnZXR0aW5nIHRoZSBkZXRhaWxzLCBpdCBjYW4gZm9yd2FyZCB0aGUg
d2F0Y2ggdG8gdGhlIHJlbW90ZSBob3N0IChCKSwgb24NCj4gd2hpY2ggYWxzbyBub3RpZnlmcyBp
cyBydW5uaW5nLCBhbmQgbGlzdG5pbmcgdG8gYSB0Y3AgcG9ydC4NCj4gVGhpcyBub3RpZnlmcyBw
cm9jZXNzIG9uIGhpcyB0dXJuIHdhdGNoZXMgdGhlIGRlc2lyZWQgZnMgb2JqZWN0IHVzaW5nIHRo
ZQ0KPiBhYmlsaXRpZXMgb24gdGhhdCBob3N0LCBhbmQgc2VuZHMgYW55dGhpbmcgYmFjayB0byB0
aGUgbG9jYWwgaG9zdCAoQSkuDQo+IA0KPiBJJ3ZlIGV4cGVyaW1lbnRlZCB3aXRoIHRoaXMsIGFu
ZCBnb3QgaXQgd29ya2luZyB3aGVuIHVzaW5nIHNzaGZzLg0KPiANCj4gVGhpcyBpcyB2ZXJ5IGVh
c3kgdG8gc2V0dXAsIGFuZCBzaW1wbGUgdG8gdW5kZXJzdGFuZCwgYnV0IHNvbWUgZHJhd2JhY2tz
Og0KPiANCj4gMS4gd2hpbGUgdGhlIGZpbGVzeXN0ZW1zIGFyZSB1c2luZyBjcmVkZW50YWlscyBv
ciB0aWNrZXRzIHRvIGdldCBhY2Nlc3MgdG8gYQ0KPiByZW1vdGUgcmVzb3VyY2UsIHRoaXMgaXMg
YSBiaXQgZGlmZmljdWx0IGZvciBub3RpZnlmcy4NCj4gTm90aWZ5ZnMgYnlwYXNzZXMgdGhhdC4g
TWF5YmUgdGhpcyBsZWFkcyB0byBwZXJtaXNzaW9ucy9hYnVzZSBJIGNhbm5vdCBzZWUNCj4gZGly
ZWN0bHkuDQoNCkxhY2sgb2Ygc2VjdXJpdHkgaXMgYSBzaG93c3RvcHBlci4gVGhlcmUgYXJlIGdv
b2QgcmVhc29ucyB3aHkgaW5vdGlmeSB3b24ndCBhbGxvdyB5b3UgdG8gbW9uaXRvciBmaWxlcyBm
b3Igd2hpY2ggeW91IGRvbid0IGhhdmUgYWNjZXNzIHBlcm1pc3Npb25zLg0KDQo+IDIuIHRoZSBm
aWxlc3lzdGVtIHN0aWxsIGRvZXMgbm90IGtub3cgdGhhdCBzb21ldGhpbmcgaGFzIGNoYW5nZWQg
KG9uIEIpLiBJdA0KPiBzaG91bGQgYmUgbWFkZSBhd2FyZSBhcyBmYXN0IGFzIHBvc3NpYmxlIHRv
IG1ha2UgdGhlIGNoYW5nZXMgZWZmZWN0aXZlIG9uDQo+IGxvY2FsaG9zdCBBLiBNYXliZSBub3Rp
ZnlmcyBzaG91bGQganVzdCB0cmlnZ2VyIHRoYXQgYnkgZG9pbmcgYSBsb29rdXAgb24gYW4NCj4g
ZW50cnkgd2hlbiBpdCBoYXMgYmVlbiByZW1vdmVkIG9uIHRoZSByZW1vdGUgaG9zdCBmb3IgZXhh
bXBsZS4NCj4gDQo+IGIuIG5vdGlmeWZzIGZvcndhcmRzIHRoZSB3YXRjaCB0byBuZnMgKGFuZCBj
aWZzKSB1c2luZyBuZXRsaW5rLiBTaW5jZSBuZnMgYW5kDQo+IGNpZnMgYXJlIGluIGtlcm5lbHNw
YWNlLCBhbmQgbm90aWZ5ZnMgaXMgaW4gdXNlcnNwYWNlLCBuZXRsaW5rIGlzIHRoZSB3YXkgdG8g
Z28gdG8NCj4gY29tbXVuaWNhdGUuVGhpcyBpcyBwb3NzaWJsZSwgYW5kIEkndmUgdGhvdWdodCB0
aGlzIGlzIHRoZSBiZXN0IHdheSB0byBnbyBmb3IgYQ0KPiBsb25nIHRpbWUsIGJ1dCBjb21wbGlj
YXRlZCB3aGVuIGNvbXBhcmVkIHRvIHRoZSBsYXN0IG1ldGhvZDoNCj4gDQo+IGMuICBtYWtlIGZp
bGVzeXN0ZW1zIGxpa2UgbmZzIGFuZCBjaWZzIGF3YXJlIGEgaW5vdGlmeSB3YXRjaCBoYXMgYmVl
biBzZXQuIFVudGlsbA0KPiBub3cgdGhlIGZpbGVzeXN0ZW1zIHNlbGYgYXJlIHRvdGFsbHkgbm90
IG5vdGlmaWVkIGEgd2F0Y2ggaGFzIGJlZW4gc2V0IChhbmQgb2YNCj4gY291cnNlIGFsc28gY2hh
bmdlZC9yZW1vdmVkKS4gVGhpcyBpcyBieSBkZXNpZ24uDQo+IEZvciBsb2NhbCBmaWxlc3lzdGVt
cyBpdCdzIG5vdCByZXF1aXJlZCwgaXQgd2lsbCBub3QgZ2V0IGFueSBleHRyYSwgYnV0IG5mcyBh
bmQgY2lmcw0KPiBhbmQgZnVzZSBmcydzIGNhbiB1c2UgdGhpcyBpbmZvIHRvIHNldCBhIHdhdGNo
IG9uIHRoZSBiYWNrZW5kIHVzaW5nIHRoZQ0KPiBtZXRob2RzL2FwaSBzcGVjaWZpYyB0byB0aGUg
ZmlsZXN5c3RlbS4gRm9yIGV4YW1wbGUgY2lmcyB3aWxsIHNlbmQgYQ0KPiBTTUIyX0NIQU5HRV9O
T1RJRlkgdG8gdGhlIHNlcnZlciwgYW5kIHdhaXRpbmcgZm9yIHJlc3VsdHMuDQo+IA0KPiBXaGF0
IGRvIHlvdSB0aGluaywgaXMgdGhlIGxhdGVzdCBvcHRpb24gcG9zc2libGU/Pw0KDQpTbyB3aGF0
IGlzIHRoZSBraWxsZXIgYXBwIGZvciBpbm90aWZ5IG9uIE5GUy9DSUZTL0ZVU0U/IFdoYXQgcHJv
Z3JhbXMgZG8geW91IG5lZWQgdG8gcnVuIG9uIGEgTkZTL0NJRlMvRlVTRSBjbGllbnQgdGhhdCB1
c2UgaW5vdGlmeSBhbmQgdGhhdCB3b3VsZG4ndCBiZSBiZXR0ZXIgb2ZmIHJ1bm5pbmcgb24gdGhl
IHNlcnZlciBpbnN0ZWFkPw0KDQpJT1c6IHdob3NlIHByb2JsZW0gYXJlIHlvdSB0cnlpbmcgdG8g
c29sdmU/DQoNClRyb25kDQo=

2012-11-30 07:19:52

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/30 Stef Bon <[email protected]>:
> 2012/11/30 Steve French <[email protected]>:

>> match closely to the existing Linux API.
>
> Hi,
>
> I start working on the seperate notifyfs servers.
> The whole advantage of that approach is that you can design the api
> from scratch. Notifyfs is not bound to the api of inotify. There
> remains the problem of mapping it to the fs event api on the host,
> which will be inotify probably on linux.
>
> At this moment the different events look like:
>


Maybe I will add locking information as well. I see that the
/proc/locks file can be monitored as well.

To me, information is very valuable. It would be very nice (yes my
opinion) to have information about a file on a nfs (or cifs) share
about it lockstatus, and by who, and get notified when it's unlocked.

And no nobody asked me to do so.

Stef

2012-11-29 15:33:18

by Myklebust, Trond

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

T24gVGh1LCAyMDEyLTExLTI5IGF0IDEwOjIyIC0wNTAwLCBzaW1vIHdyb3RlOg0KPiBPbiBUaHUs
IDIwMTItMTEtMjkgYXQgMTU6NDkgKzAxMDAsIFN0ZWYgQm9uIHdyb3RlOg0KPiA+IDIwMTIvMTEv
MjkgTXlrbGVidXN0LCBUcm9uZCA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+Og0KPiA+ID4+
IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPj4NCj4gPiA+PiAxLiB3aGlsZSB0aGUg
ZmlsZXN5c3RlbXMgYXJlIHVzaW5nIGNyZWRlbnRhaWxzIG9yIHRpY2tldHMgdG8gZ2V0IGFjY2Vz
cyB0byBhDQo+ID4gPj4gcmVtb3RlIHJlc291cmNlLCB0aGlzIGlzIGEgYml0IGRpZmZpY3VsdCBm
b3Igbm90aWZ5ZnMuDQo+ID4gPj4gTm90aWZ5ZnMgYnlwYXNzZXMgdGhhdC4gTWF5YmUgdGhpcyBs
ZWFkcyB0byBwZXJtaXNzaW9ucy9hYnVzZSBJIGNhbm5vdCBzZWUNCj4gPiA+PiBkaXJlY3RseS4N
Cj4gPiA+DQo+ID4gPiBMYWNrIG9mIHNlY3VyaXR5IGlzIGEgc2hvd3N0b3BwZXIuIFRoZXJlIGFy
ZSBnb29kIHJlYXNvbnMgd2h5IGlub3RpZnkgd29uJ3QgYWxsb3cgeW91IHRvIG1vbml0b3IgZmls
ZXMgZm9yIHdoaWNoIHlvdSBkb24ndCBoYXZlIGFjY2VzcyBwZXJtaXNzaW9ucy4NCj4gPiA+DQo+
ID4gDQo+ID4gTGV0IG1lIGV4cGxhaW4sIEkgdGhpbmsgeW91IG5vdCB1bmRlcnN0YW5kIGZ1bGx5
Lg0KPiA+IA0KPiA+IE5vdGlmeWZzIGRvZXMgbm90IGFsbG93IHVzZXJzL2NsaWVudHMgdG8gc2V0
IGEgd2F0Y2ggaWYgdGhlcmUgYXJlIG5vDQo+ID4gcmVhZCBwZXJtaXNzaW9ucyAodGhlIG9iamVj
dCBhbmQgYWNjZXNzIGZvciB0aGUgd2hvbGUgcGF0aCB0byBpdCksIHNvDQo+ID4gdGhlcmUgYXJl
IG5vIHNlY3VyaXR5IGlzc3VlcyB0aGVyZS4NCj4gPiANCj4gPiBXaGF0IEkgbWVhbiBpcyB0aGF0
IGFueSBwcm9ncmFtIGNhbiBjb250YWN0IHRoZSByZW1vdGUgbm90aWZ5ZnMNCj4gPiBzZXJ2ZXIs
IGFuZCB0aGlzIHJlbW90ZSBub3RpZnlmcyBzZXJ2ZXIgY2Fubm90IGZpZ3VyZSBvdXQgaXQncyBh
IHZhbGlkDQo+ID4gcmVxdWVzdCBmcm9tIGFub3RoZXIgbm90aWZ5ZnMgc2VydmVyLCBvciBhIHBy
b2dyYW0gZmFraW5nIHRoYXQuDQo+ID4gSW4gdGhlIGNvbnN0cnVjdGlvbiBJIGRlc2NyaWJlIGl0
IGRvZXMgbm90IGNoZWNrIHRoYXQgKHlldCkuDQo+ID4gDQo+ID4gPj4NCj4gPiA+PiBXaGF0IGRv
IHlvdSB0aGluaywgaXMgdGhlIGxhdGVzdCBvcHRpb24gcG9zc2libGU/Pw0KPiA+ID4NCj4gPiA+
IFNvIHdoYXQgaXMgdGhlIGtpbGxlciBhcHAgZm9yIGlub3RpZnkgb24gTkZTL0NJRlMvRlVTRT8g
V2hhdCBwcm9ncmFtcyBkbyB5b3UgbmVlZCB0byBydW4gb24gYSBORlMvQ0lGUy9GVVNFIGNsaWVu
dCB0aGF0IHVzZSBpbm90aWZ5IGFuZCB0aGF0IHdvdWxkbid0IGJlIGJldHRlciBvZmYgcnVubmlu
ZyBvbiB0aGUgc2VydmVyIGluc3RlYWQ/DQo+ID4gPg0KPiA+IA0KPiA+IFdoYXQgZG8geW91IG1l
YW4gd2l0aCAiYmV0dGVyIG9mZiBydW5uaW5nIG9uIHRoZSBzZXJ2ZXIgaW5zdGVhZCI/DQo+ID4g
VGhlcmUgYXJlIGEgbG90IG9mIHByb2dyYW1zIGludGVyZXN0ZWQgaW4gZnMgY2hhbmdlcywgbGlr
ZSBhIHNpbXBsZQ0KPiA+IGZpbGUgbWFuYWdlci4gSSB0aGluayBpdCdzIGEgdmVyeSBuaWNlIGZl
YXR1cmUgdG8gc2VlIGNoYW5nZXMgcmlnaHQNCj4gPiBhd2F5IGluIHRoZSB2aWV3Lg0KPiA+IEl0
J3Mgbm90IGEga2lsbGVyIGFwcCwgYnV0IGEgdGhpbmsgdGhlIHdob2xlIHVzZXIgZXhwZXJpZW5j
ZSBpcw0KPiA+IGltcHJvdmluZyB3aGVuIHlvdXIgc3lzdGVtIGlzIGFibGUgdG8ga2VlcCBhIHZp
ZXcgKGxpa2UgYSB2aWV3IGluIHRoZQ0KPiA+IGZpbGUgbWFuYWdlcikgdXAgdG8gZGF0ZS4NCj4g
PiANCj4gPiA+IElPVzogd2hvc2UgcHJvYmxlbSBhcmUgeW91IHRyeWluZyB0byBzb2x2ZT8NCj4g
PiANCj4gPiBJIHRoaW5rIHRoYXQgZW5hYmxpbmcgZnMgbm90aWZ5IG9uIG5ldHdvcmsgZmlsZXN5
c3RlbXMgbGlrZSBuZnMsIGNpZnMNCj4gPiBhbmQgZnVzZSBpcyBhIGdvb2QgdGhpbmcgKHNlZSBh
Ym92ZSkuIE9uIHN5c3RlbXMgbGlrZSBXaW5kb3dzIGFuZCBpT1MNCj4gPiBzaW5jZSBsb25nIHRp
bWUgdGhpcyB3b3Jrcy4NCj4gDQo+IENJRlMgaGFzIG5vdGlmaWNhdGlvbiBjYXBhYmlsaXRpZXMg
YnVpbHQgaW4gKG9wbG9ja3MpLCBhcyBkb2VzIE5GUw0KPiAobGVhc2VzKSwgaXMgdGhpcyBub3Qg
c3VmZmljaWVudCA/DQoNCk5GU3Y0LjEgYWN0dWFsbHkgaGFzIGRpcmVjdG9yeSBub3RpZmljYXRp
b25zIHdoaWNoIGR1cGxpY2F0ZSBtb3N0IG9mIHRoZQ0KZnVuY3Rpb25hbGl0eSBvZiBkbm90aWZ5
LiBUaGUgcXVlc3Rpb24gSSdtIGFza2luZyBpcyAiV2h5IHNob3VsZCB3ZSBkbw0KaXQ/Iiwgbm90
ICJjYW4gd2UgZG8gaXQ/Ii4NCg0KQW5zd2VycyBsaWtlICJ3ZWxsIFdpbmRvd3MgYW5kIGlPUyBk
byBpdCIgYXJlbid0IGhlbHBmdWwgdW5sZXNzIHRoZXkNCmluY2x1ZGUgYSBkZXNjcmlwdGlvbiBv
ZiB3aGF0IFdpbmRvd3MgYW5kIGlPUyBhcHBzIHVzZSBpdCBmb3IgdGhhdCB3ZQ0KY2FuJ3QgYWxy
ZWFkeSBkbyBvbiBMaW51eC4NCklmIHRoZSBvbmx5IGFwcGxpY2F0aW9uIGlzIGJlYWdsZSwgdGhl
biBMaW51eCBoYXMgdGhpcyByZWFsbHkgaGVscGZ1bA0KdXRpbGl0eSBjYWxsZWQgInNzaCIsIHdo
aWNoIGFsbG93cyB5b3UgdG8gYXZvaWQgd2FzdGluZyBhIGxvYWQgb2YNCm5ldHdvcmsgYmFuZHdp
ZHRoLi4uDQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWlu
ZXINCg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20N
Cg==

2012-11-29 21:16:28

by Al Viro

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, Nov 29, 2012 at 09:31:05PM +0100, Stef Bon wrote:
> 2012/11/29 Al Viro <[email protected]>:
> > On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:
> >
> >> And yes there is no such thing as a killer app, but in my opinion it's
> >> the user experience.
> >
> > Ah, yes... UX - the killer argument. "Doing that would make qi flow better"
> > and any objections are waved away, since the person objecting has no idea
> > how qi behaves (since it doesn't exist, to start with). Could we please keep
> > the intellectual level above the Feng Shui scammers^W^W^WUX theorists usual?
>
> Wow. Like I've said, I do not have a killer app, or anything else
> which is as obvious like 1+1=2. No, but I give you my opinion. To me,
> boldly stated, a system which is able to do fs notify events over
> network filesystems is turning from a dump system into an intelligent
> one.
>
> With Ubuntu people started Unity, also without a killer app, but just
> with an opinion, an impression, an idea. And anyone can dislike or
> like it, it's a succes as far as I can see. What the hell is wrong
> with suggesting things based upon an idea/impression?

IOW, it was a true positive for "([Uu]ser experience)|UX" as BS predictor.
Nice to know...

Look, you'd been asked to give good reasons for doing some thing; replying with
"it's My Opinion(tm), piss off if that's not enough reason for you" would at
least have been honest. References to UX, as your last reply has confirmed,
had been basically an obfuscated equivalent of that.

Sorry for being harsh, but I've had it with the GNOME crowd and their ilk
using just such references to handwave away any questions.

There's nothing wrong with having opinions/impressions/etc. Everyone got
their own, etc. But "I believe so, period" is not a sufficient answer to
"what makes that a good idea?" and obfuscating it up is even worse.

2012-11-30 14:20:10

by Bernd Petrovitsch

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

Hi!

Since you introduced the "user experience" ignoring the fact that each
user is quite different - especially if it comes to the "Average Joes
using Ubuntu vs techies" department -, you should be prepared to eat
your own dog food:

On Fre, 2012-11-30 at 08:10 +0100, Stef Bon wrote:
[...]
> At this moment the different events look like:

Perhaps you should have used hexadecimal notation so that it is easier
for the usual/average users of source code to see the bits more easily.
In short: Please improve the user experience. Thank you.

> #define NOTIFYFS_FSEVENT_META_ATTRIB_NOTSET 2
> #define NOTIFYFS_FSEVENT_META_ATTRIB_MODE 4
> #define NOTIFYFS_FSEVENT_META_ATTRIB_OWNER 8
> #define NOTIFYFS_FSEVENT_META_ATTRIB_GROUP 16
> #define NOTIFYFS_FSEVENT_META_ATTRIB 28
And that number seems pretty random - not a power of 2 or a bit mask or
the sum of the above.

> #define NOTIFYFS_FSEVENT_META_XATTR_NOTSET 32
> #define NOTIFYFS_FSEVENT_META_XATTR_CREATE 64
> #define NOTIFYFS_FSEVENT_META_XATTR_MODIFY 128
> #define NOTIFYFS_FSEVENT_META_XATTR_DELETE 256
> #define NOTIFYFS_FSEVENT_META_XATTR 448
And that number seems pretty random too - not a power of 2 or a bit mask
or the sum of the above.

> #define NOTIFYFS_FSEVENT_FILE_NOTSET 512
> #define NOTIFYFS_FSEVENT_FILE_MODIFIED 1024
> #define NOTIFYFS_FSEVENT_FILE_SIZE 2048
> #define NOTIFYFS_FSEVENT_FILE_OPEN 4096
> #define NOTIFYFS_FSEVENT_FILE_READ 8192
> #define NOTIFYFS_FSEVENT_FILE_CLOSE_WRITE 16384
> #define NOTIFYFS_FSEVENT_FILE_CLOSE_NOWRITE 32768

> #define NOTIFYFS_FSEVENT_MOVE_NOTSET 65536

And from here own I'm too lazy to calculate (or even guess) if these are
actually powers of 2 or not.
> #define NOTIFYFS_FSEVENT_MOVE_CREATED 131072
> #define NOTIFYFS_FSEVENT_MOVE_MOVED 262144
> #define NOTIFYFS_FSEVENT_MOVE_MOVED_FROM 524288
> #define NOTIFYFS_FSEVENT_MOVE_MOVED_TO 1048576
> #define NOTIFYFS_FSEVENT_MOVE_DELETED 2097152
>
> #define NOTIFYFS_FSEVENT_FS_NOTSET 4194304
> #define NOTIFYFS_FSEVENT_FS_MOUNT 8388608
> #define NOTIFYFS_FSEVENT_FS_UNMOUNT 16777216
> #define NOTIFYFS_FSEVENT_FS_NLINKS 33554432

MfG,
Bernd
--
Bernd Petrovitsch Email : [email protected]
LUGA : http://www.luga.at


2012-11-29 23:14:37

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/29 Al Viro <[email protected]>:
> On Thu, Nov 29, 2012 at 09:31:05PM +0100, Stef Bon wrote:
>> 2012/11/29 Al Viro <[email protected]>:
>> > On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:
>> >

>> With Ubuntu people started Unity, also without a killer app, but just
>> with an opinion, an impression, an idea. And anyone can dislike or
>> like it, it's a succes as far as I can see. What the hell is wrong
>> with suggesting things based upon an idea/impression?
>
> IOW, it was a true positive for "([Uu]ser experience)|UX" as BS predictor.
> Nice to know...
>
> Look, you'd been asked to give good reasons for doing some thing; replying with
> "it's My Opinion(tm), piss off if that's not enough reason for you" would at
> least have been honest. References to UX, as your last reply has confirmed,
> had been basically an obfuscated equivalent of that.

No no, I do not say "piss off". If that is what my message looked
like, I'm sorry for that. But I do not have any killer app or
something. I cannot convince you otherwise that saying it's my
opinion. I do not have numbers or something like that proving I have a
point. I wish I have. But I do understand you do not agree with me,
and see things different.
>
> Sorry for being harsh, but I've had it with the GNOME crowd and their ilk
> using just such references to handwave away any questions.
>
> There's nothing wrong with having opinions/impressions/etc. Everyone got
> their own, etc. But "I believe so, period" is not a sufficient answer to
> "what makes that a good idea?" and obfuscating it up is even worse.

So, no I do not say "I believe so, period" . But I cannot convince you
...and we have different opinions here.. I don't know any method here
of "proving having a point".

Stef

2012-11-29 15:27:51

by simo

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, 2012-11-29 at 15:49 +0100, Stef Bon wrote:
> 2012/11/29 Myklebust, Trond <[email protected]>:
> >> -----Original Message-----
> >>
> >> 1. while the filesystems are using credentails or tickets to get access to a
> >> remote resource, this is a bit difficult for notifyfs.
> >> Notifyfs bypasses that. Maybe this leads to permissions/abuse I cannot see
> >> directly.
> >
> > Lack of security is a showstopper. There are good reasons why inotify won't allow you to monitor files for which you don't have access permissions.
> >
>
> Let me explain, I think you not understand fully.
>
> Notifyfs does not allow users/clients to set a watch if there are no
> read permissions (the object and access for the whole path to it), so
> there are no security issues there.
>
> What I mean is that any program can contact the remote notifyfs
> server, and this remote notifyfs server cannot figure out it's a valid
> request from another notifyfs server, or a program faking that.
> In the construction I describe it does not check that (yet).
>
> >>
> >> What do you think, is the latest option possible??
> >
> > So what is the killer app for inotify on NFS/CIFS/FUSE? What programs do you need to run on a NFS/CIFS/FUSE client that use inotify and that wouldn't be better off running on the server instead?
> >
>
> What do you mean with "better off running on the server instead"?
> There are a lot of programs interested in fs changes, like a simple
> file manager. I think it's a very nice feature to see changes right
> away in the view.
> It's not a killer app, but a think the whole user experience is
> improving when your system is able to keep a view (like a view in the
> file manager) up to date.
>
> > IOW: whose problem are you trying to solve?
>
> I think that enabling fs notify on network filesystems like nfs, cifs
> and fuse is a good thing (see above). On systems like Windows and iOS
> since long time this works.

CIFS has notification capabilities built in (oplocks), as does NFS
(leases), is this not sufficient ?

Simo.

--
Simo Sorce
Samba Team GPL Compliance Officer <[email protected]>
Principal Software Engineer at Red Hat, Inc. <[email protected]>


2012-11-29 20:20:43

by Al Viro

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:

> And yes there is no such thing as a killer app, but in my opinion it's
> the user experience.

Ah, yes... UX - the killer argument. "Doing that would make qi flow better"
and any objections are waved away, since the person objecting has no idea
how qi behaves (since it doesn't exist, to start with). Could we please keep
the intellectual level above the Feng Shui scammers^W^W^WUX theorists usual?

2012-11-29 14:49:02

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/29 Myklebust, Trond <[email protected]>:
>> -----Original Message-----
>>
>> 1. while the filesystems are using credentails or tickets to get access to a
>> remote resource, this is a bit difficult for notifyfs.
>> Notifyfs bypasses that. Maybe this leads to permissions/abuse I cannot see
>> directly.
>
> Lack of security is a showstopper. There are good reasons why inotify won't allow you to monitor files for which you don't have access permissions.
>

Let me explain, I think you not understand fully.

Notifyfs does not allow users/clients to set a watch if there are no
read permissions (the object and access for the whole path to it), so
there are no security issues there.

What I mean is that any program can contact the remote notifyfs
server, and this remote notifyfs server cannot figure out it's a valid
request from another notifyfs server, or a program faking that.
In the construction I describe it does not check that (yet).

>>
>> What do you think, is the latest option possible??
>
> So what is the killer app for inotify on NFS/CIFS/FUSE? What programs do you need to run on a NFS/CIFS/FUSE client that use inotify and that wouldn't be better off running on the server instead?
>

What do you mean with "better off running on the server instead"?
There are a lot of programs interested in fs changes, like a simple
file manager. I think it's a very nice feature to see changes right
away in the view.
It's not a killer app, but a think the whole user experience is
improving when your system is able to keep a view (like a view in the
file manager) up to date.

> IOW: whose problem are you trying to solve?

I think that enabling fs notify on network filesystems like nfs, cifs
and fuse is a good thing (see above). On systems like Windows and iOS
since long time this works.

You do not agree??

Stef Bon

2012-11-29 22:26:39

by simo

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, 2012-11-29 at 21:09 +0000, Myklebust, Trond wrote:
> On Thu, 2012-11-29 at 15:05 -0500, simo wrote:
> > On Thu, 2012-11-29 at 17:11 +0000, Myklebust, Trond wrote:
> > > The idea that "if we build a bridge, they will come to us" is what
> > > gave
> > > rise to dnotify, then inotify (nobody came, so let's build a bigger
> > > bridge) and fsnotify (make it a toll bridge). Nobody is using those
> > > interfaces much on local filesystems, so why is adding it to NFS and
> > > CIFS going to be such a game changer?
> >
> > Sorry Trond,
> > but this is simply not true.
> >
> > inotify is a key feature at least in some packages I know, and makes
> > some things *a lot* easier.
> >
> > Of course this are less visible features, that are easy to overlook, but
> > they are not unused for sure.
>
> So why is it so difficult for anyone in this thread to name these killer
> applications that would benefit from NFS+CIFS support? I keep asking,
> and all I get is wishy-washy and hand-wavy answers.
>
> Without use cases, there is nothing to discuss on the implementation
> front.

Sorry for the misunderstanding, I wasn't talking about inotify *on
CIFS/NFS*, I do not care about that use case, I'll gladly let Stef pitch
it for himself :)

Simo.

--
Simo Sorce
Samba Team GPL Compliance Officer <[email protected]>
Principal Software Engineer at Red Hat, Inc. <[email protected]>


2012-11-29 20:05:58

by simo

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, 2012-11-29 at 17:11 +0000, Myklebust, Trond wrote:
> The idea that "if we build a bridge, they will come to us" is what
> gave
> rise to dnotify, then inotify (nobody came, so let's build a bigger
> bridge) and fsnotify (make it a toll bridge). Nobody is using those
> interfaces much on local filesystems, so why is adding it to NFS and
> CIFS going to be such a game changer?

Sorry Trond,
but this is simply not true.

inotify is a key feature at least in some packages I know, and makes
some things *a lot* easier.

Of course this are less visible features, that are easy to overlook, but
they are not unused for sure.

(dnotify on the other hand was underused because it was almost useless)

Simo.

--
Simo Sorce
Samba Team GPL Compliance Officer <[email protected]>
Principal Software Engineer at Red Hat, Inc. <[email protected]>


2012-11-29 20:10:47

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/29 Myklebust, Trond <[email protected]>:
> On Thu, 2012-11-29 at 17:39 +0100, Stef Bon wrote:
>> 2012/11/29 Myklebust, Trond <[email protected]>:
>> > On Thu, 2012-11-29 at 10:22 -0500, simo wrote:
>> >> On Thu, 2012-11-29 at 15:49 +0100, Stef Bon wrote:
>> >> > 2012/11/29 Myklebust, Trond <[email protected]>:
>> >> > >> -----Original Message-----
>> >> > >>

>>
>> No, sure, you should not do anything because others do it. But on the
>> other hand, if others do it, why don't you? Better copy a good idea
>> than be stubborn for ever.
>
> The idea that "if we build a bridge, they will come to us" is what gave
> rise to dnotify, then inotify (nobody came, so let's build a bigger
> bridge) and fsnotify (make it a toll bridge). Nobody is using those
> interfaces much on local filesystems, so why is adding it to NFS and
> CIFS going to be such a game changer?

Well that's a good question. I've been working on this issue for some
time now, and do not find much interest in it. I do not understand!

Although I find this a bit frustating, I still beleive it's a good
feature. And yes, I believe that what you suggest with building a
bridge (having a good idea) they will come to us.

And I've [posted on qt dev platforms, and they mentioned me that they
need much more fine grained methods, that just "something has
changed". But ok, that has nothing to do with nfs ...


>
>> And yes there is no such thing as a killer app, but in my opinion it's
>> the user experience. It's so much better when your system keeps track
>> on shared network resources.
>
> What does it enable us to do that we can't already do? This is precisely
> the question that I asked you in the previous email.

Huh? Is it possible to monitor directories and other objects for
changes already with network fs's??

By polling? The poor mans fs notify service?

Stef

2012-11-29 20:31:06

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/29 Al Viro <[email protected]>:
> On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:
>
>> And yes there is no such thing as a killer app, but in my opinion it's
>> the user experience.
>
> Ah, yes... UX - the killer argument. "Doing that would make qi flow better"
> and any objections are waved away, since the person objecting has no idea
> how qi behaves (since it doesn't exist, to start with). Could we please keep
> the intellectual level above the Feng Shui scammers^W^W^WUX theorists usual?

Wow. Like I've said, I do not have a killer app, or anything else
which is as obvious like 1+1=2. No, but I give you my opinion. To me,
boldly stated, a system which is able to do fs notify events over
network filesystems is turning from a dump system into an intelligent
one.

With Ubuntu people started Unity, also without a killer app, but just
with an opinion, an impression, an idea. And anyone can dislike or
like it, it's a succes as far as I can see. What the hell is wrong
with suggesting things based upon an idea/impression?

Stef

2012-11-29 17:11:24

by Myklebust, Trond

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

T24gVGh1LCAyMDEyLTExLTI5IGF0IDE3OjM5ICswMTAwLCBTdGVmIEJvbiB3cm90ZToNCj4gMjAx
Mi8xMS8yOSBNeWtsZWJ1c3QsIFRyb25kIDxUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbT46DQo+
ID4gT24gVGh1LCAyMDEyLTExLTI5IGF0IDEwOjIyIC0wNTAwLCBzaW1vIHdyb3RlOg0KPiA+PiBP
biBUaHUsIDIwMTItMTEtMjkgYXQgMTU6NDkgKzAxMDAsIFN0ZWYgQm9uIHdyb3RlOg0KPiA+PiA+
IDIwMTIvMTEvMjkgTXlrbGVidXN0LCBUcm9uZCA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+
Og0KPiA+PiA+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ID4gPj4NCj4gPiBO
RlN2NC4xIGFjdHVhbGx5IGhhcyBkaXJlY3Rvcnkgbm90aWZpY2F0aW9ucyB3aGljaCBkdXBsaWNh
dGUgbW9zdCBvZiB0aGUNCj4gPiBmdW5jdGlvbmFsaXR5IG9mIGRub3RpZnkuIFRoZSBxdWVzdGlv
biBJJ20gYXNraW5nIGlzICJXaHkgc2hvdWxkIHdlIGRvDQo+ID4gaXQ/Iiwgbm90ICJjYW4gd2Ug
ZG8gaXQ/Ii4NCj4gPg0KPiA+IEFuc3dlcnMgbGlrZSAid2VsbCBXaW5kb3dzIGFuZCBpT1MgZG8g
aXQiIGFyZW4ndCBoZWxwZnVsIHVubGVzcyB0aGV5DQo+ID4gaW5jbHVkZSBhIGRlc2NyaXB0aW9u
IG9mIHdoYXQgV2luZG93cyBhbmQgaU9TIGFwcHMgdXNlIGl0IGZvciB0aGF0IHdlDQo+ID4gY2Fu
J3QgYWxyZWFkeSBkbyBvbiBMaW51eC4NCj4gPiBJZiB0aGUgb25seSBhcHBsaWNhdGlvbiBpcyBi
ZWFnbGUsIHRoZW4gTGludXggaGFzIHRoaXMgcmVhbGx5IGhlbHBmdWwNCj4gPiB1dGlsaXR5IGNh
bGxlZCAic3NoIiwgd2hpY2ggYWxsb3dzIHlvdSB0byBhdm9pZCB3YXN0aW5nIGEgbG9hZCBvZg0K
PiA+IG5ldHdvcmsgYmFuZHdpZHRoLi4uDQo+IA0KPiBObywgc3VyZSwgeW91IHNob3VsZCBub3Qg
ZG8gYW55dGhpbmcgYmVjYXVzZSBvdGhlcnMgZG8gaXQuIEJ1dCBvbiB0aGUNCj4gb3RoZXIgaGFu
ZCwgaWYgb3RoZXJzIGRvIGl0LCB3aHkgZG9uJ3QgeW91PyBCZXR0ZXIgY29weSBhIGdvb2QgaWRl
YQ0KPiB0aGFuIGJlIHN0dWJib3JuIGZvciBldmVyLg0KDQpUaGUgaWRlYSB0aGF0ICJpZiB3ZSBi
dWlsZCBhIGJyaWRnZSwgdGhleSB3aWxsIGNvbWUgdG8gdXMiIGlzIHdoYXQgZ2F2ZQ0KcmlzZSB0
byBkbm90aWZ5LCB0aGVuIGlub3RpZnkgKG5vYm9keSBjYW1lLCBzbyBsZXQncyBidWlsZCBhIGJp
Z2dlcg0KYnJpZGdlKSBhbmQgZnNub3RpZnkgKG1ha2UgaXQgYSB0b2xsIGJyaWRnZSkuIE5vYm9k
eSBpcyB1c2luZyB0aG9zZQ0KaW50ZXJmYWNlcyBtdWNoIG9uIGxvY2FsIGZpbGVzeXN0ZW1zLCBz
byB3aHkgaXMgYWRkaW5nIGl0IHRvIE5GUyBhbmQNCkNJRlMgZ29pbmcgdG8gYmUgc3VjaCBhIGdh
bWUgY2hhbmdlcj8NCg0KPiBBbmQgeWVzIHRoZXJlIGlzIG5vIHN1Y2ggdGhpbmcgYXMgYSBraWxs
ZXIgYXBwLCBidXQgaW4gbXkgb3BpbmlvbiBpdCdzDQo+IHRoZSB1c2VyIGV4cGVyaWVuY2UuIEl0
J3Mgc28gbXVjaCBiZXR0ZXIgd2hlbiB5b3VyIHN5c3RlbSBrZWVwcyB0cmFjaw0KPiBvbiBzaGFy
ZWQgbmV0d29yayByZXNvdXJjZXMuDQoNCldoYXQgZG9lcyBpdCBlbmFibGUgdXMgdG8gZG8gdGhh
dCB3ZSBjYW4ndCBhbHJlYWR5IGRvPyBUaGlzIGlzIHByZWNpc2VseQ0KdGhlIHF1ZXN0aW9uIHRo
YXQgSSBhc2tlZCB5b3UgaW4gdGhlIHByZXZpb3VzIGVtYWlsLg0KDQotLSANClRyb25kIE15a2xl
YnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVi
dXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

2012-11-29 14:54:03

by Myklebust, Trond

[permalink] [raw]
Subject: RE: Possible to make nfs aware of a inotify watch has been set.

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVmIEJvbiBbbWFpbHRvOnN0
ZWZib25AZ21haWwuY29tXQ0KPiBTZW50OiBUaHVyc2RheSwgTm92ZW1iZXIgMjksIDIwMTIgOTo0
OSBBTQ0KPiBUbzogTXlrbGVidXN0LCBUcm9uZA0KPiBDYzogbGludXgtbmZzQHZnZXIua2VybmVs
Lm9yZzsgbGludXgtY2lmcw0KPiBTdWJqZWN0OiBSZTogUG9zc2libGUgdG8gbWFrZSBuZnMgYXdh
cmUgb2YgYSBpbm90aWZ5IHdhdGNoIGhhcyBiZWVuIHNldC4NCj4gDQo+IDIwMTIvMTEvMjkgTXlr
bGVidXN0LCBUcm9uZCA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+Og0KPiA+PiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+Pg0KPiA+PiAxLiB3aGlsZSB0aGUgZmlsZXN5c3RlbXMg
YXJlIHVzaW5nIGNyZWRlbnRhaWxzIG9yIHRpY2tldHMgdG8gZ2V0DQo+ID4+IGFjY2VzcyB0byBh
IHJlbW90ZSByZXNvdXJjZSwgdGhpcyBpcyBhIGJpdCBkaWZmaWN1bHQgZm9yIG5vdGlmeWZzLg0K
PiA+PiBOb3RpZnlmcyBieXBhc3NlcyB0aGF0LiBNYXliZSB0aGlzIGxlYWRzIHRvIHBlcm1pc3Np
b25zL2FidXNlIEkNCj4gPj4gY2Fubm90IHNlZSBkaXJlY3RseS4NCj4gPg0KPiA+IExhY2sgb2Yg
c2VjdXJpdHkgaXMgYSBzaG93c3RvcHBlci4gVGhlcmUgYXJlIGdvb2QgcmVhc29ucyB3aHkgaW5v
dGlmeSB3b24ndA0KPiBhbGxvdyB5b3UgdG8gbW9uaXRvciBmaWxlcyBmb3Igd2hpY2ggeW91IGRv
bid0IGhhdmUgYWNjZXNzIHBlcm1pc3Npb25zLg0KPiA+DQo+IA0KPiBMZXQgbWUgZXhwbGFpbiwg
SSB0aGluayB5b3Ugbm90IHVuZGVyc3RhbmQgZnVsbHkuDQo+IA0KPiBOb3RpZnlmcyBkb2VzIG5v
dCBhbGxvdyB1c2Vycy9jbGllbnRzIHRvIHNldCBhIHdhdGNoIGlmIHRoZXJlIGFyZSBubyByZWFk
DQo+IHBlcm1pc3Npb25zICh0aGUgb2JqZWN0IGFuZCBhY2Nlc3MgZm9yIHRoZSB3aG9sZSBwYXRo
IHRvIGl0KSwgc28gdGhlcmUgYXJlIG5vDQo+IHNlY3VyaXR5IGlzc3VlcyB0aGVyZS4NCj4gDQo+
IFdoYXQgSSBtZWFuIGlzIHRoYXQgYW55IHByb2dyYW0gY2FuIGNvbnRhY3QgdGhlIHJlbW90ZSBu
b3RpZnlmcyBzZXJ2ZXIsIGFuZA0KPiB0aGlzIHJlbW90ZSBub3RpZnlmcyBzZXJ2ZXIgY2Fubm90
IGZpZ3VyZSBvdXQgaXQncyBhIHZhbGlkIHJlcXVlc3QgZnJvbSBhbm90aGVyDQo+IG5vdGlmeWZz
IHNlcnZlciwgb3IgYSBwcm9ncmFtIGZha2luZyB0aGF0Lg0KPiBJbiB0aGUgY29uc3RydWN0aW9u
IEkgZGVzY3JpYmUgaXQgZG9lcyBub3QgY2hlY2sgdGhhdCAoeWV0KS4NCj4gDQo+ID4+DQo+ID4+
IFdoYXQgZG8geW91IHRoaW5rLCBpcyB0aGUgbGF0ZXN0IG9wdGlvbiBwb3NzaWJsZT8/DQo+ID4N
Cj4gPiBTbyB3aGF0IGlzIHRoZSBraWxsZXIgYXBwIGZvciBpbm90aWZ5IG9uIE5GUy9DSUZTL0ZV
U0U/IFdoYXQgcHJvZ3JhbXMgZG8NCj4geW91IG5lZWQgdG8gcnVuIG9uIGEgTkZTL0NJRlMvRlVT
RSBjbGllbnQgdGhhdCB1c2UgaW5vdGlmeSBhbmQgdGhhdCB3b3VsZG4ndA0KPiBiZSBiZXR0ZXIg
b2ZmIHJ1bm5pbmcgb24gdGhlIHNlcnZlciBpbnN0ZWFkPw0KPiA+DQo+IA0KPiBXaGF0IGRvIHlv
dSBtZWFuIHdpdGggImJldHRlciBvZmYgcnVubmluZyBvbiB0aGUgc2VydmVyIGluc3RlYWQiPw0K
PiBUaGVyZSBhcmUgYSBsb3Qgb2YgcHJvZ3JhbXMgaW50ZXJlc3RlZCBpbiBmcyBjaGFuZ2VzLCBs
aWtlIGEgc2ltcGxlIGZpbGUNCj4gbWFuYWdlci4gSSB0aGluayBpdCdzIGEgdmVyeSBuaWNlIGZl
YXR1cmUgdG8gc2VlIGNoYW5nZXMgcmlnaHQgYXdheSBpbiB0aGUgdmlldy4NCj4gSXQncyBub3Qg
YSBraWxsZXIgYXBwLCBidXQgYSB0aGluayB0aGUgd2hvbGUgdXNlciBleHBlcmllbmNlIGlzIGlt
cHJvdmluZyB3aGVuDQo+IHlvdXIgc3lzdGVtIGlzIGFibGUgdG8ga2VlcCBhIHZpZXcgKGxpa2Ug
YSB2aWV3IGluIHRoZSBmaWxlIG1hbmFnZXIpIHVwIHRvDQo+IGRhdGUuDQo+IA0KPiA+IElPVzog
d2hvc2UgcHJvYmxlbSBhcmUgeW91IHRyeWluZyB0byBzb2x2ZT8NCj4gDQo+IEkgdGhpbmsgdGhh
dCBlbmFibGluZyBmcyBub3RpZnkgb24gbmV0d29yayBmaWxlc3lzdGVtcyBsaWtlIG5mcywgY2lm
cyBhbmQgZnVzZSBpcw0KPiBhIGdvb2QgdGhpbmcgKHNlZSBhYm92ZSkuIE9uIHN5c3RlbXMgbGlr
ZSBXaW5kb3dzIGFuZCBpT1Mgc2luY2UgbG9uZyB0aW1lDQo+IHRoaXMgd29ya3MuDQo+IA0KPiBZ
b3UgZG8gbm90IGFncmVlPz8NCg0KTm8uIEkgbGlrZSBoYXZpbmcgYSByZWFzb24gZm9yIGFkZGlu
ZyBrZXJuZWwgZnVuY3Rpb25hbGl0eS4NCg0KVHJvbmQNCiANCg==

2012-11-29 21:09:56

by Myklebust, Trond

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

T24gVGh1LCAyMDEyLTExLTI5IGF0IDE1OjA1IC0wNTAwLCBzaW1vIHdyb3RlOg0KPiBPbiBUaHUs
IDIwMTItMTEtMjkgYXQgMTc6MTEgKzAwMDAsIE15a2xlYnVzdCwgVHJvbmQgd3JvdGU6DQo+ID4g
VGhlIGlkZWEgdGhhdCAiaWYgd2UgYnVpbGQgYSBicmlkZ2UsIHRoZXkgd2lsbCBjb21lIHRvIHVz
IiBpcyB3aGF0DQo+ID4gZ2F2ZQ0KPiA+IHJpc2UgdG8gZG5vdGlmeSwgdGhlbiBpbm90aWZ5IChu
b2JvZHkgY2FtZSwgc28gbGV0J3MgYnVpbGQgYSBiaWdnZXINCj4gPiBicmlkZ2UpIGFuZCBmc25v
dGlmeSAobWFrZSBpdCBhIHRvbGwgYnJpZGdlKS4gTm9ib2R5IGlzIHVzaW5nIHRob3NlDQo+ID4g
aW50ZXJmYWNlcyBtdWNoIG9uIGxvY2FsIGZpbGVzeXN0ZW1zLCBzbyB3aHkgaXMgYWRkaW5nIGl0
IHRvIE5GUyBhbmQNCj4gPiBDSUZTIGdvaW5nIHRvIGJlIHN1Y2ggYSBnYW1lIGNoYW5nZXI/DQo+
IA0KPiBTb3JyeSBUcm9uZCwNCj4gYnV0IHRoaXMgaXMgc2ltcGx5IG5vdCB0cnVlLg0KPiANCj4g
aW5vdGlmeSBpcyBhIGtleSBmZWF0dXJlIGF0IGxlYXN0IGluIHNvbWUgcGFja2FnZXMgSSBrbm93
LCBhbmQgbWFrZXMNCj4gc29tZSB0aGluZ3MgKmEgbG90KiBlYXNpZXIuDQo+IA0KPiBPZiBjb3Vy
c2UgdGhpcyBhcmUgbGVzcyB2aXNpYmxlIGZlYXR1cmVzLCB0aGF0IGFyZSBlYXN5IHRvIG92ZXJs
b29rLCBidXQNCj4gdGhleSBhcmUgbm90IHVudXNlZCBmb3Igc3VyZS4NCg0KU28gd2h5IGlzIGl0
IHNvIGRpZmZpY3VsdCBmb3IgYW55b25lIGluIHRoaXMgdGhyZWFkIHRvIG5hbWUgdGhlc2Uga2ls
bGVyDQphcHBsaWNhdGlvbnMgdGhhdCB3b3VsZCBiZW5lZml0IGZyb20gTkZTK0NJRlMgc3VwcG9y
dD8gSSBrZWVwIGFza2luZywNCmFuZCBhbGwgSSBnZXQgaXMgd2lzaHktd2FzaHkgYW5kIGhhbmQt
d2F2eSBhbnN3ZXJzLg0KDQpXaXRob3V0IHVzZSBjYXNlcywgdGhlcmUgaXMgbm90aGluZyB0byBk
aXNjdXNzIG9uIHRoZSBpbXBsZW1lbnRhdGlvbg0KZnJvbnQuDQoNCi0tIA0KVHJvbmQgTXlrbGVi
dXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1
c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==

2012-11-30 07:10:25

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/30 Steve French <[email protected]>:
> On Thu, Nov 29, 2012 at 5:14 PM, Stef Bon <[email protected]> wrote:
>> 2012/11/29 Al Viro <[email protected]>:
>>> On Thu, Nov 29, 2012 at 09:31:05PM +0100, Stef Bon wrote:
>>>> 2012/11/29 Al Viro <[email protected]>:
>>>> > On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:
>>>> >
>>
>>>> With Ubuntu people started Unity, also without a killer app, but just
>>>> with an opinion, an impression, an idea. And anyone can dislike or
>>>> like it, it's a succes as far as I can see. What the hell is wrong
>>>> with suggesting things based upon an idea/impression?
>>>
>>> IOW, it was a true positive for "([Uu]ser experience)|UX" as BS predictor.
>>> Nice to know...
>>>
>>> Look, you'd been asked to give good reasons for doing some thing; replying with
>>> "it's My Opinion(tm), piss off if that's not enough reason for you" would at
>>> least have been honest. References to UX, as your last reply has confirmed,
>>> had been basically an obfuscated equivalent of that.
>>
>> No no, I do not say "piss off". If that is what my message looked
>> like, I'm sorry for that. But I do not have any killer app or
>> something. I cannot convince you otherwise that saying it's my
>> opinion. I do not have numbers or something like that proving I have a
>> point. I wish I have. But I do understand you do not agree with me,
>> and see things different.
>>>
>>> Sorry for being harsh, but I've had it with the GNOME crowd and their ilk
>>> using just such references to handwave away any questions.
>>>
>>> There's nothing wrong with having opinions/impressions/etc. Everyone got
>>> their own, etc. But "I believe so, period" is not a sufficient answer to
>>> "what makes that a good idea?" and obfuscating it up is even worse.
>>
>> So, no I do not say "I believe so, period" . But I cannot convince you
>> ...and we have different opinions here.. I don't know any method here
>> of "proving having a point".
>
> One question that hasn't been answered AFAIK is how well the current
> Linux directory change notification API maps to the needs of its principal user
> (presumably Samba, and kde/gnome file managers), and a loosely
> related question of how well the change notification API maps to the
> corresponding network file system protocol operations
> (TRANSACT_NOTIFY_CHANGE) which is a fairly common operation
> sent to Samba, and obviously for the cifs/smb2/smb3 case would limit
> what types of events the client can notify the desktop/file manager about.
>
> In particular, the 10 or 15 filter flags, and the (at least) 8 events
> (add/remove/modify/rename etc.),
> available in tne network protocol AFAIK have not been matched to see if they
> match closely to the existing Linux API.

Hi,

I start working on the seperate notifyfs servers.
The whole advantage of that approach is that you can design the api
from scratch. Notifyfs is not bound to the api of inotify. There
remains the problem of mapping it to the fs event api on the host,
which will be inotify probably on linux.

At this moment the different events look like:

#define NOTIFYFS_FSEVENT_META_ATTRIB_NOTSET 2
#define NOTIFYFS_FSEVENT_META_ATTRIB_MODE 4
#define NOTIFYFS_FSEVENT_META_ATTRIB_OWNER 8
#define NOTIFYFS_FSEVENT_META_ATTRIB_GROUP 16
#define NOTIFYFS_FSEVENT_META_ATTRIB 28

#define NOTIFYFS_FSEVENT_META_XATTR_NOTSET 32
#define NOTIFYFS_FSEVENT_META_XATTR_CREATE 64
#define NOTIFYFS_FSEVENT_META_XATTR_MODIFY 128
#define NOTIFYFS_FSEVENT_META_XATTR_DELETE 256
#define NOTIFYFS_FSEVENT_META_XATTR 448

#define NOTIFYFS_FSEVENT_FILE_NOTSET 512
#define NOTIFYFS_FSEVENT_FILE_MODIFIED 1024
#define NOTIFYFS_FSEVENT_FILE_SIZE 2048
#define NOTIFYFS_FSEVENT_FILE_OPEN 4096
#define NOTIFYFS_FSEVENT_FILE_READ 8192
#define NOTIFYFS_FSEVENT_FILE_CLOSE_WRITE 16384
#define NOTIFYFS_FSEVENT_FILE_CLOSE_NOWRITE 32768

#define NOTIFYFS_FSEVENT_MOVE_NOTSET 65536
#define NOTIFYFS_FSEVENT_MOVE_CREATED 131072
#define NOTIFYFS_FSEVENT_MOVE_MOVED 262144
#define NOTIFYFS_FSEVENT_MOVE_MOVED_FROM 524288
#define NOTIFYFS_FSEVENT_MOVE_MOVED_TO 1048576
#define NOTIFYFS_FSEVENT_MOVE_DELETED 2097152

#define NOTIFYFS_FSEVENT_FS_NOTSET 4194304
#define NOTIFYFS_FSEVENT_FS_MOUNT 8388608
#define NOTIFYFS_FSEVENT_FS_UNMOUNT 16777216
#define NOTIFYFS_FSEVENT_FS_NLINKS 33554432

Note that there is no such thing as watching changing xattr at this
moment right now on linux. The field st_ctime is changed, nothing
more.

Stef

2012-11-29 16:28:05

by Steve French

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, Nov 29, 2012 at 9:33 AM, Myklebust, Trond
<[email protected]> wrote:
> On Thu, 2012-11-29 at 10:22 -0500, simo wrote:
>> On Thu, 2012-11-29 at 15:49 +0100, Stef Bon wrote:
>> > 2012/11/29 Myklebust, Trond <[email protected]>:
>> > >> -----Original Message-----
>> > >>
>> > >> 1. while the filesystems are using credentails or tickets to get access to a
>> > >> remote resource, this is a bit difficult for notifyfs.
>> > >> Notifyfs bypasses that. Maybe this leads to permissions/abuse I cannot see
>> > >> directly.
>> > >
>> > > Lack of security is a showstopper. There are good reasons why inotify won't allow you to monitor files for which you don't have access permissions.
>> > >
>> >
>> > Let me explain, I think you not understand fully.
>> >
>> > Notifyfs does not allow users/clients to set a watch if there are no
>> > read permissions (the object and access for the whole path to it), so
>> > there are no security issues there.
>> >
>> > What I mean is that any program can contact the remote notifyfs
>> > server, and this remote notifyfs server cannot figure out it's a valid
>> > request from another notifyfs server, or a program faking that.
>> > In the construction I describe it does not check that (yet).
>> >
>> > >>
>> > >> What do you think, is the latest option possible??
>> > >
>> > > So what is the killer app for inotify on NFS/CIFS/FUSE? What programs do you need to run on a NFS/CIFS/FUSE client that use inotify and that wouldn't be better off running on the server instead?
>> > >
>> >
>> > What do you mean with "better off running on the server instead"?
>> > There are a lot of programs interested in fs changes, like a simple
>> > file manager. I think it's a very nice feature to see changes right
>> > away in the view.
>> > It's not a killer app, but a think the whole user experience is
>> > improving when your system is able to keep a view (like a view in the
>> > file manager) up to date.
>> >
>> > > IOW: whose problem are you trying to solve?
>> >
>> > I think that enabling fs notify on network filesystems like nfs, cifs
>> > and fuse is a good thing (see above). On systems like Windows and iOS
>> > since long time this works.
>>
>> CIFS has notification capabilities built in (oplocks), as does NFS
>> (leases), is this not sufficient ?
>
> NFSv4.1 actually has directory notifications which duplicate most of the
> functionality of dnotify. The question I'm asking is "Why should we do
> it?", not "can we do it?".
>
> Answers like "well Windows and iOS do it" aren't helpful unless they
> include a description of what Windows and iOS apps use it for that we
> can't already do on Linux.
> If the only application is beagle, then Linux has this really helpful
> utility called "ssh", which allows you to avoid wasting a load of
> network bandwidth...

Of course one reason that file managers like directory notifications is that
it avoids wasting a load of network bandwidth - as the alternative to
directory change
notifications is polling the directories for every open window. Also note that
network file systems where possible should implement similar semantics and
interfaces to the local file systems, and inotify/dnotify are mappable
to network
'file systems (presumably not just cifs/smb2/smb3). Support for
keeping file manager
views reasonably up to date would be a good enough reason for something like
inotify/dnotify (and it has been used by file managers for over 20
years, even before
Windows and iOS) - and we want to get to the point where home directories
work as well on network mounts as on local storage.

Directory oplocks (in smb3) may be helpful in reducing the number of remote
directory change notifications further.



--
Thanks,

Steve

2012-11-29 23:44:05

by Steve French

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

On Thu, Nov 29, 2012 at 5:14 PM, Stef Bon <[email protected]> wrote:
> 2012/11/29 Al Viro <[email protected]>:
>> On Thu, Nov 29, 2012 at 09:31:05PM +0100, Stef Bon wrote:
>>> 2012/11/29 Al Viro <[email protected]>:
>>> > On Thu, Nov 29, 2012 at 05:39:27PM +0100, Stef Bon wrote:
>>> >
>
>>> With Ubuntu people started Unity, also without a killer app, but just
>>> with an opinion, an impression, an idea. And anyone can dislike or
>>> like it, it's a succes as far as I can see. What the hell is wrong
>>> with suggesting things based upon an idea/impression?
>>
>> IOW, it was a true positive for "([Uu]ser experience)|UX" as BS predictor.
>> Nice to know...
>>
>> Look, you'd been asked to give good reasons for doing some thing; replying with
>> "it's My Opinion(tm), piss off if that's not enough reason for you" would at
>> least have been honest. References to UX, as your last reply has confirmed,
>> had been basically an obfuscated equivalent of that.
>
> No no, I do not say "piss off". If that is what my message looked
> like, I'm sorry for that. But I do not have any killer app or
> something. I cannot convince you otherwise that saying it's my
> opinion. I do not have numbers or something like that proving I have a
> point. I wish I have. But I do understand you do not agree with me,
> and see things different.
>>
>> Sorry for being harsh, but I've had it with the GNOME crowd and their ilk
>> using just such references to handwave away any questions.
>>
>> There's nothing wrong with having opinions/impressions/etc. Everyone got
>> their own, etc. But "I believe so, period" is not a sufficient answer to
>> "what makes that a good idea?" and obfuscating it up is even worse.
>
> So, no I do not say "I believe so, period" . But I cannot convince you
> ...and we have different opinions here.. I don't know any method here
> of "proving having a point".

One question that hasn't been answered AFAIK is how well the current
Linux directory change notification API maps to the needs of its principal user
(presumably Samba, and kde/gnome file managers), and a loosely
related question of how well the change notification API maps to the
corresponding network file system protocol operations
(TRANSACT_NOTIFY_CHANGE) which is a fairly common operation
sent to Samba, and obviously for the cifs/smb2/smb3 case would limit
what types of events the client can notify the desktop/file manager about.

In particular, the 10 or 15 filter flags, and the (at least) 8 events
(add/remove/modify/rename etc.),
available in tne network protocol AFAIK have not been matched to see if they
match closely to the existing Linux API.

--
Thanks,

Steve

2012-11-29 16:39:28

by Stef Bon

[permalink] [raw]
Subject: Re: Possible to make nfs aware of a inotify watch has been set.

2012/11/29 Myklebust, Trond <[email protected]>:
> On Thu, 2012-11-29 at 10:22 -0500, simo wrote:
>> On Thu, 2012-11-29 at 15:49 +0100, Stef Bon wrote:
>> > 2012/11/29 Myklebust, Trond <[email protected]>:
>> > >> -----Original Message-----
>> > >>
> NFSv4.1 actually has directory notifications which duplicate most of the
> functionality of dnotify. The question I'm asking is "Why should we do
> it?", not "can we do it?".
>
> Answers like "well Windows and iOS do it" aren't helpful unless they
> include a description of what Windows and iOS apps use it for that we
> can't already do on Linux.
> If the only application is beagle, then Linux has this really helpful
> utility called "ssh", which allows you to avoid wasting a load of
> network bandwidth...

No, sure, you should not do anything because others do it. But on the
other hand, if others do it, why don't you? Better copy a good idea
than be stubborn for ever.

And yes there is no such thing as a killer app, but in my opinion it's
the user experience. It's so much better when your system keeps track
on shared network resources.

But first things first,I will try to make the first option work, which
was the use of a remote notifyfs server. This works of course only
with linux like systems: so not when the remote server is a Windows
server.

I know that SMB (2?) has the ability to send a SMB2_CHANGE_NOTIFY
request. Steve French and I had earlier a conversation about that.

Before trying to make this work with cifs I've tried to make this work
using a FUSE fs with libsmbclient. I did not get it working since the
adding of an fd to the FUSE fs's eventloop was not easy, and required
a rewrite of libsmbclient.
The writing of the FUSE fs was easy, since there are already some examples.

Stef