In NFSv4, the server can report which attributes it chose to return in a
READDIR reply.
A customer has come across a server which does not return the filehandle
information (is that allowed?).
A consequence of this is that Linux/NFS gets confused.
nfs_readdir_page_filler calls nfs_prime_dcache() (because it was a readdir
plus request that was sent) and nfs_prime_dcache goes ahead and creates an
inode based on the filehandle that it has.
However decode_attr_filehandle() had happily decoded nothing as the
FATTR4_WORD0_FILEHANDLE bit wasn't set.
So the inode gets created with a zero-length filehandle and when this gets
sent back to the server to act on the inode, it gets NFS4ERR_BADHANDLE to
the PUTFH op.
So should nfs_prime_dcache() abort if the filehandle doesn't exist (patch
below) or should nfs_fhget() return an error if the filehandle is empty?
Or maybe this behaviour should be detected and readdir should be disabled for
that server?
Suggestions?
Thanks,
NeilBrown
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 627f108..148d09c 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -442,6 +442,10 @@ void nfs_prime_dcache(struct dentry *parent, struct nfs_entry *entry)
if (filename.len == 2 && filename.name[1] == '.')
return;
}
+ if (entry->fh.size == 0)
+ /* Server didn't return a filehandle */
+ return;
+
filename.hash = full_name_hash(filename.name, filename.len);
dentry = d_lookup(parent, &filename);
On Fri, 21 Sep 2012 02:44:10 +0000
"Myklebust, Trond" <[email protected]> wrote:
> On Tue, 2012-09-18 at 12:04 +1000, NeilBrown wrote:
> > On Mon, 17 Sep 2012 12:51:33 +0000 "Myklebust, Trond"
> > <[email protected]> wrote:
> >
> > > > -----Original Message-----
> > > > From: [email protected] [mailto:linux-nfs-
> > > > [email protected]] On Behalf Of NeilBrown
> > > > Sent: Sunday, September 16, 2012 7:06 PM
> > > > To: [email protected]
> > > > Subject: What is NFSv4 READDIR doesn't return a filehandle....
> > > >
> > > >
> > > > In NFSv4, the server can report which attributes it chose to return in a
> > > > READDIR reply.
> > > >
> > > > A customer has come across a server which does not return the filehandle
> > > > information (is that allowed?).
> > >
> > > The filehandle attribute is a mandatory attribute according to RFC3530, so I believe that the answer is "no".
> > >
> > > > A consequence of this is that Linux/NFS gets confused.
> > > > nfs_readdir_page_filler calls nfs_prime_dcache() (because it was a readdir
> > > > plus request that was sent) and nfs_prime_dcache goes ahead and creates
> > > > an inode based on the filehandle that it has.
> > > > However decode_attr_filehandle() had happily decoded nothing as the
> > > > FATTR4_WORD0_FILEHANDLE bit wasn't set.
> > > > So the inode gets created with a zero-length filehandle and when this gets
> > > > sent back to the server to act on the inode, it gets NFS4ERR_BADHANDLE to
> > > > the PUTFH op.
> > > >
> > > > So should nfs_prime_dcache() abort if the filehandle doesn't exist (patch
> > > > below) or should nfs_fhget() return an error if the filehandle is empty?
> > > >
> > > > Or maybe this behaviour should be detected and readdir should be disabled
> > > > for that server?
> > > >
> > >
> > > I don't want to have to code the client to deal with broken servers. If we start down that path, then we'll end up doing nothing else.
> > >
> > > I can, however, see a case for extending the "nordirplus" mount option to cover NFSv4. Currently it only acts on NFSv3 mounts...
> > >
> > >
> >
> > Thanks Trond.
> > I'm happy with this position - less work for me :-)
> >
> > As it happens, nordirplus *does* work for NFSv4 and customer had already
> > found that this is a successful work around. They didn't want to have to use
> > it though. I've pointed out that is really isn't our problem.
>
> Good! I was worried that nordirplus wasn't working for NFSv4.
>
> If there is an existing workaround, then I do not at all accept the
> argument that we need to add client-side patches to accommodate
> brokenness on the NFS servers.
>
> If there is no existing workaround, then I'm willing to help people with
> a temporary fix while they wait for the server vendors to get their act
> together. However ultimately, I don't believe in fixing server bugs on
> the client: those temporary fixes should certainly not be finding their
> way into the upstream kernel, and as a consequence they should probably
> not go into distribution kernels either (although that is your call, and
> not mine :-)).
>
> > Just a thought: while coping with broken servers would not be a good path to
> > follow, detecting protocol violations and reporting an error might be...
> > should the NFS client treat a missing filehandle and a malformed reply?
>
> My concern is that the client can't objectively judge what constitutes a
> valid filehandle and what does not until it tries to use it in an RPC
> call.
> Given that premise, it makes more sense to concentrate on handling the
> cases where the usage fails. Jeff Layton's vfs ESTALE patches are a good
> case in point.
>
Wait though -- is it not safe to assume that a zero length filehandle
is invalid?
Neil's earlier patch checked for entry->fh.size == 0, would it not be
reasonable to warn once per server when we get back such a fh?
--
Jeff Layton <[email protected]>
> -----Original Message-----
> From: [email protected] [mailto:linux-nfs-
> [email protected]] On Behalf Of NeilBrown
> Sent: Sunday, September 16, 2012 7:06 PM
> To: [email protected]
> Subject: What is NFSv4 READDIR doesn't return a filehandle....
>
>
> In NFSv4, the server can report which attributes it chose to return in a
> READDIR reply.
>
> A customer has come across a server which does not return the filehandle
> information (is that allowed?).
The filehandle attribute is a mandatory attribute according to RFC3530, so I believe that the answer is "no".
> A consequence of this is that Linux/NFS gets confused.
> nfs_readdir_page_filler calls nfs_prime_dcache() (because it was a readdir
> plus request that was sent) and nfs_prime_dcache goes ahead and creates
> an inode based on the filehandle that it has.
> However decode_attr_filehandle() had happily decoded nothing as the
> FATTR4_WORD0_FILEHANDLE bit wasn't set.
> So the inode gets created with a zero-length filehandle and when this gets
> sent back to the server to act on the inode, it gets NFS4ERR_BADHANDLE to
> the PUTFH op.
>
> So should nfs_prime_dcache() abort if the filehandle doesn't exist (patch
> below) or should nfs_fhget() return an error if the filehandle is empty?
>
> Or maybe this behaviour should be detected and readdir should be disabled
> for that server?
>
I don't want to have to code the client to deal with broken servers. If we start down that path, then we'll end up doing nothing else.
I can, however, see a case for extending the "nordirplus" mount option to cover NFSv4. Currently it only acts on NFSv3 mounts...
Cheers
Trond
T24gU3VuLCAyMDEyLTA5LTIzIGF0IDIwOjQxIC0wNDAwLCBKZWZmIExheXRvbiB3cm90ZToNCj4g
V2FpdCB0aG91Z2ggLS0gaXMgaXQgbm90IHNhZmUgdG8gYXNzdW1lIHRoYXQgYSB6ZXJvIGxlbmd0
aCBmaWxlaGFuZGxlDQo+IGlzIGludmFsaWQ/DQo+IA0KPiBOZWlsJ3MgZWFybGllciBwYXRjaCBj
aGVja2VkIGZvciBlbnRyeS0+Zmguc2l6ZSA9PSAwLCB3b3VsZCBpdCBub3QgYmUNCj4gcmVhc29u
YWJsZSB0byB3YXJuIG9uY2UgcGVyIHNlcnZlciB3aGVuIHdlIGdldCBiYWNrIHN1Y2ggYSBmaD8N
Cg0KVGhlIHBvaW50IGlzIHRoYXQgc2FuaXR5IGNoZWNraW5nIG9mIGZpbGVoYW5kbGVzIGJ5IHRo
ZSBjbGllbnQgaXMgYQ0KdG90YWxseSBfaW5zYW5lXyBjb25jZXB0LiBZZXMgd2UgY2FuIGRvIGl0
IGZvciBhIGNvdXBsZSBvZiBjb3JuZXIgY2FzZXMuDQpObywgd2Ugc3VyZSBhcyBoZWxsIGRvIE5P
VCB3YW50IHRvIGdldCBzdHVjayBkb2luZyBpdCBmb3IgdGhlIGdlbmVyYWwNCmNhc2UuDQoNClRy
b25kDQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXIN
Cg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==
T24gVHVlLCAyMDEyLTA5LTE4IGF0IDEyOjA0ICsxMDAwLCBOZWlsQnJvd24gd3JvdGU6DQo+IE9u
IE1vbiwgMTcgU2VwIDIwMTIgMTI6NTE6MzMgKzAwMDAgIk15a2xlYnVzdCwgVHJvbmQiDQo+IDxU
cm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbT4gd3JvdGU6DQo+IA0KPiA+ID4gLS0tLS1PcmlnaW5h
bCBNZXNzYWdlLS0tLS0NCj4gPiA+IEZyb206IGxpbnV4LW5mcy1vd25lckB2Z2VyLmtlcm5lbC5v
cmcgW21haWx0bzpsaW51eC1uZnMtDQo+ID4gPiBvd25lckB2Z2VyLmtlcm5lbC5vcmddIE9uIEJl
aGFsZiBPZiBOZWlsQnJvd24NCj4gPiA+IFNlbnQ6IFN1bmRheSwgU2VwdGVtYmVyIDE2LCAyMDEy
IDc6MDYgUE0NCj4gPiA+IFRvOiBsaW51eC1uZnNAdmdlci5rZXJuZWwub3JnDQo+ID4gPiBTdWJq
ZWN0OiBXaGF0IGlzIE5GU3Y0IFJFQURESVIgZG9lc24ndCByZXR1cm4gYSBmaWxlaGFuZGxlLi4u
Lg0KPiA+ID4gDQo+ID4gPiANCj4gPiA+IEluIE5GU3Y0LCB0aGUgc2VydmVyIGNhbiByZXBvcnQg
d2hpY2ggYXR0cmlidXRlcyBpdCBjaG9zZSB0byByZXR1cm4gaW4gYQ0KPiA+ID4gUkVBRERJUiBy
ZXBseS4NCj4gPiA+IA0KPiA+ID4gQSBjdXN0b21lciBoYXMgY29tZSBhY3Jvc3MgYSBzZXJ2ZXIg
d2hpY2ggZG9lcyBub3QgcmV0dXJuIHRoZSBmaWxlaGFuZGxlDQo+ID4gPiBpbmZvcm1hdGlvbiAo
aXMgdGhhdCBhbGxvd2VkPykuDQo+ID4gDQo+ID4gVGhlIGZpbGVoYW5kbGUgYXR0cmlidXRlIGlz
IGEgbWFuZGF0b3J5IGF0dHJpYnV0ZSBhY2NvcmRpbmcgdG8gUkZDMzUzMCwgc28gSSBiZWxpZXZl
IHRoYXQgdGhlIGFuc3dlciBpcyAibm8iLg0KPiA+IA0KPiA+ID4gQSBjb25zZXF1ZW5jZSBvZiB0
aGlzIGlzIHRoYXQgTGludXgvTkZTIGdldHMgY29uZnVzZWQuDQo+ID4gPiBuZnNfcmVhZGRpcl9w
YWdlX2ZpbGxlciBjYWxscyBuZnNfcHJpbWVfZGNhY2hlKCkgKGJlY2F1c2UgaXQgd2FzIGEgcmVh
ZGRpcg0KPiA+ID4gcGx1cyByZXF1ZXN0IHRoYXQgd2FzIHNlbnQpIGFuZCBuZnNfcHJpbWVfZGNh
Y2hlIGdvZXMgYWhlYWQgYW5kIGNyZWF0ZXMNCj4gPiA+IGFuIGlub2RlIGJhc2VkIG9uIHRoZSBm
aWxlaGFuZGxlIHRoYXQgaXQgaGFzLg0KPiA+ID4gSG93ZXZlciBkZWNvZGVfYXR0cl9maWxlaGFu
ZGxlKCkgaGFkIGhhcHBpbHkgZGVjb2RlZCBub3RoaW5nIGFzIHRoZQ0KPiA+ID4gRkFUVFI0X1dP
UkQwX0ZJTEVIQU5ETEUgYml0IHdhc24ndCBzZXQuDQo+ID4gPiBTbyB0aGUgaW5vZGUgZ2V0cyBj
cmVhdGVkIHdpdGggYSB6ZXJvLWxlbmd0aCBmaWxlaGFuZGxlIGFuZCB3aGVuIHRoaXMgZ2V0cw0K
PiA+ID4gc2VudCBiYWNrIHRvIHRoZSBzZXJ2ZXIgdG8gYWN0IG9uIHRoZSBpbm9kZSwgaXQgZ2V0
cyBORlM0RVJSX0JBREhBTkRMRSB0bw0KPiA+ID4gdGhlIFBVVEZIIG9wLg0KPiA+ID4gDQo+ID4g
PiBTbyBzaG91bGQgbmZzX3ByaW1lX2RjYWNoZSgpIGFib3J0IGlmIHRoZSBmaWxlaGFuZGxlIGRv
ZXNuJ3QgZXhpc3QgKHBhdGNoDQo+ID4gPiBiZWxvdykgb3Igc2hvdWxkIG5mc19maGdldCgpIHJl
dHVybiBhbiBlcnJvciBpZiB0aGUgZmlsZWhhbmRsZSBpcyBlbXB0eT8NCj4gPiA+IA0KPiA+ID4g
T3IgbWF5YmUgdGhpcyBiZWhhdmlvdXIgc2hvdWxkIGJlIGRldGVjdGVkIGFuZCByZWFkZGlyIHNo
b3VsZCBiZSBkaXNhYmxlZA0KPiA+ID4gZm9yIHRoYXQgc2VydmVyPw0KPiA+ID4gDQo+ID4gDQo+
ID4gSSBkb24ndCB3YW50IHRvIGhhdmUgdG8gY29kZSB0aGUgY2xpZW50IHRvIGRlYWwgd2l0aCBi
cm9rZW4gc2VydmVycy4gSWYgd2Ugc3RhcnQgZG93biB0aGF0IHBhdGgsIHRoZW4gd2UnbGwgZW5k
IHVwIGRvaW5nIG5vdGhpbmcgZWxzZS4NCj4gPiANCj4gPiBJIGNhbiwgaG93ZXZlciwgc2VlIGEg
Y2FzZSBmb3IgZXh0ZW5kaW5nIHRoZSAibm9yZGlycGx1cyIgbW91bnQgb3B0aW9uIHRvIGNvdmVy
IE5GU3Y0LiBDdXJyZW50bHkgaXQgb25seSBhY3RzIG9uIE5GU3YzIG1vdW50cy4uLg0KPiA+IA0K
PiA+DQo+IA0KPiBUaGFua3MgVHJvbmQuDQo+IEknbSBoYXBweSB3aXRoIHRoaXMgcG9zaXRpb24g
LSBsZXNzIHdvcmsgZm9yIG1lIDotKQ0KPiANCj4gQXMgaXQgaGFwcGVucywgbm9yZGlycGx1cyAq
ZG9lcyogd29yayBmb3IgTkZTdjQgYW5kIGN1c3RvbWVyIGhhZCBhbHJlYWR5DQo+IGZvdW5kIHRo
YXQgdGhpcyBpcyBhIHN1Y2Nlc3NmdWwgd29yayBhcm91bmQuICBUaGV5IGRpZG4ndCB3YW50IHRv
IGhhdmUgdG8gdXNlDQo+IGl0IHRob3VnaC4gIEkndmUgcG9pbnRlZCBvdXQgdGhhdCBpcyByZWFs
bHkgaXNuJ3Qgb3VyIHByb2JsZW0uDQoNCkdvb2QhIEkgd2FzIHdvcnJpZWQgdGhhdCBub3JkaXJw
bHVzIHdhc24ndCB3b3JraW5nIGZvciBORlN2NC4NCg0KSWYgdGhlcmUgaXMgYW4gZXhpc3Rpbmcg
d29ya2Fyb3VuZCwgdGhlbiBJIGRvIG5vdCBhdCBhbGwgYWNjZXB0IHRoZQ0KYXJndW1lbnQgdGhh
dCB3ZSBuZWVkIHRvIGFkZCBjbGllbnQtc2lkZSBwYXRjaGVzIHRvIGFjY29tbW9kYXRlDQpicm9r
ZW5uZXNzIG9uIHRoZSBORlMgc2VydmVycy4NCg0KSWYgdGhlcmUgaXMgbm8gZXhpc3Rpbmcgd29y
a2Fyb3VuZCwgdGhlbiBJJ20gd2lsbGluZyB0byBoZWxwIHBlb3BsZSB3aXRoDQphIHRlbXBvcmFy
eSBmaXggd2hpbGUgdGhleSB3YWl0IGZvciB0aGUgc2VydmVyIHZlbmRvcnMgdG8gZ2V0IHRoZWly
IGFjdA0KdG9nZXRoZXIuIEhvd2V2ZXIgdWx0aW1hdGVseSwgSSBkb24ndCBiZWxpZXZlIGluIGZp
eGluZyBzZXJ2ZXIgYnVncyBvbg0KdGhlIGNsaWVudDogdGhvc2UgdGVtcG9yYXJ5IGZpeGVzIHNo
b3VsZCBjZXJ0YWlubHkgbm90IGJlIGZpbmRpbmcgdGhlaXINCndheSBpbnRvIHRoZSB1cHN0cmVh
bSBrZXJuZWwsIGFuZCBhcyBhIGNvbnNlcXVlbmNlIHRoZXkgc2hvdWxkIHByb2JhYmx5DQpub3Qg
Z28gaW50byBkaXN0cmlidXRpb24ga2VybmVscyBlaXRoZXIgKGFsdGhvdWdoIHRoYXQgaXMgeW91
ciBjYWxsLCBhbmQNCm5vdCBtaW5lIDotKSkuDQoNCj4gSnVzdCBhIHRob3VnaHQ6IHdoaWxlIGNv
cGluZyB3aXRoIGJyb2tlbiBzZXJ2ZXJzIHdvdWxkIG5vdCBiZSBhIGdvb2QgcGF0aCB0bw0KPiBm
b2xsb3csIGRldGVjdGluZyBwcm90b2NvbCB2aW9sYXRpb25zIGFuZCByZXBvcnRpbmcgYW4gZXJy
b3IgbWlnaHQgYmUuLi4NCj4gc2hvdWxkIHRoZSBORlMgY2xpZW50IHRyZWF0IGEgbWlzc2luZyBm
aWxlaGFuZGxlIGFuZCBhIG1hbGZvcm1lZCByZXBseT8NCg0KTXkgY29uY2VybiBpcyB0aGF0IHRo
ZSBjbGllbnQgY2FuJ3Qgb2JqZWN0aXZlbHkganVkZ2Ugd2hhdCBjb25zdGl0dXRlcyBhDQp2YWxp
ZCBmaWxlaGFuZGxlIGFuZCB3aGF0IGRvZXMgbm90IHVudGlsIGl0IHRyaWVzIHRvIHVzZSBpdCBp
biBhbiBSUEMNCmNhbGwuDQpHaXZlbiB0aGF0IHByZW1pc2UsIGl0IG1ha2VzIG1vcmUgc2Vuc2Ug
dG8gY29uY2VudHJhdGUgb24gaGFuZGxpbmcgdGhlDQpjYXNlcyB3aGVyZSB0aGUgdXNhZ2UgZmFp
bHMuIEplZmYgTGF5dG9uJ3MgdmZzIEVTVEFMRSBwYXRjaGVzIGFyZSBhIGdvb2QNCmNhc2UgaW4g
cG9pbnQuDQoNCkNoZWVycw0KICBUcm9uZA0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXgg
TkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5j
b20NCnd3dy5uZXRhcHAuY29tDQo=
On Mon, 17 Sep 2012 12:51:33 +0000 "Myklebust, Trond"
<[email protected]> wrote:
> > -----Original Message-----
> > From: [email protected] [mailto:linux-nfs-
> > [email protected]] On Behalf Of NeilBrown
> > Sent: Sunday, September 16, 2012 7:06 PM
> > To: [email protected]
> > Subject: What is NFSv4 READDIR doesn't return a filehandle....
> >
> >
> > In NFSv4, the server can report which attributes it chose to return in a
> > READDIR reply.
> >
> > A customer has come across a server which does not return the filehandle
> > information (is that allowed?).
>
> The filehandle attribute is a mandatory attribute according to RFC3530, so I believe that the answer is "no".
>
> > A consequence of this is that Linux/NFS gets confused.
> > nfs_readdir_page_filler calls nfs_prime_dcache() (because it was a readdir
> > plus request that was sent) and nfs_prime_dcache goes ahead and creates
> > an inode based on the filehandle that it has.
> > However decode_attr_filehandle() had happily decoded nothing as the
> > FATTR4_WORD0_FILEHANDLE bit wasn't set.
> > So the inode gets created with a zero-length filehandle and when this gets
> > sent back to the server to act on the inode, it gets NFS4ERR_BADHANDLE to
> > the PUTFH op.
> >
> > So should nfs_prime_dcache() abort if the filehandle doesn't exist (patch
> > below) or should nfs_fhget() return an error if the filehandle is empty?
> >
> > Or maybe this behaviour should be detected and readdir should be disabled
> > for that server?
> >
>
> I don't want to have to code the client to deal with broken servers. If we start down that path, then we'll end up doing nothing else.
>
> I can, however, see a case for extending the "nordirplus" mount option to cover NFSv4. Currently it only acts on NFSv3 mounts...
>
>
Thanks Trond.
I'm happy with this position - less work for me :-)
As it happens, nordirplus *does* work for NFSv4 and customer had already
found that this is a successful work around. They didn't want to have to use
it though. I've pointed out that is really isn't our problem.
Just a thought: while coping with broken servers would not be a good path to
follow, detecting protocol violations and reporting an error might be...
should the NFS client treat a missing filehandle and a malformed reply?
Thanks,
NeilBrown
J. Bruce Fields wrote:
> On Wed, Mar 20, 2013 at 09:41:40PM -0400, Rick Macklem wrote:
> > Well, I don't see any difference between a mandatory attribute and a
> > recommended attribute that the server claims to support via the
> > supported_attributes attribute.
> >
> > I do believe that the server can choose not to return all of the
> > mandatory and supported recommended attributes in the readdir reply.
> > (If not, why have a bitmap of returned attributes?)
> >
> > One example here is the mounted_on_fileid, which some servers choose
> > to only "support" for server mount points. (The FreeBSD server
> > returns
> > this attribute for all file handles, setting it to the same value as
> > fileid for non-mount-points, but I am pretty sure some other servers
> > do not return mounted_on_fileid for non-mount-points.)
>
> That doesn't sound like traditional unix readdir behavior, and isn't
> the
> behavior described by the spec; looking at 5661 5.8.2.23 (haven't
> compared other rfcs):
>
> "If the server detects that there is no mounted point at the
> target file object, then the value for mounted_on_fileid that it
> returns is the same as that of the fileid attribute."
>
> Also, the supported_attrs attribute is a filesystem-wide attribute. I
> don't think we generally allow attributes to vary between "supported"
> and "unsupported" on a single filesystem.
>
> --b.
Well, I'm pretty sure some server did (maybe it has been changed more
recently) would only return mounted_on_fileid in the Readdir reply
if it was at a mount point.
I remember because it broke the FreeBSD client and I had to tweak
the code.
I'll try and remember to stick something in the FreeBSD client to
spot this for the next Bakeathon.
rick
On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> Trond Myklebust wrote:
> > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > >
> > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <[email protected]>
> > > wrote:
> > >
> > > > All,
> > > >
> > > > Sorry for this late posting, a coworker was made aware of this
> > > > thread
> > > > recently and
> > > > has these replies to make. Our server implementation is the one
> > > > being
> > > > complained about in this thread.
> > > >
> > > >
> > > > Quoted text is from various entries in this forum.
> > > >
> > > >
> > > > Neil Brown:
> > > > ===========
> > > >> Just a thought: while coping with broken servers would not be a
> > > >> good
> > > > path to
> > > >> follow, detecting protocol violations and reporting an error
> > > >> might be...
> > > >> should the NFS client treat a missing filehandle and a malformed
> > > >> reply?
> > > >
> > > > The server is allowed to remove an attribute bit from an entry in
> > > > the
> > > > readdir
> > > > reply, this is not "broken" nor is the reply "malformed". The lack
> > > > of a
> > > > filehandle in the reply is deterministic.
> > > >
> > > >
> > > >
> > > > Trond Myklebust:
> > > > ================
> > > >>> A customer has come across a server which does not return the
> > > > filehandle
> > > >>> information (is that allowed?).
> > > >>
> > > >> The filehandle attribute is a mandatory attribute according to
> > > >> RFC3530,
> > > > so I believe that the answer is "no".
> > > >
> > > > Mandatory is described in RFS 3530 as that the server must return
> > > > the
> > > > attribute
> > > > on a GETATTR. (Section 5, page 36). There is nothing saying that
> > > > it is
> > > > mandatory to return on a READDIR. Our server will return the
> > > > filehandle
> > > > on a LOOKUP/GETATTR every time.
> > >
> > >
> > >
> > > GETATTR and READDIR both return attributes. To be precise, they
> > > return
> > > a fattr4.
> > >
> > > Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> > > copy)
> > > of 3530bis (3530 is on the way to being replaced), READDIR states:
> > >
> > > The READDIR operation retrieves a variable number of entries from
> > > a
> > > filesystem directory and returns client requested attributes for
> > > each
> > > entry along with information to allow the client to request
> > > additional directory entries in a subsequent READDIR.
> > > ...
> > > On successful return, the server's response will provide a list
> > > of
> > > directory entries. Each of these entries contains the name of the
> > > directory entry, a cookie value for that entry, and the
> > > associated
> > > attributes as requested. The "eof" flag has a value of TRUE if
> > > there
> > > are no more entries in the directory.
> > >
> > > Any client implementation has no way to request that any server
> > > implementation
> > > return the filehandle because the filehandle is not an attribute
> > > which
> > > can be requested. I.e., it is mandatory.
> > >
> > > If the intent was to allow the server to not return a filehandle for
> > > READDIR but to
> > > allow it to return one for GETATTR, then it would have been made
> > > OPTIONAL.
> > >
> Well, I don't see any difference between a mandatory attribute and a
> recommended attribute that the server claims to support via the
> supported_attributes attribute.
>
> I do believe that the server can choose not to return all of the
> mandatory and supported recommended attributes in the readdir reply.
> (If not, why have a bitmap of returned attributes?)
That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
states:
"A client may ask for any of these attributes to be returned by setting
a bit in the GETATTR request but must handle the case where the server
does not return them."
Section 5.1 says:
"These MUST be supported by every NFS version 4 client and server in
order to ensure a minimum level of interoperability." There are no
exceptions made for READDIR.
> One example here is the mounted_on_fileid, which some servers choose
> to only "support" for server mount points. (The FreeBSD server returns
> this attribute for all file handles, setting it to the same value as
> fileid for non-mount-points, but I am pretty sure some other servers
> do not return mounted_on_fileid for non-mount-points.)
>
> > > Whether or not the client used to work with such a server
> > > implementation
> > > or not is immaterial as far as standard compliance is concerned.
> > >
> > > If you like, we can clean up the corresponding language in 3530bis
> > > to
> > > explicitly state that REQUIRED attributes are indeed required
> > > whether
> > > in response to GETATTR or READDIR.
> I assume that by REQUIRED you are referring to "mandatory attributes".
I'm using the language from RFC5661 and RFC3530-bis, since that is the
most up-to-date.
> > It might rather be useful to add language to point out that the
> > "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> > have
> > GETFH, and AFAICR, all the language around referrals etc is written
> > with
> > the assumption that clients _won't_ use GETATTR to retrieve the
> > filehandle.
> >
> > Ditto for the case of "rdattr_error", which makes absolutely no sense
> > whatsoever in a GETATTR request.
> >
> And as I mentioned above, can a server choose to return mounted_on_fileid
> for only some FHs when it claims to support the attribute? (I think that
> is allowed and since I don't see a difference between mandatory and supported
> recommended attributes I think the same applies to FH.)
The fileid and mounted_on_fileid are both OPTIONAL attributes, and as
section 5.2 says, the client is required to be able to deal with the
case where the server does not return them. As stated earlier, there is
no such exception in 5.1.
> Btw, when a server chooses to not return an FH in the readdir reply although
> it was requested, the FreeBSD client "readdirplus" essentially falls back to
> a basic "readdir" for the file name (ie. it doesn't prime the name and attribute
> caches for it).
Implementations are only relevant here in as far as they limit the
protocol changes that RFC3530bis can make. Given that there are clients
out there that assume a strict interpretation of RFC3530, then it is too
late to have RFC3530bis relax that interpretation.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
Tom Haynes wrote:
> Please note that I have cc'ed the NFSv4 list over at the IETF.
>
> On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <[email protected]>
> wrote:
>
> > All,
> >
> > Sorry for this late posting, a coworker was made aware of this
> > thread
> > recently and
> > has these replies to make. Our server implementation is the one
> > being
> > complained about in this thread.
> >
> >
> > Quoted text is from various entries in this forum.
> >
> >
> > Neil Brown:
> > ===========
> >> Just a thought: while coping with broken servers would not be a
> >> good
> > path to
> >> follow, detecting protocol violations and reporting an error might
> >> be...
> >> should the NFS client treat a missing filehandle and a malformed
> >> reply?
> >
> > The server is allowed to remove an attribute bit from an entry in
> > the
> > readdir
> > reply, this is not "broken" nor is the reply "malformed". The lack
> > of a
> > filehandle in the reply is deterministic.
> >
> >
> >
> > Trond Myklebust:
> > ================
> >>> A customer has come across a server which does not return the
> > filehandle
> >>> information (is that allowed?).
> >>
> >> The filehandle attribute is a mandatory attribute according to
> >> RFC3530,
> > so I believe that the answer is "no".
> >
> > Mandatory is described in RFS 3530 as that the server must return
> > the
> > attribute
> > on a GETATTR. (Section 5, page 36). There is nothing saying that it
> > is
> > mandatory to return on a READDIR. Our server will return the
> > filehandle
> > on a LOOKUP/GETATTR every time.
>
>
>
> GETATTR and READDIR both return attributes. To be precise, they return
> a fattr4.
>
> Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> copy)
> of 3530bis (3530 is on the way to being replaced), READDIR states:
>
> The READDIR operation retrieves a variable number of entries from a
> filesystem directory and returns client requested attributes for each
> entry along with information to allow the client to request
> additional directory entries in a subsequent READDIR.
> ...
> On successful return, the server's response will provide a list of
> directory entries. Each of these entries contains the name of the
> directory entry, a cookie value for that entry, and the associated
> attributes as requested. The "eof" flag has a value of TRUE if there
> are no more entries in the directory.
>
> Any client implementation has no way to request that any server
> implementation
> return the filehandle because the filehandle is not an attribute which
> can be requested. I.e., it is mandatory.
>
When I read this, all I think "requested" refers to is whether or not
the client sets the bit "requesting" that attribute.
> If the intent was to allow the server to not return a filehandle for
> READDIR but to
> allow it to return one for GETATTR, then it would have been made
> OPTIONAL.
>
I don't know what the author's intent was, but since 5.1 only refers
to GETATTR, nothing in RFC3530 indicates that the same rule applies
to READDIR (ie. MANDATORY-->must always return it for READDIR).
> Whether or not the client used to work with such a server
> implementation
> or not is immaterial as far as standard compliance is concerned.
>
> If you like, we can clean up the corresponding language in 3530bis to
> explicitly state that REQUIRED attributes are indeed required whether
> in response to GETATTR or READDIR.
>
Personally, I think it should be "clarified" that for READDIR, the server
can choose to not reply any of the attributes. That allows the server implementor
more flexibility and since the old spec. didn't clearly require the REQUIRED
ones, I don't see why the new spec. should, unless it in impractical for clients
to allow this flexibility (and that is where the implementation evidence is
relevent, imho).
rick
>
>
>
>
>
> >
> >
> > Trond Myklebust:
> > ================
> >> My concern is that the client can't objectively judge what
> >> constitutes a
> >> valid filehandle and what does not until it tries to use it in an
> >> RPC
> >> call.
> >
> > Sure it can. In the context of this discussion, if the readdir entry
> > has the filehandle attribute bit off in the reply, there is no
> > filehandle.
> >
> > I would note in the scenario we sent a trace for, the Linux client
> > had
> > a filehandle for the node in question, and "misplaced" it after a
> > readdir to the directory containing that node did not return the
> > filehandle
> > for that entry. So calling the server "broken" is objectively
> > inaccurate.
> >
> > I would also note that this problem occurred after we upgraded to
> > RHEL
> > 6.3;
> > the prior version did not have this issue, and our server did not
> > change
> > its
> > behavior. My conclusion is that the means to obtain the filehandle
> > was
> > either broken by the client adding logic to assume a filehandle and
> > obliterate
> > the prior value after a readdir reply chose not to return it, or
> > previously
> > had a code path to lookup the node and obtain the filehandle, and
> > removed
> > that code path for some reason. So again, pointing to server
> > "breakage"
> > is objectively inaccurate.
> >
> > There are many references to "brokenness" or "server bugs" in this
> > thread, and I take exception to it from a design and protocol
> > conformance
> > point of view.
> >
> > This condition is easily recoverable, as ANY missing attribute on a
> > readdir
> > is with a lookup/readdir. Our decision to not return the FH on a
> > readdir
> > reply under certain narrow conditions is not one of convenience but
> > of
> > limitations of some underlying filesystem types, and if
> > "convenience" is
> > to be used as an accusative, it can easily be returned to the
> > client's
> > refusal to deal with the legal withholding of an attribute on a
> > readdir
> > reply.
> >
> >
> > Signed,
> > Steve Huntington
> > IBM z/OS NFS
> >
> >
> > Christopher Vogan
> > Dept. W98 NFS Development & Test
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> > in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> _______________________________________________
> nfsv4 mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/nfsv4
On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> Please note that I have cc'ed the NFSv4 list over at the IETF.
>
> On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <[email protected]> wrote:
>
> > All,
> >
> > Sorry for this late posting, a coworker was made aware of this thread
> > recently and
> > has these replies to make. Our server implementation is the one being
> > complained about in this thread.
> >
> >
> > Quoted text is from various entries in this forum.
> >
> >
> > Neil Brown:
> > ===========
> >> Just a thought: while coping with broken servers would not be a good
> > path to
> >> follow, detecting protocol violations and reporting an error might be...
> >> should the NFS client treat a missing filehandle and a malformed reply?
> >
> > The server is allowed to remove an attribute bit from an entry in the
> > readdir
> > reply, this is not "broken" nor is the reply "malformed". The lack of a
> > filehandle in the reply is deterministic.
> >
> >
> >
> > Trond Myklebust:
> > ================
> >>> A customer has come across a server which does not return the
> > filehandle
> >>> information (is that allowed?).
> >>
> >> The filehandle attribute is a mandatory attribute according to RFC3530,
> > so I believe that the answer is "no".
> >
> > Mandatory is described in RFS 3530 as that the server must return the
> > attribute
> > on a GETATTR. (Section 5, page 36). There is nothing saying that it is
> > mandatory to return on a READDIR. Our server will return the filehandle
> > on a LOOKUP/GETATTR every time.
>
>
>
> GETATTR and READDIR both return attributes. To be precise, they return
> a fattr4.
>
> Looking at Section 15.26.4 (roughly pages 277-279 of the most recent copy)
> of 3530bis (3530 is on the way to being replaced), READDIR states:
>
> The READDIR operation retrieves a variable number of entries from a
> filesystem directory and returns client requested attributes for each
> entry along with information to allow the client to request
> additional directory entries in a subsequent READDIR.
> ...
> On successful return, the server's response will provide a list of
> directory entries. Each of these entries contains the name of the
> directory entry, a cookie value for that entry, and the associated
> attributes as requested. The "eof" flag has a value of TRUE if there
> are no more entries in the directory.
>
> Any client implementation has no way to request that any server implementation
> return the filehandle because the filehandle is not an attribute which
> can be requested. I.e., it is mandatory.
>
> If the intent was to allow the server to not return a filehandle for READDIR but to
> allow it to return one for GETATTR, then it would have been made
> OPTIONAL.
>
> Whether or not the client used to work with such a server implementation
> or not is immaterial as far as standard compliance is concerned.
>
> If you like, we can clean up the corresponding language in 3530bis to
> explicitly state that REQUIRED attributes are indeed required whether
> in response to GETATTR or READDIR.
It might rather be useful to add language to point out that the
"filehandle" attribute SHOULD NOT be used for the GETATTR case. We have
GETFH, and AFAICR, all the language around referrals etc is written with
the assumption that clients _won't_ use GETATTR to retrieve the
filehandle.
Ditto for the case of "rdattr_error", which makes absolutely no sense
whatsoever in a GETATTR request.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
Please note that I have cc'ed the NFSv4 list over at the IETF.
On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <[email protected]> wrote:
> All,
>
> Sorry for this late posting, a coworker was made aware of this thread
> recently and
> has these replies to make. Our server implementation is the one being
> complained about in this thread.
>
>
> Quoted text is from various entries in this forum.
>
>
> Neil Brown:
> ===========
>> Just a thought: while coping with broken servers would not be a good
> path to
>> follow, detecting protocol violations and reporting an error might be...
>> should the NFS client treat a missing filehandle and a malformed reply?
>
> The server is allowed to remove an attribute bit from an entry in the
> readdir
> reply, this is not "broken" nor is the reply "malformed". The lack of a
> filehandle in the reply is deterministic.
>
>
>
> Trond Myklebust:
> ================
>>> A customer has come across a server which does not return the
> filehandle
>>> information (is that allowed?).
>>
>> The filehandle attribute is a mandatory attribute according to RFC3530,
> so I believe that the answer is "no".
>
> Mandatory is described in RFS 3530 as that the server must return the
> attribute
> on a GETATTR. (Section 5, page 36). There is nothing saying that it is
> mandatory to return on a READDIR. Our server will return the filehandle
> on a LOOKUP/GETATTR every time.
GETATTR and READDIR both return attributes. To be precise, they return
a fattr4.
Looking at Section 15.26.4 (roughly pages 277-279 of the most recent copy)
of 3530bis (3530 is on the way to being replaced), READDIR states:
The READDIR operation retrieves a variable number of entries from a
filesystem directory and returns client requested attributes for each
entry along with information to allow the client to request
additional directory entries in a subsequent READDIR.
...
On successful return, the server's response will provide a list of
directory entries. Each of these entries contains the name of the
directory entry, a cookie value for that entry, and the associated
attributes as requested. The "eof" flag has a value of TRUE if there
are no more entries in the directory.
Any client implementation has no way to request that any server implementation
return the filehandle because the filehandle is not an attribute which
can be requested. I.e., it is mandatory.
If the intent was to allow the server to not return a filehandle for READDIR but to
allow it to return one for GETATTR, then it would have been made
OPTIONAL.
Whether or not the client used to work with such a server implementation
or not is immaterial as far as standard compliance is concerned.
If you like, we can clean up the corresponding language in 3530bis to
explicitly state that REQUIRED attributes are indeed required whether
in response to GETATTR or READDIR.
>
>
> Trond Myklebust:
> ================
>> My concern is that the client can't objectively judge what constitutes a
>> valid filehandle and what does not until it tries to use it in an RPC
>> call.
>
> Sure it can. In the context of this discussion, if the readdir entry
> has the filehandle attribute bit off in the reply, there is no filehandle.
>
> I would note in the scenario we sent a trace for, the Linux client had
> a filehandle for the node in question, and "misplaced" it after a
> readdir to the directory containing that node did not return the
> filehandle
> for that entry. So calling the server "broken" is objectively inaccurate.
>
> I would also note that this problem occurred after we upgraded to RHEL
> 6.3;
> the prior version did not have this issue, and our server did not change
> its
> behavior. My conclusion is that the means to obtain the filehandle was
> either broken by the client adding logic to assume a filehandle and
> obliterate
> the prior value after a readdir reply chose not to return it, or
> previously
> had a code path to lookup the node and obtain the filehandle, and removed
> that code path for some reason. So again, pointing to server "breakage"
> is objectively inaccurate.
>
> There are many references to "brokenness" or "server bugs" in this
> thread, and I take exception to it from a design and protocol conformance
> point of view.
>
> This condition is easily recoverable, as ANY missing attribute on a
> readdir
> is with a lookup/readdir. Our decision to not return the FH on a readdir
> reply under certain narrow conditions is not one of convenience but of
> limitations of some underlying filesystem types, and if "convenience" is
> to be used as an accusative, it can easily be returned to the client's
> refusal to deal with the legal withholding of an attribute on a readdir
> reply.
>
>
> Signed,
> Steve Huntington
> IBM z/OS NFS
>
>
> Christopher Vogan
> Dept. W98 NFS Development & Test
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Mar 20, 2013 at 05:40:11PM -0500, Christopher T Vogan wrote:
> This condition is easily recoverable, as ANY missing attribute on a
> readdir
> is with a lookup/readdir.
By the same token, the server can just as well recover by doing that
extra lookup itself, can't it? (That's more-or-less how the Linux
server gets attributes on readdir.)
Understood that it can be a pain (the Linux server's readdir
implementation has needed some minor surgery over the years to get this
right), but I do think it's the server's responsibility....
--b.
> Our decision to not return the FH on a readdir
> reply under certain narrow conditions is not one of convenience but of
> limitations of some underlying filesystem types, and if "convenience" is
> to be used as an accusative, it can easily be returned to the client's
> refusal to deal with the legal withholding of an attribute on a readdir
> reply.
On Wed, Mar 20, 2013 at 09:41:40PM -0400, Rick Macklem wrote:
> Well, I don't see any difference between a mandatory attribute and a
> recommended attribute that the server claims to support via the
> supported_attributes attribute.
>
> I do believe that the server can choose not to return all of the
> mandatory and supported recommended attributes in the readdir reply.
> (If not, why have a bitmap of returned attributes?)
>
> One example here is the mounted_on_fileid, which some servers choose
> to only "support" for server mount points. (The FreeBSD server returns
> this attribute for all file handles, setting it to the same value as
> fileid for non-mount-points, but I am pretty sure some other servers
> do not return mounted_on_fileid for non-mount-points.)
That doesn't sound like traditional unix readdir behavior, and isn't the
behavior described by the spec; looking at 5661 5.8.2.23 (haven't
compared other rfcs):
"If the server detects that there is no mounted point at the
target file object, then the value for mounted_on_fileid that it
returns is the same as that of the fileid attribute."
Also, the supported_attrs attribute is a filesystem-wide attribute. I
don't think we generally allow attributes to vary between "supported"
and "unsupported" on a single filesystem.
--b.
On Thu, 2013-03-21 at 02:04 +0000, Myklebust, Trond wrote:
> Implementations are only relevant here in as far as they limit the
> protocol changes that RFC3530bis can make. Given that there are clients
> out there that assume a strict interpretation of RFC3530, then it is too
> late to have RFC3530bis relax that interpretation.
To put it differently: if you disagree, then please point to the text in
RFC3530, or RFC5661 (or RFC3530bis at this point) that supports your
position that Mandatory/REQUIRED attributes such as "filehandle" may be
ignored by a server in READDIR requests.
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
Adding back in the Ccs...
On Wed, 2013-03-20 at 17:40 -0500, Christopher T Vogan wrote:
> Trond Myklebust:
> ================
> > > A customer has come across a server which does not return the
> filehandle
> > > information (is that allowed?).
> >
> > The filehandle attribute is a mandatory attribute according to RFC3530,
> so I believe that the answer is "no".
>
> Mandatory is described in RFS 3530 as that the server must return the
> attribute
> on a GETATTR. (Section 5, page 36). There is nothing saying that it is
> mandatory to return on a READDIR. Our server will return the filehandle
> on a LOOKUP/GETATTR every time.
Section 5.5 lists the filehandle as being REQUIRED, and "primarily for
readdir requests". What's the point of listing an attribute as REQUIRED,
but not for the primary (read "only!") operation where it is useful?
There is an exception allowed: if the object is a referral, then you can
return NFS4ERR_MOVED in the rdattr_error (which is also a REQUIRED
attribute that only applies to READDIR) if the client requests it.
Otherwise, you return NFS4ERR_MOVED in the READDIR result...
IOW: I strongly disagree with your argument, and repeat that your server
is broken as far as the protocol goes...
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
All,
Sorry for this late posting, a coworker was made aware of this thread
recently and
has these replies to make. Our server implementation is the one being
complained about in this thread.
Quoted text is from various entries in this forum.
Neil Brown:
===========
> Just a thought: while coping with broken servers would not be a good
path to
> follow, detecting protocol violations and reporting an error might be...
> should the NFS client treat a missing filehandle and a malformed reply?
The server is allowed to remove an attribute bit from an entry in the
readdir
reply, this is not "broken" nor is the reply "malformed". The lack of a
filehandle in the reply is deterministic.
Trond Myklebust:
================
> > A customer has come across a server which does not return the
filehandle
> > information (is that allowed?).
>
> The filehandle attribute is a mandatory attribute according to RFC3530,
so I believe that the answer is "no".
Mandatory is described in RFS 3530 as that the server must return the
attribute
on a GETATTR. (Section 5, page 36). There is nothing saying that it is
mandatory to return on a READDIR. Our server will return the filehandle
on a LOOKUP/GETATTR every time.
Trond Myklebust:
================
> My concern is that the client can't objectively judge what constitutes a
> valid filehandle and what does not until it tries to use it in an RPC
> call.
Sure it can. In the context of this discussion, if the readdir entry
has the filehandle attribute bit off in the reply, there is no filehandle.
I would note in the scenario we sent a trace for, the Linux client had
a filehandle for the node in question, and "misplaced" it after a
readdir to the directory containing that node did not return the
filehandle
for that entry. So calling the server "broken" is objectively inaccurate.
I would also note that this problem occurred after we upgraded to RHEL
6.3;
the prior version did not have this issue, and our server did not change
its
behavior. My conclusion is that the means to obtain the filehandle was
either broken by the client adding logic to assume a filehandle and
obliterate
the prior value after a readdir reply chose not to return it, or
previously
had a code path to lookup the node and obtain the filehandle, and removed
that code path for some reason. So again, pointing to server "breakage"
is objectively inaccurate.
There are many references to "brokenness" or "server bugs" in this
thread, and I take exception to it from a design and protocol conformance
point of view.
This condition is easily recoverable, as ANY missing attribute on a
readdir
is with a lookup/readdir. Our decision to not return the FH on a readdir
reply under certain narrow conditions is not one of convenience but of
limitations of some underlying filesystem types, and if "convenience" is
to be used as an accusative, it can easily be returned to the client's
refusal to deal with the legal withholding of an attribute on a readdir
reply.
Signed,
Steve Huntington
IBM z/OS NFS
Christopher Vogan
Dept. W98 NFS Development & Test
Trond Myklebust wrote:
> On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> > Trond Myklebust wrote:
> > > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > > >
> > > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan
> > > > <[email protected]>
> > > > wrote:
> > > >
> > > > > All,
> > > > >
> > > > > Sorry for this late posting, a coworker was made aware of this
> > > > > thread
> > > > > recently and
> > > > > has these replies to make. Our server implementation is the
> > > > > one
> > > > > being
> > > > > complained about in this thread.
> > > > >
> > > > >
> > > > > Quoted text is from various entries in this forum.
> > > > >
> > > > >
> > > > > Neil Brown:
> > > > > ===========
> > > > >> Just a thought: while coping with broken servers would not be
> > > > >> a
> > > > >> good
> > > > > path to
> > > > >> follow, detecting protocol violations and reporting an error
> > > > >> might be...
> > > > >> should the NFS client treat a missing filehandle and a
> > > > >> malformed
> > > > >> reply?
> > > > >
> > > > > The server is allowed to remove an attribute bit from an entry
> > > > > in
> > > > > the
> > > > > readdir
> > > > > reply, this is not "broken" nor is the reply "malformed". The
> > > > > lack
> > > > > of a
> > > > > filehandle in the reply is deterministic.
> > > > >
> > > > >
> > > > >
> > > > > Trond Myklebust:
> > > > > ================
> > > > >>> A customer has come across a server which does not return
> > > > >>> the
> > > > > filehandle
> > > > >>> information (is that allowed?).
> > > > >>
> > > > >> The filehandle attribute is a mandatory attribute according
> > > > >> to
> > > > >> RFC3530,
> > > > > so I believe that the answer is "no".
> > > > >
> > > > > Mandatory is described in RFS 3530 as that the server must
> > > > > return
> > > > > the
> > > > > attribute
> > > > > on a GETATTR. (Section 5, page 36). There is nothing saying
> > > > > that
> > > > > it is
> > > > > mandatory to return on a READDIR. Our server will return the
> > > > > filehandle
> > > > > on a LOOKUP/GETATTR every time.
> > > >
> > > >
> > > >
> > > > GETATTR and READDIR both return attributes. To be precise, they
> > > > return
> > > > a fattr4.
> > > >
> > > > Looking at Section 15.26.4 (roughly pages 277-279 of the most
> > > > recent
> > > > copy)
> > > > of 3530bis (3530 is on the way to being replaced), READDIR
> > > > states:
> > > >
> > > > The READDIR operation retrieves a variable number of entries
> > > > from
> > > > a
> > > > filesystem directory and returns client requested attributes
> > > > for
> > > > each
> > > > entry along with information to allow the client to request
> > > > additional directory entries in a subsequent READDIR.
> > > > ...
> > > > On successful return, the server's response will provide a
> > > > list
> > > > of
> > > > directory entries. Each of these entries contains the name of
> > > > the
> > > > directory entry, a cookie value for that entry, and the
> > > > associated
> > > > attributes as requested. The "eof" flag has a value of TRUE
> > > > if
> > > > there
> > > > are no more entries in the directory.
> > > >
> > > > Any client implementation has no way to request that any server
> > > > implementation
> > > > return the filehandle because the filehandle is not an attribute
> > > > which
> > > > can be requested. I.e., it is mandatory.
> > > >
> > > > If the intent was to allow the server to not return a filehandle
> > > > for
> > > > READDIR but to
> > > > allow it to return one for GETATTR, then it would have been made
> > > > OPTIONAL.
> > > >
> > Well, I don't see any difference between a mandatory attribute and a
> > recommended attribute that the server claims to support via the
> > supported_attributes attribute.
> >
> > I do believe that the server can choose not to return all of the
> > mandatory and supported recommended attributes in the readdir reply.
> > (If not, why have a bitmap of returned attributes?)
>
> That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
> states:
>
> "A client may ask for any of these attributes to be returned by
> setting
> a bit in the GETATTR request but must handle the case where the server
> does not return them."
>
> Section 5.1 says:
>
> "These MUST be supported by every NFS version 4 client and server in
> order to ensure a minimum level of interoperability." There are no
> exceptions made for READDIR.
>
Yes, I just read them. I had forgotten (never realized) that for GETATTR
there is the rule that mandatory/required attributes must be returned.
You are also correct that there is no exception for READDIR. In fact,
I don't see any mention of READDIR in Sec. 5.1, 5.2. Both Sec. 5.1 and 5.2
refer specifically to GETATTR when stating whether they must be returned.
Then, when I read the description for READDIR, it seems to say that
the server is to set the rderror attribute if it cannot return all the
requested attributes (no exception for recommended/optional ones) or
fail the entire READDIR if the client hasn't asked for rderror.
Since this isn't what actually happens for mounted_on_fileid, I still
think the same should apply to other attributes requested by the
READDIR request and I don't see that the 5.1, 5.2 rule for GETATTR
applies to READDIR (ie. FH must be returned, but mounted_on_fileid
doesn't need to be).
> > One example here is the mounted_on_fileid, which some servers choose
> > to only "support" for server mount points. (The FreeBSD server
> > returns
> > this attribute for all file handles, setting it to the same value as
> > fileid for non-mount-points, but I am pretty sure some other servers
> > do not return mounted_on_fileid for non-mount-points.)
> >
> > > > Whether or not the client used to work with such a server
> > > > implementation
> > > > or not is immaterial as far as standard compliance is concerned.
> > > >
> > > > If you like, we can clean up the corresponding language in
> > > > 3530bis
> > > > to
> > > > explicitly state that REQUIRED attributes are indeed required
> > > > whether
> > > > in response to GETATTR or READDIR.
> > I assume that by REQUIRED you are referring to "mandatory
> > attributes".
>
> I'm using the language from RFC5661 and RFC3530-bis, since that is the
> most up-to-date.
>
I could nitpick and note that rfc3530bis is simply a draft at this point
in time and that RFC3530 is at this time the specification for NFSv4.0.
(The truth is I just haven't bothered reading rfc3530bis. Once it becomes
an RFC and I get bit by changes, like the "uid # in the owner string"
change, then I guess I'll have to.;-)
Btw, Sec. 5.1 also states that a client should be able to handle a
server that only supports the mandatory/required attributes.
Have you tested against a server that doesn't support the "fileid"
attribute yet? (I can guarantee that the FreeBSD client will
never work against such a server, so I don't have to worry about
trying to be fully conformant.;-)
rick
> > > It might rather be useful to add language to point out that the
> > > "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> > > have
> > > GETFH, and AFAICR, all the language around referrals etc is
> > > written
> > > with
> > > the assumption that clients _won't_ use GETATTR to retrieve the
> > > filehandle.
> > >
> > > Ditto for the case of "rdattr_error", which makes absolutely no
> > > sense
> > > whatsoever in a GETATTR request.
> > >
> > And as I mentioned above, can a server choose to return
> > mounted_on_fileid
> > for only some FHs when it claims to support the attribute? (I think
> > that
> > is allowed and since I don't see a difference between mandatory and
> > supported
> > recommended attributes I think the same applies to FH.)
>
> The fileid and mounted_on_fileid are both OPTIONAL attributes, and as
> section 5.2 says, the client is required to be able to deal with the
> case where the server does not return them. As stated earlier, there
> is
> no such exception in 5.1.
>
> > Btw, when a server chooses to not return an FH in the readdir reply
> > although
> > it was requested, the FreeBSD client "readdirplus" essentially falls
> > back to
> > a basic "readdir" for the file name (ie. it doesn't prime the name
> > and attribute
> > caches for it).
>
> Implementations are only relevant here in as far as they limit the
> protocol changes that RFC3530bis can make. Given that there are
> clients
> out there that assume a strict interpretation of RFC3530, then it is
> too
> late to have RFC3530bis relax that interpretation.
>
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> [email protected]
> http://www.netapp.com
> _______________________________________________
> nfsv4 mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/nfsv4
Trond Myklebust wrote:
> On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > Please note that I have cc'ed the NFSv4 list over at the IETF.
> >
> > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <[email protected]>
> > wrote:
> >
> > > All,
> > >
> > > Sorry for this late posting, a coworker was made aware of this
> > > thread
> > > recently and
> > > has these replies to make. Our server implementation is the one
> > > being
> > > complained about in this thread.
> > >
> > >
> > > Quoted text is from various entries in this forum.
> > >
> > >
> > > Neil Brown:
> > > ===========
> > >> Just a thought: while coping with broken servers would not be a
> > >> good
> > > path to
> > >> follow, detecting protocol violations and reporting an error
> > >> might be...
> > >> should the NFS client treat a missing filehandle and a malformed
> > >> reply?
> > >
> > > The server is allowed to remove an attribute bit from an entry in
> > > the
> > > readdir
> > > reply, this is not "broken" nor is the reply "malformed". The lack
> > > of a
> > > filehandle in the reply is deterministic.
> > >
> > >
> > >
> > > Trond Myklebust:
> > > ================
> > >>> A customer has come across a server which does not return the
> > > filehandle
> > >>> information (is that allowed?).
> > >>
> > >> The filehandle attribute is a mandatory attribute according to
> > >> RFC3530,
> > > so I believe that the answer is "no".
> > >
> > > Mandatory is described in RFS 3530 as that the server must return
> > > the
> > > attribute
> > > on a GETATTR. (Section 5, page 36). There is nothing saying that
> > > it is
> > > mandatory to return on a READDIR. Our server will return the
> > > filehandle
> > > on a LOOKUP/GETATTR every time.
> >
> >
> >
> > GETATTR and READDIR both return attributes. To be precise, they
> > return
> > a fattr4.
> >
> > Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> > copy)
> > of 3530bis (3530 is on the way to being replaced), READDIR states:
> >
> > The READDIR operation retrieves a variable number of entries from
> > a
> > filesystem directory and returns client requested attributes for
> > each
> > entry along with information to allow the client to request
> > additional directory entries in a subsequent READDIR.
> > ...
> > On successful return, the server's response will provide a list
> > of
> > directory entries. Each of these entries contains the name of the
> > directory entry, a cookie value for that entry, and the
> > associated
> > attributes as requested. The "eof" flag has a value of TRUE if
> > there
> > are no more entries in the directory.
> >
> > Any client implementation has no way to request that any server
> > implementation
> > return the filehandle because the filehandle is not an attribute
> > which
> > can be requested. I.e., it is mandatory.
> >
> > If the intent was to allow the server to not return a filehandle for
> > READDIR but to
> > allow it to return one for GETATTR, then it would have been made
> > OPTIONAL.
> >
Well, I don't see any difference between a mandatory attribute and a
recommended attribute that the server claims to support via the
supported_attributes attribute.
I do believe that the server can choose not to return all of the
mandatory and supported recommended attributes in the readdir reply.
(If not, why have a bitmap of returned attributes?)
One example here is the mounted_on_fileid, which some servers choose
to only "support" for server mount points. (The FreeBSD server returns
this attribute for all file handles, setting it to the same value as
fileid for non-mount-points, but I am pretty sure some other servers
do not return mounted_on_fileid for non-mount-points.)
> > Whether or not the client used to work with such a server
> > implementation
> > or not is immaterial as far as standard compliance is concerned.
> >
> > If you like, we can clean up the corresponding language in 3530bis
> > to
> > explicitly state that REQUIRED attributes are indeed required
> > whether
> > in response to GETATTR or READDIR.
I assume that by REQUIRED you are referring to "mandatory attributes".
>
> It might rather be useful to add language to point out that the
> "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> have
> GETFH, and AFAICR, all the language around referrals etc is written
> with
> the assumption that clients _won't_ use GETATTR to retrieve the
> filehandle.
>
> Ditto for the case of "rdattr_error", which makes absolutely no sense
> whatsoever in a GETATTR request.
>
And as I mentioned above, can a server choose to return mounted_on_fileid
for only some FHs when it claims to support the attribute? (I think that
is allowed and since I don't see a difference between mandatory and supported
recommended attributes I think the same applies to FH.)
Btw, when a server chooses to not return an FH in the readdir reply although
it was requested, the FreeBSD client "readdirplus" essentially falls back to
a basic "readdir" for the file name (ie. it doesn't prime the name and attribute
caches for it).
rick
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> [email protected]
> http://www.netapp.com
> _______________________________________________
> nfsv4 mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/nfsv4
On Wed, 2013-03-20 at 23:38 -0400, Rick Macklem wrote:
> Trond Myklebust wrote:
> > On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> > > Trond Myklebust wrote:
> > > > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > > > >
> > > > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan
> > > > > <[email protected]>
> > > > > wrote:
> > > > >
> > > > > > All,
> > > > > >
> > > > > > Sorry for this late posting, a coworker was made aware of this
> > > > > > thread
> > > > > > recently and
> > > > > > has these replies to make. Our server implementation is the
> > > > > > one
> > > > > > being
> > > > > > complained about in this thread.
> > > > > >
> > > > > >
> > > > > > Quoted text is from various entries in this forum.
> > > > > >
> > > > > >
> > > > > > Neil Brown:
> > > > > > ===========
> > > > > >> Just a thought: while coping with broken servers would not be
> > > > > >> a
> > > > > >> good
> > > > > > path to
> > > > > >> follow, detecting protocol violations and reporting an error
> > > > > >> might be...
> > > > > >> should the NFS client treat a missing filehandle and a
> > > > > >> malformed
> > > > > >> reply?
> > > > > >
> > > > > > The server is allowed to remove an attribute bit from an entry
> > > > > > in
> > > > > > the
> > > > > > readdir
> > > > > > reply, this is not "broken" nor is the reply "malformed". The
> > > > > > lack
> > > > > > of a
> > > > > > filehandle in the reply is deterministic.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Trond Myklebust:
> > > > > > ================
> > > > > >>> A customer has come across a server which does not return
> > > > > >>> the
> > > > > > filehandle
> > > > > >>> information (is that allowed?).
> > > > > >>
> > > > > >> The filehandle attribute is a mandatory attribute according
> > > > > >> to
> > > > > >> RFC3530,
> > > > > > so I believe that the answer is "no".
> > > > > >
> > > > > > Mandatory is described in RFS 3530 as that the server must
> > > > > > return
> > > > > > the
> > > > > > attribute
> > > > > > on a GETATTR. (Section 5, page 36). There is nothing saying
> > > > > > that
> > > > > > it is
> > > > > > mandatory to return on a READDIR. Our server will return the
> > > > > > filehandle
> > > > > > on a LOOKUP/GETATTR every time.
> > > > >
> > > > >
> > > > >
> > > > > GETATTR and READDIR both return attributes. To be precise, they
> > > > > return
> > > > > a fattr4.
> > > > >
> > > > > Looking at Section 15.26.4 (roughly pages 277-279 of the most
> > > > > recent
> > > > > copy)
> > > > > of 3530bis (3530 is on the way to being replaced), READDIR
> > > > > states:
> > > > >
> > > > > The READDIR operation retrieves a variable number of entries
> > > > > from
> > > > > a
> > > > > filesystem directory and returns client requested attributes
> > > > > for
> > > > > each
> > > > > entry along with information to allow the client to request
> > > > > additional directory entries in a subsequent READDIR.
> > > > > ...
> > > > > On successful return, the server's response will provide a
> > > > > list
> > > > > of
> > > > > directory entries. Each of these entries contains the name of
> > > > > the
> > > > > directory entry, a cookie value for that entry, and the
> > > > > associated
> > > > > attributes as requested. The "eof" flag has a value of TRUE
> > > > > if
> > > > > there
> > > > > are no more entries in the directory.
> > > > >
> > > > > Any client implementation has no way to request that any server
> > > > > implementation
> > > > > return the filehandle because the filehandle is not an attribute
> > > > > which
> > > > > can be requested. I.e., it is mandatory.
> > > > >
> > > > > If the intent was to allow the server to not return a filehandle
> > > > > for
> > > > > READDIR but to
> > > > > allow it to return one for GETATTR, then it would have been made
> > > > > OPTIONAL.
> > > > >
> > > Well, I don't see any difference between a mandatory attribute and a
> > > recommended attribute that the server claims to support via the
> > > supported_attributes attribute.
> > >
> > > I do believe that the server can choose not to return all of the
> > > mandatory and supported recommended attributes in the readdir reply.
> > > (If not, why have a bitmap of returned attributes?)
> >
> > That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
> > states:
> >
> > "A client may ask for any of these attributes to be returned by
> > setting
> > a bit in the GETATTR request but must handle the case where the server
> > does not return them."
> >
> > Section 5.1 says:
> >
> > "These MUST be supported by every NFS version 4 client and server in
> > order to ensure a minimum level of interoperability." There are no
> > exceptions made for READDIR.
> >
> Yes, I just read them. I had forgotten (never realized) that for GETATTR
> there is the rule that mandatory/required attributes must be returned.
>
> You are also correct that there is no exception for READDIR. In fact,
> I don't see any mention of READDIR in Sec. 5.1, 5.2. Both Sec. 5.1 and 5.2
> refer specifically to GETATTR when stating whether they must be returned.
No, but section 5.5 states very specifically that the "filehandle"
attribute is "primarily for readdir requests". What is the meaning of a
"Mandatory/Required" attribute if it doesn't apply to the primary (read
"only useful") case?
Ditto question for rdattr_error, which (as indicated earlier) _only_
applies to READDIR.
> Then, when I read the description for READDIR, it seems to say that
> the server is to set the rderror attribute if it cannot return all the
> requested attributes (no exception for recommended/optional ones) or
> fail the entire READDIR if the client hasn't asked for rderror.
I accept that.
> Since this isn't what actually happens for mounted_on_fileid, I still
> think the same should apply to other attributes requested by the
> READDIR request and I don't see that the 5.1, 5.2 rule for GETATTR
> applies to READDIR (ie. FH must be returned, but mounted_on_fileid
> doesn't need to be).
Where do you see support for this in the spec?
> > > One example here is the mounted_on_fileid, which some servers choose
> > > to only "support" for server mount points. (The FreeBSD server
> > > returns
> > > this attribute for all file handles, setting it to the same value as
> > > fileid for non-mount-points, but I am pretty sure some other servers
> > > do not return mounted_on_fileid for non-mount-points.)
> > >
> > > > > Whether or not the client used to work with such a server
> > > > > implementation
> > > > > or not is immaterial as far as standard compliance is concerned.
> > > > >
> > > > > If you like, we can clean up the corresponding language in
> > > > > 3530bis
> > > > > to
> > > > > explicitly state that REQUIRED attributes are indeed required
> > > > > whether
> > > > > in response to GETATTR or READDIR.
> > > I assume that by REQUIRED you are referring to "mandatory
> > > attributes".
> >
> > I'm using the language from RFC5661 and RFC3530-bis, since that is the
> > most up-to-date.
> >
> I could nitpick and note that rfc3530bis is simply a draft at this point
> in time and that RFC3530 is at this time the specification for NFSv4.0.
> (The truth is I just haven't bothered reading rfc3530bis. Once it becomes
> an RFC and I get bit by changes, like the "uid # in the owner string"
> change, then I guess I'll have to.;-)
>
> Btw, Sec. 5.1 also states that a client should be able to handle a
> server that only supports the mandatory/required attributes.
> Have you tested against a server that doesn't support the "fileid"
> attribute yet? (I can guarantee that the FreeBSD client will
> never work against such a server, so I don't have to worry about
> trying to be fully conformant.;-)
I'm not certain that we've tested against a server that only supports
the mandatory attributes, but I do see some protocol problems there. Not
least, there is a known problem with determining the current value for
the change attribute; hence my proposal of the "change_attr_type"
OPTIONAL attribute for NFSv4.2. Unless someone has a valid objection,
I'm planning on proposing that attribute for REQUIRED status in
NFSv4.3...
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com