Hi folks,
I’m seeing a client behavior that I can’t explain the reason behind
(i.e. spec) so want to ask for help.
Question #1: Should a reclaim of open state handle BAD_SEQID the same
way the normal open handles BAD_SEQID (which is: it just retries). I
can see that during recovery we might not want the recovery to be
trying forever and that’s why we fail? I don’t see anything in the
spec that talk about this…
Question #2: Incrementing seqid of open owner. I see that when LOCK
receives an error BAD_STATEID/ADMIN_REVOKED, sequence isn’t
incremented and (recovery) OPEN is sent with the same seqid. The code
explicitly sets seqid unconfirmed. Again, I don’t know where to look
in the spec for something like this. What happens is that server
replies back with BAD_SEQID.
So normally, when an open fails with BAD_SEQID the client retries but
combine it with a LOCK that failed with BAD_STATEID. Then it leads to
the application failing with “bad file descriptor” because after
receiving an error for the lock and trying to do open recovery, the
code doesn’t increment seqid and open fails with BAD_SEQID and we
don’t retry the open. So now we don’t have a file descriptor and we
fail the lock.
Thank you.
DQoNCk9uIDUvMjAvMTYsIDE2OjM4LCAibGludXgtbmZzLW93bmVyQHZnZXIua2VybmVsLm9yZyBv
biBiZWhhbGYgb2YgT2xnYSBLb3JuaWV2c2thaWEiIDxsaW51eC1uZnMtb3duZXJAdmdlci5rZXJu
ZWwub3JnIG9uIGJlaGFsZiBvZiBhZ2xvQHVtaWNoLmVkdT4gd3JvdGU6DQoNCj5IaSBmb2xrcywN
Cj4NCj5J4oCZbSBzZWVpbmcgYSBjbGllbnQgYmVoYXZpb3IgdGhhdCBJIGNhbuKAmXQgZXhwbGFp
biB0aGUgcmVhc29uIGJlaGluZA0KPihpLmUuIHNwZWMpIHNvIHdhbnQgdG8gYXNrIGZvciBoZWxw
Lg0KPg0KPlF1ZXN0aW9uICMxOiBTaG91bGQgYSByZWNsYWltIG9mIG9wZW4gc3RhdGUgaGFuZGxl
IEJBRF9TRVFJRCB0aGUgc2FtZQ0KPndheSB0aGUgbm9ybWFsIG9wZW4gaGFuZGxlcyBCQURfU0VR
SUQgKHdoaWNoIGlzOiBpdCBqdXN0IHJldHJpZXMpLiBJDQo+Y2FuIHNlZSB0aGF0IGR1cmluZyBy
ZWNvdmVyeSB3ZSBtaWdodCBub3Qgd2FudCB0aGUgcmVjb3ZlcnkgdG8gYmUNCj50cnlpbmcgZm9y
ZXZlciBhbmQgdGhhdOKAmXMgd2h5IHdlIGZhaWw/IEkgZG9u4oCZdCBzZWUgYW55dGhpbmcgaW4g
dGhlDQo+c3BlYyB0aGF0IHRhbGsgYWJvdXQgdGhpc+KApg0KPg0KPlF1ZXN0aW9uICMyOiBJbmNy
ZW1lbnRpbmcgc2VxaWQgb2Ygb3BlbiBvd25lci4gSSBzZWUgdGhhdCB3aGVuIExPQ0sNCj5yZWNl
aXZlcyBhbiBlcnJvciBCQURfU1RBVEVJRC9BRE1JTl9SRVZPS0VELCBzZXF1ZW5jZSBpc27igJl0
DQo+aW5jcmVtZW50ZWQgYW5kIChyZWNvdmVyeSkgT1BFTiBpcyBzZW50IHdpdGggdGhlIHNhbWUg
c2VxaWQuIFRoZSBjb2RlDQo+ZXhwbGljaXRseSBzZXRzIHNlcWlkIHVuY29uZmlybWVkLiBBZ2Fp
biwgSSBkb27igJl0IGtub3cgd2hlcmUgdG8gbG9vaw0KPmluIHRoZSBzcGVjIGZvciBzb21ldGhp
bmcgbGlrZSB0aGlzLiBXaGF0IGhhcHBlbnMgaXMgdGhhdCBzZXJ2ZXINCj5yZXBsaWVzIGJhY2sg
d2l0aCBCQURfU0VRSUQuDQo+DQo+U28gbm9ybWFsbHksIHdoZW4gYW4gb3BlbiBmYWlscyB3aXRo
IEJBRF9TRVFJRCB0aGUgY2xpZW50IHJldHJpZXMgYnV0DQo+Y29tYmluZSBpdCB3aXRoIGEgTE9D
SyB0aGF0IGZhaWxlZCB3aXRoIEJBRF9TVEFURUlELiBUaGVuIGl0IGxlYWRzIHRvDQo+dGhlIGFw
cGxpY2F0aW9uIGZhaWxpbmcgd2l0aCDigJxiYWQgZmlsZSBkZXNjcmlwdG9y4oCdIGJlY2F1c2Ug
YWZ0ZXINCj5yZWNlaXZpbmcgYW4gZXJyb3IgZm9yIHRoZSBsb2NrIGFuZCB0cnlpbmcgdG8gZG8g
b3BlbiByZWNvdmVyeSwgdGhlDQo+Y29kZSBkb2VzbuKAmXQgaW5jcmVtZW50IHNlcWlkIGFuZCBv
cGVuIGZhaWxzIHdpdGggQkFEX1NFUUlEIGFuZCB3ZQ0KPmRvbuKAmXQgcmV0cnkgdGhlIG9wZW4u
IFNvIG5vdyB3ZSBkb27igJl0IGhhdmUgYSBmaWxlIGRlc2NyaXB0b3IgYW5kIHdlDQo+ZmFpbCB0
aGUgbG9jay4NCj4NCg0KU2VlOg0KaHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL3JmYzc1MzAj
c2VjdGlvbi05LjEuNw0KDQogICBUaGUgY2xpZW50IE1VU1QgYWR2YW5jZSB0aGUgc2VxdWVuY2Ug
bnVtYmVyIGZvciB0aGUgQ0xPU0UsIExPQ0ssDQogICBMT0NLVSwgT1BFTiwgT1BFTl9DT05GSVJN
LCBhbmQgT1BFTl9ET1dOR1JBREUgb3BlcmF0aW9ucy4gIFRoaXMgaXMNCiAgIHRydWUgZXZlbiBp
biB0aGUgZXZlbnQgdGhhdCB0aGUgcHJldmlvdXMgb3BlcmF0aW9uIHRoYXQgdXNlZCB0aGUNCiAg
IHNlcXVlbmNlIG51bWJlciByZWNlaXZlZCBhbiBlcnJvci4gIFRoZSBvbmx5IGV4Y2VwdGlvbiB0
byB0aGlzIHJ1bGUNCiAgIGlzIGlmIHRoZSBwcmV2aW91cyBvcGVyYXRpb24gcmVjZWl2ZWQgb25l
IG9mIHRoZSBmb2xsb3dpbmcgZXJyb3JzOg0KICAgTkZTNEVSUl9TVEFMRV9DTElFTlRJRCwgTkZT
NEVSUl9TVEFMRV9TVEFURUlELCBORlM0RVJSX0JBRF9TVEFURUlELA0KICAgTkZTNEVSUl9CQURf
U0VRSUQsIE5GUzRFUlJfQkFEWERSLCBORlM0RVJSX1JFU09VUkNFLA0KICAgTkZTNEVSUl9OT0ZJ
TEVIQU5ETEUsIG9yIE5GUzRFUlJfTU9WRUQuDQoNCkNoZWVycw0KICBUcm9uZA0KDQo=
On Fri, May 20, 2016 at 4:43 PM, Trond Myklebust
<[email protected]> wrote:
>
>
> On 5/20/16, 16:38, "[email protected] on behalf of Olga
> Kornievskaia" <[email protected] on behalf of [email protected]>
> wrote:
>
>>Hi folks,
>>
>>I’m seeing a client behavior that I can’t explain the reason behind
>>(i.e. spec) so want to ask for help.
>>
>>Question #1: Should a reclaim of open state handle BAD_SEQID the same
>>way the normal open handles BAD_SEQID (which is: it just retries). I
>>can see that during recovery we might not want the recovery to be
>>trying forever and that’s why we fail? I don’t see anything in the
>>spec that talk about this…
>>
>>Question #2: Incrementing seqid of open owner. I see that when LOCK
>>receives an error BAD_STATEID/ADMIN_REVOKED, sequence isn’t
>>incremented and (recovery) OPEN is sent with the same seqid. The code
>>explicitly sets seqid unconfirmed. Again, I don’t know where to look
>>in the spec for something like this. What happens is that server
>>replies back with BAD_SEQID.
>>
>>So normally, when an open fails with BAD_SEQID the client retries but
>>combine it with a LOCK that failed with BAD_STATEID. Then it leads to
>>the application failing with “bad file descriptor” because after
>>receiving an error for the lock and trying to do open recovery, the
>>code doesn’t increment seqid and open fails with BAD_SEQID and we
>>don’t retry the open. So now we don’t have a file descriptor and we
>>fail the lock.
>>
>
> See:
> https://tools.ietf.org/html/rfc7530#section-9.1.7
>
> The client MUST advance the sequence number for the CLOSE, LOCK,
> LOCKU, OPEN, OPEN_CONFIRM, and OPEN_DOWNGRADE operations. This is
> true even in the event that the previous operation that used the
> sequence number received an error. The only exception to this rule
> is if the previous operation received one of the following errors:
> NFS4ERR_STALE_CLIENTID, NFS4ERR_STALE_STATEID, NFS4ERR_BAD_STATEID,
> NFS4ERR_BAD_SEQID, NFS4ERR_BADXDR, NFS4ERR_RESOURCE,
> NFS4ERR_NOFILEHANDLE, or NFS4ERR_MOVED.
>
Thanks Trond.
> Cheers
> Trond
>
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
DQo+IE9uIE1heSAyMCwgMjAxNiwgYXQgMTozOCBQTSwgT2xnYSBLb3JuaWV2c2thaWEgPGFnbG9A
dW1pY2guZWR1PiB3cm90ZToNCj4gDQo+IEhpIGZvbGtzLA0KPiANCj4gSeKAmW0gc2VlaW5nIGEg
Y2xpZW50IGJlaGF2aW9yIHRoYXQgSSBjYW7igJl0IGV4cGxhaW4gdGhlIHJlYXNvbiBiZWhpbmQN
Cj4gKGkuZS4gc3BlYykgc28gd2FudCB0byBhc2sgZm9yIGhlbHAuDQo+IA0KPiBRdWVzdGlvbiAj
MTogU2hvdWxkIGEgcmVjbGFpbSBvZiBvcGVuIHN0YXRlIGhhbmRsZSBCQURfU0VRSUQgdGhlIHNh
bWUNCj4gd2F5IHRoZSBub3JtYWwgb3BlbiBoYW5kbGVzIEJBRF9TRVFJRCAod2hpY2ggaXM6IGl0
IGp1c3QgcmV0cmllcykuIEkNCj4gY2FuIHNlZSB0aGF0IGR1cmluZyByZWNvdmVyeSB3ZSBtaWdo
dCBub3Qgd2FudCB0aGUgcmVjb3ZlcnkgdG8gYmUNCj4gdHJ5aW5nIGZvcmV2ZXIgYW5kIHRoYXTi
gJlzIHdoeSB3ZSBmYWlsPyBJIGRvbuKAmXQgc2VlIGFueXRoaW5nIGluIHRoZQ0KPiBzcGVjIHRo
YXQgdGFsayBhYm91dCB0aGlz4oCmDQo+IA0KPiBRdWVzdGlvbiAjMjogSW5jcmVtZW50aW5nIHNl
cWlkIG9mIG9wZW4gb3duZXIuIEkgc2VlIHRoYXQgd2hlbiBMT0NLDQo+IHJlY2VpdmVzIGFuIGVy
cm9yIEJBRF9TVEFURUlEL0FETUlOX1JFVk9LRUQsIHNlcXVlbmNlIGlzbuKAmXQNCj4gaW5jcmVt
ZW50ZWQgYW5kIChyZWNvdmVyeSkgT1BFTiBpcyBzZW50IHdpdGggdGhlIHNhbWUgc2VxaWQuIFRo
ZSBjb2RlDQo+IGV4cGxpY2l0bHkgc2V0cyBzZXFpZCB1bmNvbmZpcm1lZC4gQWdhaW4sIEkgZG9u
4oCZdCBrbm93IHdoZXJlIHRvIGxvb2sNCj4gaW4gdGhlIHNwZWMgZm9yIHNvbWV0aGluZyBsaWtl
IHRoaXMuIFdoYXQgaGFwcGVucyBpcyB0aGF0IHNlcnZlcg0KPiByZXBsaWVzIGJhY2sgd2l0aCBC
QURfU0VRSUQuDQoNClJGQyA1NjYxDQoNCkl0IGRvZXNu4oCZdCBzYXkgaXQgZXhwbGljaXRseSwg
YnV0IHlvdSBvbmx5IGJ1bXAgdGhlIHNlcWlkIG9uIHN1Y2Nlc3MuDQoNCjMuMy4xMi4gIHN0YXRl
aWQ0DQoNCiAgVGhlIHN0YXJ0aW5nIHZhbHVlIG9mIHRoZSAic2VxaWQiIGZpZWxkIGlzDQogICB1
bmRlZmluZWQuICBUaGUgc2VydmVyIGlzIHJlcXVpcmVkIHRvIGluY3JlbWVudCB0aGUgInNlcWlk
IiBmaWVsZCBieQ0KICAgb25lIGF0IGVhY2ggdHJhbnNpdGlvbiBvZiB0aGUgc3RhdGVpZC4gDQoN
Cg0KOC4yLjEuICBTdGF0ZWlkIFR5cGVzDQoNCiAgIG8gIFN0YXRlaWRzIG1heSByZXByZXNlbnQg
c2V0cyBvZiBieXRlLXJhbmdlIGxvY2tzLg0KDQogICAgICBBbGwgbG9ja3MgaGVsZCBvbiBhIHBh
cnRpY3VsYXIgZmlsZSBieSBhIHBhcnRpY3VsYXIgb3duZXIgYW5kDQogICAgICBnb3R0ZW4gdW5k
ZXIgdGhlIGFlZ2lzIG9mIGEgcGFydGljdWxhciBvcGVuIGZpbGUgYXJlIGFzc29jaWF0ZWQNCiAg
ICAgIHdpdGggYSBzaW5nbGUgc3RhdGVpZCB3aXRoIHRoZSBzZXFpZCBiZWluZyBpbmNyZW1lbnRl
ZCB3aGVuZXZlcg0KICAgICAgTE9DSyBhbmQgTE9DS1Ugb3BlcmF0aW9ucyBhZmZlY3QgdGhhdCBz
ZXQgb2YgbG9ja3MuDQoNCjguMi4yLiAgU3RhdGVpZCBTdHJ1Y3R1cmUNCg0KICAgV2hlbiBzdWNo
IGEgc2V0IG9mIGxvY2tzIGlzIGZpcnN0IGNyZWF0ZWQsIHRoZSBzZXJ2ZXIgcmV0dXJucyBhDQog
ICBzdGF0ZWlkIHdpdGggc2VxaWQgdmFsdWUgb2Ygb25lLiAgT24gc3Vic2VxdWVudCBvcGVyYXRp
b25zIHRoYXQNCiAgIG1vZGlmeSB0aGUgc2V0IG9mIGxvY2tzLCB0aGUgc2VydmVyIGlzIHJlcXVp
cmVkIHRvIGluY3JlbWVudCB0aGUNCiAgICJzZXFpZCIgZmllbGQgYnkgb25lIHdoZW5ldmVyIGl0
IHJldHVybnMgYSBzdGF0ZWlkIGZvciB0aGUgc2FtZQ0KICAgc3RhdGUtb3duZXIvZmlsZS90eXBl
IGNvbWJpbmF0aW9uIGFuZCB0aGVyZSBpcyBzb21lIGNoYW5nZSBpbiB0aGUgc2V0DQogICBvZiBs
b2NrcyBhY3R1YWxseSBkZXNpZ25hdGVkLiAgSW4gdGhpcyBjYXNlLCB0aGUgc2VydmVyIHdpbGwg
cmV0dXJuIGENCiAgIHN0YXRlaWQgd2l0aCBhbiAib3RoZXIiIGZpZWxkIHRoZSBzYW1lIGFzIHBy
ZXZpb3VzbHkgdXNlZCBmb3IgdGhhdA0KICAgc3RhdGUtb3duZXIvZmlsZS90eXBlIGNvbWJpbmF0
aW9uLCB3aXRoIGFuIGluY3JlbWVudGVkICJzZXFpZCIgZmllbGQuDQogICBUaGlzIHBhdHRlcm4g
Y29udGludWVzIHVudGlsIHRoZSBzZXFpZCBpcyBpbmNyZW1lbnRlZCBwYXN0DQogICBORlM0X1VJ
TlQzMl9NQVgsIGFuZCBvbmUgKG5vdCB6ZXJvKSBpcyB0aGUgbmV4dCBzZXFpZCB2YWx1ZS4NCg0K
TW9yZSBvZiB0aGUgc2FtZSBpbg0KDQo5LjQuICBTdGF0ZWlkIFNlcWlkIFZhbHVlcyBhbmQgQnl0
ZS1SYW5nZSBMb2Nrcw0KDQoxMi41LjMgaGFzIHRoaXMgdG8gc2F5IGFib3V0IGxheW91dCBzdGF0
ZWlkcywgd2hpbGUgYWNrbm93bGVkZ2luZyB0aGV5DQphcmUgZGlmZmVyZW50IGZyb20gbm9ybWFs
IHN0YXRlaWRzOg0KDQogIFRoZQ0KICAgY29ycmVjdCAic2VxaWQiIGlzIGRlZmluZWQgYXMgdGhl
IGhpZ2hlc3QgInNlcWlkIiB2YWx1ZSBmcm9tDQogICByZXNwb25zZXMgb2YgZnVsbHkgcHJvY2Vz
c2VkIExBWU9VVEdFVCBvciBMQVlPVVRSRVRVUk4gb3BlcmF0aW9ucyBvcg0KICAgYXJndW1lbnRz
IG9mIGEgZnVsbHkgcHJvY2Vzc2VkIENCX0xBWU9VVFJFQ0FMTCBvcGVyYXRpb24uDQoNCkkgdGFr
ZSDigJxmdWxseSBwcm9jZXNzZWTigJ0gdG8gbWVhbiBzdWNjZXNzZnVsLg0KDQoNCj4gDQo+IFNv
IG5vcm1hbGx5LCB3aGVuIGFuIG9wZW4gZmFpbHMgd2l0aCBCQURfU0VRSUQgdGhlIGNsaWVudCBy
ZXRyaWVzIGJ1dA0KPiBjb21iaW5lIGl0IHdpdGggYSBMT0NLIHRoYXQgZmFpbGVkIHdpdGggQkFE
X1NUQVRFSUQuIFRoZW4gaXQgbGVhZHMgdG8NCj4gdGhlIGFwcGxpY2F0aW9uIGZhaWxpbmcgd2l0
aCDigJxiYWQgZmlsZSBkZXNjcmlwdG9y4oCdIGJlY2F1c2UgYWZ0ZXINCj4gcmVjZWl2aW5nIGFu
IGVycm9yIGZvciB0aGUgbG9jayBhbmQgdHJ5aW5nIHRvIGRvIG9wZW4gcmVjb3ZlcnksIHRoZQ0K
PiBjb2RlIGRvZXNu4oCZdCBpbmNyZW1lbnQgc2VxaWQgYW5kIG9wZW4gZmFpbHMgd2l0aCBCQURf
U0VRSUQgYW5kIHdlDQo+IGRvbuKAmXQgcmV0cnkgdGhlIG9wZW4uIFNvIG5vdyB3ZSBkb27igJl0
IGhhdmUgYSBmaWxlIGRlc2NyaXB0b3IgYW5kIHdlDQo+IGZhaWwgdGhlIGxvY2suDQoNCg0KTG9v
ayBiYWNrIGF0IHRoZSBzZWN0aW9ucyBhYm92ZSwgdGhleSBzdGF0ZWQgdGhhdCB0aGUgc2VxaWQg
aXMNCm9ubHkgY2hhbmdlZCBpZiB0aGVyZSBpcyBhIGNoYW5nZSBpbiB0aGUgc2V0IG9mIGxvY2tz
IGRlc2lnbmF0ZWQuIFRoZQ0KZXJyb3IgZG9lcyBub3QgY2hhbmdlIHRoZSBzZXQgb2YgbG9ja3Mu
DQoNCllvdXIgc2VydmVyIHZlbmRvciBpcyBoYXZpbmcgdGhlIHNhbWUgaXNzdWVzIGFzIHlvdSBh
cmUgaW4gZGVjaXBoZXJpbmcNCnRoZSBydWxlcyBvbiBzZXFpZC4NCg0KDQo+IA0KPiBUaGFuayB5
b3UuDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5l
ICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiIGluDQo+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBt
YWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6
Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KPiANCg0K
DQoNCk9uIDUvMjAvMTYsIDE3OjExLCAibGludXgtbmZzLW93bmVyQHZnZXIua2VybmVsLm9yZyBv
biBiZWhhbGYgb2YgVGhvbWFzIEhheW5lcyIgPGxpbnV4LW5mcy1vd25lckB2Z2VyLmtlcm5lbC5v
cmcgb24gYmVoYWxmIG9mIGxvZ2h5ckBwcmltYXJ5ZGF0YS5jb20+IHdyb3RlOg0KDQo+DQo+PiBP
biBNYXkgMjAsIDIwMTYsIGF0IDE6MzggUE0sIE9sZ2EgS29ybmlldnNrYWlhIDxhZ2xvQHVtaWNo
LmVkdT4gd3JvdGU6DQo+PiANCj4+IEhpIGZvbGtzLA0KPj4gDQo+PiBJ4oCZbSBzZWVpbmcgYSBj
bGllbnQgYmVoYXZpb3IgdGhhdCBJIGNhbuKAmXQgZXhwbGFpbiB0aGUgcmVhc29uIGJlaGluZA0K
Pj4gKGkuZS4gc3BlYykgc28gd2FudCB0byBhc2sgZm9yIGhlbHAuDQo+PiANCj4+IFF1ZXN0aW9u
ICMxOiBTaG91bGQgYSByZWNsYWltIG9mIG9wZW4gc3RhdGUgaGFuZGxlIEJBRF9TRVFJRCB0aGUg
c2FtZQ0KPj4gd2F5IHRoZSBub3JtYWwgb3BlbiBoYW5kbGVzIEJBRF9TRVFJRCAod2hpY2ggaXM6
IGl0IGp1c3QgcmV0cmllcykuIEkNCj4+IGNhbiBzZWUgdGhhdCBkdXJpbmcgcmVjb3Zlcnkgd2Ug
bWlnaHQgbm90IHdhbnQgdGhlIHJlY292ZXJ5IHRvIGJlDQo+PiB0cnlpbmcgZm9yZXZlciBhbmQg
dGhhdOKAmXMgd2h5IHdlIGZhaWw/IEkgZG9u4oCZdCBzZWUgYW55dGhpbmcgaW4gdGhlDQo+PiBz
cGVjIHRoYXQgdGFsayBhYm91dCB0aGlz4oCmDQo+PiANCj4+IFF1ZXN0aW9uICMyOiBJbmNyZW1l
bnRpbmcgc2VxaWQgb2Ygb3BlbiBvd25lci4gSSBzZWUgdGhhdCB3aGVuIExPQ0sNCj4+IHJlY2Vp
dmVzIGFuIGVycm9yIEJBRF9TVEFURUlEL0FETUlOX1JFVk9LRUQsIHNlcXVlbmNlIGlzbuKAmXQN
Cj4+IGluY3JlbWVudGVkIGFuZCAocmVjb3ZlcnkpIE9QRU4gaXMgc2VudCB3aXRoIHRoZSBzYW1l
IHNlcWlkLiBUaGUgY29kZQ0KPj4gZXhwbGljaXRseSBzZXRzIHNlcWlkIHVuY29uZmlybWVkLiBB
Z2FpbiwgSSBkb27igJl0IGtub3cgd2hlcmUgdG8gbG9vaw0KPj4gaW4gdGhlIHNwZWMgZm9yIHNv
bWV0aGluZyBsaWtlIHRoaXMuIFdoYXQgaGFwcGVucyBpcyB0aGF0IHNlcnZlcg0KPj4gcmVwbGll
cyBiYWNrIHdpdGggQkFEX1NFUUlELg0KPg0KPlJGQyA1NjYxDQo+DQo+SXQgZG9lc27igJl0IHNh
eSBpdCBleHBsaWNpdGx5LCBidXQgeW91IG9ubHkgYnVtcCB0aGUgc2VxaWQgb24gc3VjY2Vzcy4N
Cj4NCj4zLjMuMTIuICBzdGF0ZWlkNA0KPg0KPiAgVGhlIHN0YXJ0aW5nIHZhbHVlIG9mIHRoZSAi
c2VxaWQiIGZpZWxkIGlzDQo+ICAgdW5kZWZpbmVkLiAgVGhlIHNlcnZlciBpcyByZXF1aXJlZCB0
byBpbmNyZW1lbnQgdGhlICJzZXFpZCIgZmllbGQgYnkNCj4gICBvbmUgYXQgZWFjaCB0cmFuc2l0
aW9uIG9mIHRoZSBzdGF0ZWlkLiANCj4NCj4NCj44LjIuMS4gIFN0YXRlaWQgVHlwZXMNCj4NCj4g
ICBvICBTdGF0ZWlkcyBtYXkgcmVwcmVzZW50IHNldHMgb2YgYnl0ZS1yYW5nZSBsb2Nrcy4NCj4N
Cj4gICAgICBBbGwgbG9ja3MgaGVsZCBvbiBhIHBhcnRpY3VsYXIgZmlsZSBieSBhIHBhcnRpY3Vs
YXIgb3duZXIgYW5kDQo+ICAgICAgZ290dGVuIHVuZGVyIHRoZSBhZWdpcyBvZiBhIHBhcnRpY3Vs
YXIgb3BlbiBmaWxlIGFyZSBhc3NvY2lhdGVkDQo+ICAgICAgd2l0aCBhIHNpbmdsZSBzdGF0ZWlk
IHdpdGggdGhlIHNlcWlkIGJlaW5nIGluY3JlbWVudGVkIHdoZW5ldmVyDQo+ICAgICAgTE9DSyBh
bmQgTE9DS1Ugb3BlcmF0aW9ucyBhZmZlY3QgdGhhdCBzZXQgb2YgbG9ja3MuDQo+DQo+OC4yLjIu
ICBTdGF0ZWlkIFN0cnVjdHVyZQ0KPg0KPiAgIFdoZW4gc3VjaCBhIHNldCBvZiBsb2NrcyBpcyBm
aXJzdCBjcmVhdGVkLCB0aGUgc2VydmVyIHJldHVybnMgYQ0KPiAgIHN0YXRlaWQgd2l0aCBzZXFp
ZCB2YWx1ZSBvZiBvbmUuICBPbiBzdWJzZXF1ZW50IG9wZXJhdGlvbnMgdGhhdA0KPiAgIG1vZGlm
eSB0aGUgc2V0IG9mIGxvY2tzLCB0aGUgc2VydmVyIGlzIHJlcXVpcmVkIHRvIGluY3JlbWVudCB0
aGUNCj4gICAic2VxaWQiIGZpZWxkIGJ5IG9uZSB3aGVuZXZlciBpdCByZXR1cm5zIGEgc3RhdGVp
ZCBmb3IgdGhlIHNhbWUNCj4gICBzdGF0ZS1vd25lci9maWxlL3R5cGUgY29tYmluYXRpb24gYW5k
IHRoZXJlIGlzIHNvbWUgY2hhbmdlIGluIHRoZSBzZXQNCj4gICBvZiBsb2NrcyBhY3R1YWxseSBk
ZXNpZ25hdGVkLiAgSW4gdGhpcyBjYXNlLCB0aGUgc2VydmVyIHdpbGwgcmV0dXJuIGENCj4gICBz
dGF0ZWlkIHdpdGggYW4gIm90aGVyIiBmaWVsZCB0aGUgc2FtZSBhcyBwcmV2aW91c2x5IHVzZWQg
Zm9yIHRoYXQNCj4gICBzdGF0ZS1vd25lci9maWxlL3R5cGUgY29tYmluYXRpb24sIHdpdGggYW4g
aW5jcmVtZW50ZWQgInNlcWlkIiBmaWVsZC4NCj4gICBUaGlzIHBhdHRlcm4gY29udGludWVzIHVu
dGlsIHRoZSBzZXFpZCBpcyBpbmNyZW1lbnRlZCBwYXN0DQo+ICAgTkZTNF9VSU5UMzJfTUFYLCBh
bmQgb25lIChub3QgemVybykgaXMgdGhlIG5leHQgc2VxaWQgdmFsdWUuDQo+DQo+TW9yZSBvZiB0
aGUgc2FtZSBpbg0KPg0KPjkuNC4gIFN0YXRlaWQgU2VxaWQgVmFsdWVzIGFuZCBCeXRlLVJhbmdl
IExvY2tzDQo+DQo+MTIuNS4zIGhhcyB0aGlzIHRvIHNheSBhYm91dCBsYXlvdXQgc3RhdGVpZHMs
IHdoaWxlIGFja25vd2xlZGdpbmcgdGhleQ0KPmFyZSBkaWZmZXJlbnQgZnJvbSBub3JtYWwgc3Rh
dGVpZHM6DQo+DQo+ICBUaGUNCj4gICBjb3JyZWN0ICJzZXFpZCIgaXMgZGVmaW5lZCBhcyB0aGUg
aGlnaGVzdCAic2VxaWQiIHZhbHVlIGZyb20NCj4gICByZXNwb25zZXMgb2YgZnVsbHkgcHJvY2Vz
c2VkIExBWU9VVEdFVCBvciBMQVlPVVRSRVRVUk4gb3BlcmF0aW9ucyBvcg0KPiAgIGFyZ3VtZW50
cyBvZiBhIGZ1bGx5IHByb2Nlc3NlZCBDQl9MQVlPVVRSRUNBTEwgb3BlcmF0aW9uLg0KPg0KPkkg
dGFrZSDigJxmdWxseSBwcm9jZXNzZWTigJ0gdG8gbWVhbiBzdWNjZXNzZnVsLg0KPg0KPg0KPj4g
DQo+PiBTbyBub3JtYWxseSwgd2hlbiBhbiBvcGVuIGZhaWxzIHdpdGggQkFEX1NFUUlEIHRoZSBj
bGllbnQgcmV0cmllcyBidXQNCj4+IGNvbWJpbmUgaXQgd2l0aCBhIExPQ0sgdGhhdCBmYWlsZWQg
d2l0aCBCQURfU1RBVEVJRC4gVGhlbiBpdCBsZWFkcyB0bw0KPj4gdGhlIGFwcGxpY2F0aW9uIGZh
aWxpbmcgd2l0aCDigJxiYWQgZmlsZSBkZXNjcmlwdG9y4oCdIGJlY2F1c2UgYWZ0ZXINCj4+IHJl
Y2VpdmluZyBhbiBlcnJvciBmb3IgdGhlIGxvY2sgYW5kIHRyeWluZyB0byBkbyBvcGVuIHJlY292
ZXJ5LCB0aGUNCj4+IGNvZGUgZG9lc27igJl0IGluY3JlbWVudCBzZXFpZCBhbmQgb3BlbiBmYWls
cyB3aXRoIEJBRF9TRVFJRCBhbmQgd2UNCj4+IGRvbuKAmXQgcmV0cnkgdGhlIG9wZW4uIFNvIG5v
dyB3ZSBkb27igJl0IGhhdmUgYSBmaWxlIGRlc2NyaXB0b3IgYW5kIHdlDQo+PiBmYWlsIHRoZSBs
b2NrLg0KPg0KPg0KPkxvb2sgYmFjayBhdCB0aGUgc2VjdGlvbnMgYWJvdmUsIHRoZXkgc3RhdGVk
IHRoYXQgdGhlIHNlcWlkIGlzDQo+b25seSBjaGFuZ2VkIGlmIHRoZXJlIGlzIGEgY2hhbmdlIGlu
IHRoZSBzZXQgb2YgbG9ja3MgZGVzaWduYXRlZC4gVGhlDQo+ZXJyb3IgZG9lcyBub3QgY2hhbmdl
IHRoZSBzZXQgb2YgbG9ja3MuDQo+DQo+WW91ciBzZXJ2ZXIgdmVuZG9yIGlzIGhhdmluZyB0aGUg
c2FtZSBpc3N1ZXMgYXMgeW91IGFyZSBpbiBkZWNpcGhlcmluZw0KPnRoZSBydWxlcyBvbiBzZXFp
ZC4NCj4NCg0KT2gsIHNvcnJ5LiBEaWQgeW91IG1lYW4gTkZTdjQuMSwgT2xnYT8gSWYgc28sIHBs
ZWFzZSBkaXNyZWdhcmQgbXkgZWFybGllciByZXBseS4NCg0KVHJvbmQNCg0KDQo=
On Fri, May 20, 2016 at 5:22 PM, Trond Myklebust
<[email protected]> wrote:
>
>
> On 5/20/16, 17:11, "[email protected] on behalf of Thomas
> Haynes" <[email protected] on behalf of
> [email protected]> wrote:
>
>>
>>> On May 20, 2016, at 1:38 PM, Olga Kornievskaia <[email protected]> wrote:
>>>
>>> Hi folks,
>>>
>>> I’m seeing a client behavior that I can’t explain the reason behind
>>> (i.e. spec) so want to ask for help.
>>>
>>> Question #1: Should a reclaim of open state handle BAD_SEQID the same
>>> way the normal open handles BAD_SEQID (which is: it just retries). I
>>> can see that during recovery we might not want the recovery to be
>>> trying forever and that’s why we fail? I don’t see anything in the
>>> spec that talk about this…
>>>
>>> Question #2: Incrementing seqid of open owner. I see that when LOCK
>>> receives an error BAD_STATEID/ADMIN_REVOKED, sequence isn’t
>>> incremented and (recovery) OPEN is sent with the same seqid. The code
>>> explicitly sets seqid unconfirmed. Again, I don’t know where to look
>>> in the spec for something like this. What happens is that server
>>> replies back with BAD_SEQID.
>>
>>RFC 5661
>>
>>It doesn’t say it explicitly, but you only bump the seqid on success.
>>
>>3.3.12. stateid4
>>
>> The starting value of the "seqid" field is
>> undefined. The server is required to increment the "seqid" field by
>> one at each transition of the stateid.
>>
>>
>>8.2.1. Stateid Types
>>
>> o Stateids may represent sets of byte-range locks.
>>
>> All locks held on a particular file by a particular owner and
>> gotten under the aegis of a particular open file are associated
>> with a single stateid with the seqid being incremented whenever
>> LOCK and LOCKU operations affect that set of locks.
Which seqid is the spec talking about: the stateid's seqid or the lock_seqid ?
>>
>>8.2.2. Stateid Structure
>>
>> When such a set of locks is first created, the server returns a
>> stateid with seqid value of one. On subsequent operations that
>> modify the set of locks, the server is required to increment the
>> "seqid" field by one whenever it returns a stateid for the same
>> state-owner/file/type combination and there is some change in the set
>> of locks actually designated. In this case, the server will return a
>> stateid with an "other" field the same as previously used for that
>> state-owner/file/type combination, with an incremented "seqid" field.
>> This pattern continues until the seqid is incremented past
>> NFS4_UINT32_MAX, and one (not zero) is the next seqid value.
>>
>>More of the same in
>>
>>9.4. Stateid Seqid Values and Byte-Range Locks
>>
>>12.5.3 has this to say about layout stateids, while acknowledging they
>>are different from normal stateids:
>>
>> The
>> correct "seqid" is defined as the highest "seqid" value from
>> responses of fully processed LAYOUTGET or LAYOUTRETURN operations or
>> arguments of a fully processed CB_LAYOUTRECALL operation.
>>
>>I take “fully processed” to mean successful.
>>
>>
>>>
>>> So normally, when an open fails with BAD_SEQID the client retries but
>>> combine it with a LOCK that failed with BAD_STATEID. Then it leads to
>>> the application failing with “bad file descriptor” because after
>>> receiving an error for the lock and trying to do open recovery, the
>>> code doesn’t increment seqid and open fails with BAD_SEQID and we
>>> don’t retry the open. So now we don’t have a file descriptor and we
>>> fail the lock.
>>
>>
>>Look back at the sections above, they stated that the seqid is
>>only changed if there is a change in the set of locks designated. The
>>error does not change the set of locks.
>>
>>Your server vendor is having the same issues as you are in deciphering
>>the rules on seqid.
>>
>
> Oh, sorry. Did you mean NFSv4.1, Olga? If so, please disregard my earlier
> reply.
I should have specified that it was 4.0.
Ok so I found that for 4.1 the seqid from the operations are ignored.
15.1.16.1. NFS4ERR_BAD_SEQID (Error Code 10026)
The sequence number (seqid) in a locking request is neither the next
expected number or the last number processed. These seqids are
ignored in NFSv4.1.
>
> Trond
>
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.