The spec says that open owner needs to be different for different
files yet the code returns the same value.
It is not clear to me what's broken about ida_get_new() that's used
for generation of the owner ids.
Can somebody comment on this?
Thank you.
Aren't sequence slots shared by the same open owners?
The problem I'm seeing is that a client with an outstanding open can't
respond to a delegation return because sequenced hasn't been
confirmed. I think it's problematic when there are two clients each
holding delegations and each sending an open. The server tries to
recall the delegations but neither clients will reply until the server
returns an open to the outstanding open. The server will try and not
reply to the outstanding opens until the delegations are returned.
On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
<[email protected]> wrote:
> Hi Olga,
>
> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
> <[email protected]> wrote:
>> The spec says that open owner needs to be different for different
>> files yet the code returns the same value.
>>
>> It is not clear to me what's broken about ida_get_new() that's used
>> for generation of the owner ids.
>>
>> Can somebody comment on this?
>>
>
> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
> that open owners be different for each file. In NFSv4.1 there is a
> requirement that they be unique to each credential/principal, but only
> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>
> The current choice of scheme where we share open_owners between files
> was largely motivated by a desire not to overload the NFSv4.0 servers
> with lots of open_owner structures. In NFSv4.0, those structures are
> required to be cached by the server for a while after the file has
> been closed.
>
> Cheers
> Trond
>
>
> --
> Trond Myklebust
>
> Linux NFS client maintainer, PrimaryData
>
> [email protected]
Hi Olga,
On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
<[email protected]> wrote:
> The spec says that open owner needs to be different for different
> files yet the code returns the same value.
>
> It is not clear to me what's broken about ida_get_new() that's used
> for generation of the owner ids.
>
> Can somebody comment on this?
>
As far as I know, there is no requirement in either NFSv4 or NFSv4.1
that open owners be different for each file. In NFSv4.1 there is a
requirement that they be unique to each credential/principal, but only
if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
The current choice of scheme where we share open_owners between files
was largely motivated by a desire not to overload the NFSv4.0 servers
with lots of open_owner structures. In NFSv4.0, those structures are
required to be cached by the server for a while after the file has
been closed.
Cheers
Trond
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
[email protected]
On Mon, Oct 6, 2014 at 11:17 AM, Trond Myklebust
<[email protected]> wrote:
> On Mon, Oct 6, 2014 at 11:07 AM, Olga Kornievskaia
> <[email protected]> wrote:
>> On Mon, Oct 6, 2014 at 10:49 AM, Trond Myklebust
>> <[email protected]> wrote:
>>> On Mon, Oct 6, 2014 at 10:12 AM, Olga Kornievskaia
>>> <[email protected]> wrote:
>>>> Aren't sequence slots shared by the same open owners?
>>>>
>>>> The problem I'm seeing is that a client with an outstanding open can't
>>>> respond to a delegation return because sequenced hasn't been
>>>> confirmed. I think it's problematic when there are two clients each
>>>> holding delegations and each sending an open. The server tries to
>>>> recall the delegations but neither clients will reply until the server
>>>> returns an open to the outstanding open. The server will try and not
>>>> reply to the outstanding opens until the delegations are returned.
>>>
>>> I'm not sure that I understand.
>>>
>>> Why should the server refuse to respond to OPEN calls just because a
>>> delegation recall is outstanding? That sounds like a catastrophic
>>> design for a server. How is the client going to convert its cached
>>> open/lock state into open stateids and lock stateids if the server
>>> won't respond to OPEN calls for that file?
>>>
>>> If the server cannot respond to the OPEN because it conflicts with a
>>> delegation being recalled, then the right thing to do is to return
>>> NFS4ERR_DELAY. That allows the client to make progress on other OPEN
>>> requests that may be required in order to return the conflicting
>>> delegation, and that may be serialised behind that blocked OPEN.
>>> I can imagine this might be the case if the client holds a delegation
>>> for a file, and then is asked to OPEN a hard link to the same file.
>>> The client has no way to know a priori that it is opening the same
>>> file through the hard link, so it doesn't know that it conflicts with
>>> the delegation.
>>>
>>
>> Sorry I should have provided a clear example. But you are right, the
>> server is not replying to the open because of the conflicting
>> opens/delegations.
>>
>> ClientA opens file A gets a write delegation.
>> ClientB open files B gets a write delegation.
>> ClientA sends an open for file B
>> ClientB sends an open for file A
>>
>> Thank you for pointing out that a server should/can return
>> NFS4ERR_DELAY to the open.
>>
>> I'm still new to this, so I apologize. But to me it seems like at
>> should be totally reasonable from the server-side to expect for the
>> delegation to file A to the returned independent of the opening of
>> file B.
>
> No. The problem with that assumption are the serialisation issues:
>
> a) There is no serialisation between OPEN and the recall callback from
> the server. The server cannot rely on the client having received the
> callback when it decides how to handle the OPEN.
> b) There is _mandatory_ serialisation between OPEN requests on the
> client. Unless the server replies to the OPEN, then all other OPEN
> requests that are serialised behind that OPEN are required by RFC3530
> to wait.
Yes, for the same open owner, the requests must be serialized. Given
the choice to use the same open owner for all the files for a given
credential/principal, it imposes strict requirements on the server
implementation.
Thanks for the explanation.
> Cheers
> Trond
>
>>
>>> Cheers
>>> Trond
>>>
>>>>
>>>> On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
>>>> <[email protected]> wrote:
>>>>> Hi Olga,
>>>>>
>>>>> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
>>>>> <[email protected]> wrote:
>>>>>> The spec says that open owner needs to be different for different
>>>>>> files yet the code returns the same value.
>>>>>>
>>>>>> It is not clear to me what's broken about ida_get_new() that's used
>>>>>> for generation of the owner ids.
>>>>>>
>>>>>> Can somebody comment on this?
>>>>>>
>>>>>
>>>>> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
>>>>> that open owners be different for each file. In NFSv4.1 there is a
>>>>> requirement that they be unique to each credential/principal, but only
>>>>> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>>>>>
>>>>> The current choice of scheme where we share open_owners between files
>>>>> was largely motivated by a desire not to overload the NFSv4.0 servers
>>>>> with lots of open_owner structures. In NFSv4.0, those structures are
>>>>> required to be cached by the server for a while after the file has
>>>>> been closed.
>>>>>
>>>>> Cheers
>>>>> Trond
>>>>>
>>>>>
>>>>> --
>>>>> Trond Myklebust
>>>>>
>>>>> Linux NFS client maintainer, PrimaryData
>>>>>
>>>>> [email protected]
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Trond Myklebust
>
> Linux NFS client maintainer, PrimaryData
>
> [email protected]
On Mon, Oct 6, 2014 at 11:07 AM, Olga Kornievskaia
<[email protected]> wrote:
> On Mon, Oct 6, 2014 at 10:49 AM, Trond Myklebust
> <[email protected]> wrote:
>> On Mon, Oct 6, 2014 at 10:12 AM, Olga Kornievskaia
>> <[email protected]> wrote:
>>> Aren't sequence slots shared by the same open owners?
>>>
>>> The problem I'm seeing is that a client with an outstanding open can't
>>> respond to a delegation return because sequenced hasn't been
>>> confirmed. I think it's problematic when there are two clients each
>>> holding delegations and each sending an open. The server tries to
>>> recall the delegations but neither clients will reply until the server
>>> returns an open to the outstanding open. The server will try and not
>>> reply to the outstanding opens until the delegations are returned.
>>
>> I'm not sure that I understand.
>>
>> Why should the server refuse to respond to OPEN calls just because a
>> delegation recall is outstanding? That sounds like a catastrophic
>> design for a server. How is the client going to convert its cached
>> open/lock state into open stateids and lock stateids if the server
>> won't respond to OPEN calls for that file?
>>
>> If the server cannot respond to the OPEN because it conflicts with a
>> delegation being recalled, then the right thing to do is to return
>> NFS4ERR_DELAY. That allows the client to make progress on other OPEN
>> requests that may be required in order to return the conflicting
>> delegation, and that may be serialised behind that blocked OPEN.
>> I can imagine this might be the case if the client holds a delegation
>> for a file, and then is asked to OPEN a hard link to the same file.
>> The client has no way to know a priori that it is opening the same
>> file through the hard link, so it doesn't know that it conflicts with
>> the delegation.
>>
>
> Sorry I should have provided a clear example. But you are right, the
> server is not replying to the open because of the conflicting
> opens/delegations.
>
> ClientA opens file A gets a write delegation.
> ClientB open files B gets a write delegation.
> ClientA sends an open for file B
> ClientB sends an open for file A
>
> Thank you for pointing out that a server should/can return
> NFS4ERR_DELAY to the open.
>
> I'm still new to this, so I apologize. But to me it seems like at
> should be totally reasonable from the server-side to expect for the
> delegation to file A to the returned independent of the opening of
> file B.
No. The problem with that assumption are the serialisation issues:
a) There is no serialisation between OPEN and the recall callback from
the server. The server cannot rely on the client having received the
callback when it decides how to handle the OPEN.
b) There is _mandatory_ serialisation between OPEN requests on the
client. Unless the server replies to the OPEN, then all other OPEN
requests that are serialised behind that OPEN are required by RFC3530
to wait.
Cheers
Trond
>
>> Cheers
>> Trond
>>
>>>
>>> On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
>>> <[email protected]> wrote:
>>>> Hi Olga,
>>>>
>>>> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
>>>> <[email protected]> wrote:
>>>>> The spec says that open owner needs to be different for different
>>>>> files yet the code returns the same value.
>>>>>
>>>>> It is not clear to me what's broken about ida_get_new() that's used
>>>>> for generation of the owner ids.
>>>>>
>>>>> Can somebody comment on this?
>>>>>
>>>>
>>>> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
>>>> that open owners be different for each file. In NFSv4.1 there is a
>>>> requirement that they be unique to each credential/principal, but only
>>>> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>>>>
>>>> The current choice of scheme where we share open_owners between files
>>>> was largely motivated by a desire not to overload the NFSv4.0 servers
>>>> with lots of open_owner structures. In NFSv4.0, those structures are
>>>> required to be cached by the server for a while after the file has
>>>> been closed.
>>>>
>>>> Cheers
>>>> Trond
>>>>
>>>>
>>>> --
>>>> Trond Myklebust
>>>>
>>>> Linux NFS client maintainer, PrimaryData
>>>>
>>>> [email protected]
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
[email protected]
On Mon, Oct 6, 2014 at 10:49 AM, Trond Myklebust
<[email protected]> wrote:
> On Mon, Oct 6, 2014 at 10:12 AM, Olga Kornievskaia
> <[email protected]> wrote:
>> Aren't sequence slots shared by the same open owners?
>>
>> The problem I'm seeing is that a client with an outstanding open can't
>> respond to a delegation return because sequenced hasn't been
>> confirmed. I think it's problematic when there are two clients each
>> holding delegations and each sending an open. The server tries to
>> recall the delegations but neither clients will reply until the server
>> returns an open to the outstanding open. The server will try and not
>> reply to the outstanding opens until the delegations are returned.
>
> I'm not sure that I understand.
>
> Why should the server refuse to respond to OPEN calls just because a
> delegation recall is outstanding? That sounds like a catastrophic
> design for a server. How is the client going to convert its cached
> open/lock state into open stateids and lock stateids if the server
> won't respond to OPEN calls for that file?
>
> If the server cannot respond to the OPEN because it conflicts with a
> delegation being recalled, then the right thing to do is to return
> NFS4ERR_DELAY. That allows the client to make progress on other OPEN
> requests that may be required in order to return the conflicting
> delegation, and that may be serialised behind that blocked OPEN.
> I can imagine this might be the case if the client holds a delegation
> for a file, and then is asked to OPEN a hard link to the same file.
> The client has no way to know a priori that it is opening the same
> file through the hard link, so it doesn't know that it conflicts with
> the delegation.
>
Sorry I should have provided a clear example. But you are right, the
server is not replying to the open because of the conflicting
opens/delegations.
ClientA opens file A gets a write delegation.
ClientB open files B gets a write delegation.
ClientA sends an open for file B
ClientB sends an open for file A
Thank you for pointing out that a server should/can return
NFS4ERR_DELAY to the open.
I'm still new to this, so I apologize. But to me it seems like at
should be totally reasonable from the server-side to expect for the
delegation to file A to the returned independent of the opening of
file B.
> Cheers
> Trond
>
>>
>> On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
>> <[email protected]> wrote:
>>> Hi Olga,
>>>
>>> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
>>> <[email protected]> wrote:
>>>> The spec says that open owner needs to be different for different
>>>> files yet the code returns the same value.
>>>>
>>>> It is not clear to me what's broken about ida_get_new() that's used
>>>> for generation of the owner ids.
>>>>
>>>> Can somebody comment on this?
>>>>
>>>
>>> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
>>> that open owners be different for each file. In NFSv4.1 there is a
>>> requirement that they be unique to each credential/principal, but only
>>> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>>>
>>> The current choice of scheme where we share open_owners between files
>>> was largely motivated by a desire not to overload the NFSv4.0 servers
>>> with lots of open_owner structures. In NFSv4.0, those structures are
>>> required to be cached by the server for a while after the file has
>>> been closed.
>>>
>>> Cheers
>>> Trond
>>>
>>>
>>> --
>>> Trond Myklebust
>>>
>>> Linux NFS client maintainer, PrimaryData
>>>
>>> [email protected]
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Oct 6, 2014 at 10:12 AM, Olga Kornievskaia
<[email protected]> wrote:
> Aren't sequence slots shared by the same open owners?
>
> The problem I'm seeing is that a client with an outstanding open can't
> respond to a delegation return because sequenced hasn't been
> confirmed. I think it's problematic when there are two clients each
> holding delegations and each sending an open. The server tries to
> recall the delegations but neither clients will reply until the server
> returns an open to the outstanding open. The server will try and not
> reply to the outstanding opens until the delegations are returned.
I'm not sure that I understand.
Why should the server refuse to respond to OPEN calls just because a
delegation recall is outstanding? That sounds like a catastrophic
design for a server. How is the client going to convert its cached
open/lock state into open stateids and lock stateids if the server
won't respond to OPEN calls for that file?
If the server cannot respond to the OPEN because it conflicts with a
delegation being recalled, then the right thing to do is to return
NFS4ERR_DELAY. That allows the client to make progress on other OPEN
requests that may be required in order to return the conflicting
delegation, and that may be serialised behind that blocked OPEN.
I can imagine this might be the case if the client holds a delegation
for a file, and then is asked to OPEN a hard link to the same file.
The client has no way to know a priori that it is opening the same
file through the hard link, so it doesn't know that it conflicts with
the delegation.
Cheers
Trond
>
> On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
> <[email protected]> wrote:
>> Hi Olga,
>>
>> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
>> <[email protected]> wrote:
>>> The spec says that open owner needs to be different for different
>>> files yet the code returns the same value.
>>>
>>> It is not clear to me what's broken about ida_get_new() that's used
>>> for generation of the owner ids.
>>>
>>> Can somebody comment on this?
>>>
>>
>> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
>> that open owners be different for each file. In NFSv4.1 there is a
>> requirement that they be unique to each credential/principal, but only
>> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>>
>> The current choice of scheme where we share open_owners between files
>> was largely motivated by a desire not to overload the NFSv4.0 servers
>> with lots of open_owner structures. In NFSv4.0, those structures are
>> required to be cached by the server for a while after the file has
>> been closed.
>>
>> Cheers
>> Trond
>>
>>
>> --
>> Trond Myklebust
>>
>> Linux NFS client maintainer, PrimaryData
>>
>> [email protected]
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Oct 6, 2014 at 11:54 AM, Olga Kornievskaia
<[email protected]> wrote:
> On Mon, Oct 6, 2014 at 11:17 AM, Trond Myklebust
> <[email protected]> wrote:
>> On Mon, Oct 6, 2014 at 11:07 AM, Olga Kornievskaia
>> <[email protected]> wrote:
>>> On Mon, Oct 6, 2014 at 10:49 AM, Trond Myklebust
>>> <[email protected]> wrote:
>>>> On Mon, Oct 6, 2014 at 10:12 AM, Olga Kornievskaia
>>>> <[email protected]> wrote:
>>>>> Aren't sequence slots shared by the same open owners?
>>>>>
>>>>> The problem I'm seeing is that a client with an outstanding open can't
>>>>> respond to a delegation return because sequenced hasn't been
>>>>> confirmed. I think it's problematic when there are two clients each
>>>>> holding delegations and each sending an open. The server tries to
>>>>> recall the delegations but neither clients will reply until the server
>>>>> returns an open to the outstanding open. The server will try and not
>>>>> reply to the outstanding opens until the delegations are returned.
>>>>
>>>> I'm not sure that I understand.
>>>>
>>>> Why should the server refuse to respond to OPEN calls just because a
>>>> delegation recall is outstanding? That sounds like a catastrophic
>>>> design for a server. How is the client going to convert its cached
>>>> open/lock state into open stateids and lock stateids if the server
>>>> won't respond to OPEN calls for that file?
>>>>
>>>> If the server cannot respond to the OPEN because it conflicts with a
>>>> delegation being recalled, then the right thing to do is to return
>>>> NFS4ERR_DELAY. That allows the client to make progress on other OPEN
>>>> requests that may be required in order to return the conflicting
>>>> delegation, and that may be serialised behind that blocked OPEN.
>>>> I can imagine this might be the case if the client holds a delegation
>>>> for a file, and then is asked to OPEN a hard link to the same file.
>>>> The client has no way to know a priori that it is opening the same
>>>> file through the hard link, so it doesn't know that it conflicts with
>>>> the delegation.
>>>>
>>>
>>> Sorry I should have provided a clear example. But you are right, the
>>> server is not replying to the open because of the conflicting
>>> opens/delegations.
>>>
>>> ClientA opens file A gets a write delegation.
>>> ClientB open files B gets a write delegation.
>>> ClientA sends an open for file B
>>> ClientB sends an open for file A
>>>
>>> Thank you for pointing out that a server should/can return
>>> NFS4ERR_DELAY to the open.
>>>
>>> I'm still new to this, so I apologize. But to me it seems like at
>>> should be totally reasonable from the server-side to expect for the
>>> delegation to file A to the returned independent of the opening of
>>> file B.
>>
>> No. The problem with that assumption are the serialisation issues:
>>
>> a) There is no serialisation between OPEN and the recall callback from
>> the server. The server cannot rely on the client having received the
>> callback when it decides how to handle the OPEN.
>> b) There is _mandatory_ serialisation between OPEN requests on the
>> client. Unless the server replies to the OPEN, then all other OPEN
>> requests that are serialised behind that OPEN are required by RFC3530
>> to wait.
>
> Yes, for the same open owner, the requests must be serialized. Given
> the choice to use the same open owner for all the files for a given
> credential/principal, it imposes strict requirements on the server
> implementation.
>
> Thanks for the explanation.
Would it be acceptable to somehow have an option that would allow for
faster-than-a-normal-RPC-timeout as an alternative to expecting to get
an ERR_DELAY from the server?
>
>
>
>> Cheers
>> Trond
>>
>>>
>>>> Cheers
>>>> Trond
>>>>
>>>>>
>>>>> On Mon, Oct 6, 2014 at 9:57 AM, Trond Myklebust
>>>>> <[email protected]> wrote:
>>>>>> Hi Olga,
>>>>>>
>>>>>> On Mon, Oct 6, 2014 at 9:48 AM, Olga Kornievskaia
>>>>>> <[email protected]> wrote:
>>>>>>> The spec says that open owner needs to be different for different
>>>>>>> files yet the code returns the same value.
>>>>>>>
>>>>>>> It is not clear to me what's broken about ida_get_new() that's used
>>>>>>> for generation of the owner ids.
>>>>>>>
>>>>>>> Can somebody comment on this?
>>>>>>>
>>>>>>
>>>>>> As far as I know, there is no requirement in either NFSv4 or NFSv4.1
>>>>>> that open owners be different for each file. In NFSv4.1 there is a
>>>>>> requirement that they be unique to each credential/principal, but only
>>>>>> if you negotiate EXCHGID4_FLAG_BIND_PRINC_STATEID.
>>>>>>
>>>>>> The current choice of scheme where we share open_owners between files
>>>>>> was largely motivated by a desire not to overload the NFSv4.0 servers
>>>>>> with lots of open_owner structures. In NFSv4.0, those structures are
>>>>>> required to be cached by the server for a while after the file has
>>>>>> been closed.
>>>>>>
>>>>>> Cheers
>>>>>> Trond
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Trond Myklebust
>>>>>>
>>>>>> Linux NFS client maintainer, PrimaryData
>>>>>>
>>>>>> [email protected]
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>> the body of a message to [email protected]
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>> --
>> Trond Myklebust
>>
>> Linux NFS client maintainer, PrimaryData
>>
>> [email protected]