Hi folks,
we run into following issue:
on a client host we have two processes - A and B.
Process A opens a file and reads some data:
OPEN -> LAYOUTGET -> READ.
Notice, that layout has a roc set to true.
Process B opens the same file:
OPEN -> READ.
Client obviously re-used layout information.
Now, process A closes the file and as a result,
layout becomes invalid. Nevertheless, process B
still assumes that layout still valid:
I thinks this is the source of the problem:
fs/nfs/pnfs.c:
1074 list_for_each_entry(ctx, &nfsi->open_files, list) {
1075 state = ctx->state;
1076 /* Don't return layout if there is open file state */
1077 if (state != NULL && state->state != 0)
1078 goto out_noroc;
1079 }
1080
this is with 3.19. The RHEL7 kernel send yet another LAYOUTGET when the process A
closes the file.
Tigran.
On Fri, 27 Mar 2015 12:24:14 +0100 (CET)
"Mkrtchyan, Tigran" <[email protected]> wrote:
>
>
> Hi folks,
>
> we run into following issue:
>
> on a client host we have two processes - A and B.
> Process A opens a file and reads some data:
>
> OPEN -> LAYOUTGET -> READ.
> Notice, that layout has a roc set to true.
>
> Process B opens the same file:
>
> OPEN -> READ.
>
> Client obviously re-used layout information.
> Now, process A closes the file and as a result,
> layout becomes invalid. Nevertheless, process B
> still assumes that layout still valid:
>
So what happened at the CLOSE here? Was there an OPEN_DOWNGRADE or
something, or do these processes represent different openowners?
My understanding is that layouts with ROC flag set are valid until the
_last_ close and until any delegations are returned. The spec says
this, which is somewhat vague, but seems to imply that that
interpretation is correct:
"A client may also hold a layout across multiple OPENs."
With the pnfs server we were playing with internally, we had an object
that tracked layouts on a per-file+per-client basis. Each open and
delegation held a reference to this object and when the last reference
was put, we returned the layout to the fs.
> I thinks this is the source of the problem:
>
> fs/nfs/pnfs.c:
>
> 1074 list_for_each_entry(ctx, &nfsi->open_files, list) {
> 1075 state = ctx->state;
> 1076 /* Don't return layout if there is open file state */
> 1077 if (state != NULL && state->state != 0)
> 1078 goto out_noroc;
> 1079 }
> 1080
>
>
> this is with 3.19. The RHEL7 kernel send yet another LAYOUTGET when the process A
> closes the file.
>
That seems like correct behavior to me. I don't think the server should
invalidate the layout in this situation.
That said, we probably need to have some clarification on the ROC flag
in the spec, or we end up with clients and servers potentially
disagreeing about when a layout is returned to the fs.
--
Jeff Layton <[email protected]>
On Fri, Mar 27, 2015 at 7:24 AM, Mkrtchyan, Tigran
<[email protected]> wrote:
>
>
> Hi folks,
>
> we run into following issue:
>
> on a client host we have two processes - A and B.
> Process A opens a file and reads some data:
>
> OPEN -> LAYOUTGET -> READ.
> Notice, that layout has a roc set to true.
>
> Process B opens the same file:
>
> OPEN -> READ.
>
> Client obviously re-used layout information.
> Now, process A closes the file and as a result,
> layout becomes invalid.
No it doesn't. See the errata.
Cheers
Trond
which means that even I I set roc flag, I still need to recall layout.
----- Original Message -----
> From: "Trond Myklebust" <[email protected]>
> To: "Mkrtchyan, Tigran" <[email protected]>
> Cc: "linux-nfs list" <[email protected]>
> Sent: Friday, March 27, 2015 3:08:39 PM
> Subject: Re: pNFS layout, ROC and multiple opens
> On Fri, Mar 27, 2015 at 7:24 AM, Mkrtchyan, Tigran
> <[email protected]> wrote:
>>
>>
>> Hi folks,
>>
>> we run into following issue:
>>
>> on a client host we have two processes - A and B.
>> Process A opens a file and reads some data:
>>
>> OPEN -> LAYOUTGET -> READ.
>> Notice, that layout has a roc set to true.
>>
>> Process B opens the same file:
>>
>> OPEN -> READ.
>>
>> Client obviously re-used layout information.
>> Now, process A closes the file and as a result,
>> layout becomes invalid.
>
> No it doesn't. See the errata.
>
> Cheers
> Trond
On Fri, Mar 27, 2015 at 10:33 AM, Mkrtchyan, Tigran
<[email protected]> wrote:
> which means that even I I set roc flag, I still need to recall layout.
No. It means that the layout is automatically returned on last close
and/or delegreturn. If you want it returned earlier then you need to
send a cb_layoutrecall.
> ----- Original Message -----
>> From: "Trond Myklebust" <[email protected]>
>> To: "Mkrtchyan, Tigran" <[email protected]>
>> Cc: "linux-nfs list" <[email protected]>
>> Sent: Friday, March 27, 2015 3:08:39 PM
>> Subject: Re: pNFS layout, ROC and multiple opens
>
>> On Fri, Mar 27, 2015 at 7:24 AM, Mkrtchyan, Tigran
>> <[email protected]> wrote:
>>>
>>>
>>> Hi folks,
>>>
>>> we run into following issue:
>>>
>>> on a client host we have two processes - A and B.
>>> Process A opens a file and reads some data:
>>>
>>> OPEN -> LAYOUTGET -> READ.
>>> Notice, that layout has a roc set to true.
>>>
>>> Process B opens the same file:
>>>
>>> OPEN -> READ.
>>>
>>> Client obviously re-used layout information.
>>> Now, process A closes the file and as a result,
>>> layout becomes invalid.
>>
>> No it doesn't. See the errata.
>>
>> Cheers
>> Trond
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
[email protected]