Hi,
In our NFSv4.1 Delegations testing we observed that NFS client sends
WRITE requests to the server even when granted WRITE_DELEGATION. I see
that only OPEN/CLOSE and LOCK operations are served locally by the client.
Please confirm if this behavior is expected and if yes, can it be
improved to avoid sending WRITEs to the server until the application
does fsync.
Thanks,
Soumya
(Resending due to accidentally using HTML instead of plain text)
On Wed, 2018-12-05 at 15:22 -0500, Anna Schumaker wrote:
> Hi Soumya,
>
> What conditions are you seeing the writes? It could be the kernel trying to
> flush out data to free up memory for other processes.
>
> I don't think preemptively writing back data is necessarily a bad thing
> though, even with a write delegation. If the server decides to recall the
> delegation, we wouldn't keep it waiting as long while we write back local
> data. Additionally, if the client crashes then any updates might have already
> had a chance to sync to the server.
>
> I hope this helps!
> Anna
>
> On Mon, Dec 3, 2018 at 2:55 AM Soumya Koduri <[email protected]> wrote:
> > Hi,
> >
> > In our NFSv4.1 Delegations testing we observed that NFS client sends
> > WRITE requests to the server even when granted WRITE_DELEGATION. I see
> > that only OPEN/CLOSE and LOCK operations are served locally by the client.
> >
> > Please confirm if this behavior is expected and if yes, can it be
> > improved to avoid sending WRITEs to the server until the application
> > does fsync.
> >
> > Thanks,
> > Soumya
Hi Anna,
Thanks a lot for the response. To understand client behavior under
various scenarios, I started with a simple "C" program which does below
operations in a loop on a nfsv4.1 mount point-
while(..)
{
OPEN,
LOCK,
READ/WRITE,
UNLCK,
CLOSE
}
While examining the pkt traces, I observed that even when granted
WRITE_DELEGATION, there are WRITEs (though asynchronous & followed by
COMMIT) sent to the server for each of those iterations. So could it be
because of the kernel flushing out the data?
Also I noticed that the behavior of WRITE_DELEGATION granted for
OPEN(O_RDWR) is different when compared to the one granted to OPEN(O_WRONLY)
1) When the file is opened with OPEN4_SHARE_ACCESS_WRITE and a
OPEN_DELEGATE_WRITE delegation is granted, subsequent OPENs are avoided
and not sent to the server as expected, whereas
If the file is opened with OPEN4_SHARE_ACCESS_BOTH and granted
OPEN_DELEGATE_WRITE, the next open is sent to server resulting in server
recalling the delegation it granted to the same client.
2) And to that recall request, client responded back with EBADHANDLE error.
This doesn't seem right. Could you please confirm.
(Maybe the server also can avoid conflicting the OPENfrom the same
client to which it granted the delegation)
Thanks,
Soumya
On 12/6/18 1:52 AM, Anna Schumaker wrote:
> Hi Soumya,
>
> What conditions are you seeing the writes? It could be the kernel
> trying to flush out data to free up memory for other processes.
>
> I don't think preemptively writing back data is necessarily a bad thing
> though, even with a write delegation. If the server decides to recall
> the delegation, we wouldn't keep it waiting as long while we write back
> local data. Additionally, if the client crashes then any updates might
> have already had a chance to sync to the server.
>
> I hope this helps!
> Anna
>
> On Mon, Dec 3, 2018 at 2:55 AM Soumya Koduri <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi,
>
> In our NFSv4.1 Delegations testing we observed that NFS client sends
> WRITE requests to the server even when granted WRITE_DELEGATION. I see
> that only OPEN/CLOSE and LOCK operations are served locally by the
> client.
>
> Please confirm if this behavior is expected and if yes, can it be
> improved to avoid sending WRITEs to the server until the application
> does fsync.
>
> Thanks,
> Soumya
>
On 12/6/18 8:40 PM, Soumya Koduri wrote:
> Hi Anna,
>
> Thanks a lot for the response. To understand client behavior under
> various scenarios, I started with a simple "C" program which does below
> operations in a loop on a nfsv4.1 mount point-
>
> while(..)
> {
> OPEN,
> LOCK,
> READ/WRITE,
> UNLCK,
> CLOSE
> }
>
> While examining the pkt traces, I observed that even when granted
> WRITE_DELEGATION, there are WRITEs (though asynchronous & followed by
> COMMIT) sent to the server for each of those iterations. So could it be
> because of the kernel flushing out the data?
>
> Also I noticed that the behavior of WRITE_DELEGATION granted for
> OPEN(O_RDWR) is different when compared to the one granted to
> OPEN(O_WRONLY)
>
> 1) When the file is opened with OPEN4_SHARE_ACCESS_WRITE and a
> OPEN_DELEGATE_WRITE delegation is granted, subsequent OPENs are avoided
> and not sent to the server as expected, whereas
> If the file is opened with OPEN4_SHARE_ACCESS_BOTH and granted
> OPEN_DELEGATE_WRITE, the next open is sent to server resulting in server
> recalling the delegation it granted to the same client.
>
> 2) And to that recall request, client responded back with EBADHANDLE error.
>
> This doesn't seem right. Could you please confirm.
>
> (Maybe the server also can avoid conflicting the OPENfrom the same
> client to which it granted the delegation)
Apologies. This seem to have resulted from a server-side issue. On
further debugging, I found that server rejected READ op with
ERR_BAD_STATEID if there is WRITE_delegation granted to that file. This
may have led nfs client to discard the delegation granted and send
subsequent OPEN to server.
I think the server should allow READs if the access is
OPEN4_SHARE_ACCESS_BOTH and its the same client which holds the write
delegation. Will send out a patch to fix it.
So the only question I have for now is if/how client can save WRITEs
from being sent to the server while holding write delegation. Please
share your thoughts.
Thanks,
Soumya
>
> Thanks,
> Soumya
>
> On 12/6/18 1:52 AM, Anna Schumaker wrote:
>> Hi Soumya,
>>
>> What conditions are you seeing the writes? It could be the kernel
>> trying to flush out data to free up memory for other processes.
>>
>> I don't think preemptively writing back data is necessarily a bad
>> thing though, even with a write delegation. If the server decides to
>> recall the delegation, we wouldn't keep it waiting as long while we
>> write back local data. Additionally, if the client crashes then any
>> updates might have already had a chance to sync to the server.
>>
>> I hope this helps!
>> Anna
>>
>> On Mon, Dec 3, 2018 at 2:55 AM Soumya Koduri <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hi,
>>
>> In our NFSv4.1 Delegations testing we observed that NFS client sends
>> WRITE requests to the server even when granted WRITE_DELEGATION. I
>> see
>> that only OPEN/CLOSE and LOCK operations are served locally by the
>> client.
>>
>> Please confirm if this behavior is expected and if yes, can it be
>> improved to avoid sending WRITEs to the server until the application
>> does fsync.
>>
>> Thanks,
>> Soumya
>>