DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6ADA92146D
Subject: Re: NFS client behavior on WRITE_DELEGATION
From:   Soumya Koduri <skoduri@redhat.com>
To:     Anna Schumaker <schumakeranna@gmail.com>
Cc:     linux-nfs@vger.kernel.org, Matt Benjamin <mbenjamin@redhat.com>
References: <68c7c07a-949d-1ce0-ec7e-5ab71ffaaa3b@redhat.com>
 <CAFX2JfmYMnKoiQ0Kjt8SGLx2H=3GwwtLHco0wMR4xVHkJpLaoQ@mail.gmail.com>
 <9ebd1b22-234c-f9ae-b3f8-015fe8e81a27@redhat.com>
Message-ID: <d20c87e0-dc56-70e6-aef8-0131e5d8fd01@redhat.com>
Date:   Fri, 7 Dec 2018 20:36:53 +0530
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.0
MIME-Version: 1.0
In-Reply-To: <9ebd1b22-234c-f9ae-b3f8-015fe8e81a27@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk


On 12/6/18 8:40 PM, Soumya Koduri wrote:
> Hi Anna,
> 
> Thanks a lot for the response. To understand client behavior under 
> various scenarios, I started with a simple "C" program which does below 
> operations in a loop on a nfsv4.1 mount point-
> 
> while(..)
> {
> OPEN,
> LOCK,
> READ/WRITE,
> UNLCK,
> CLOSE
> }
> 
> While examining the pkt traces, I observed that even when granted 
> WRITE_DELEGATION, there are WRITEs (though asynchronous & followed by 
> COMMIT) sent to the server for each of those iterations. So could it be 
> because of the kernel flushing out the data?
> 
> Also I noticed that the behavior of WRITE_DELEGATION granted for 
> OPEN(O_RDWR) is different when compared to the one granted to 
> OPEN(O_WRONLY)
> 
> 1) When the file is opened with OPEN4_SHARE_ACCESS_WRITE and a 
> OPEN_DELEGATE_WRITE delegation is granted, subsequent OPENs are avoided 
> and not sent to the server as expected, whereas
> If the file is opened with OPEN4_SHARE_ACCESS_BOTH and granted 
> OPEN_DELEGATE_WRITE, the next open is sent to server resulting in server 
> recalling the delegation it granted to the same client.
> 
> 2) And to that recall request, client responded back with EBADHANDLE error.
> 
> This doesn't seem right. Could you please confirm.
> 
> (Maybe the server also can avoid conflicting the OPENfrom the same 
> client to which it granted the delegation)

Apologies. This seem to have resulted from a server-side issue. On 
further debugging, I found that server rejected READ op with 
ERR_BAD_STATEID if there is WRITE_delegation granted to that file. This 
may have led nfs client to discard the delegation granted and send 
subsequent OPEN to server.

I think the server should allow READs if the access is 
OPEN4_SHARE_ACCESS_BOTH and its the same client which holds the write 
delegation. Will send out a patch to fix it.

So the only question I have for now is if/how client can save WRITEs 
from being sent to the server while holding write delegation. Please 
share your thoughts.

Thanks,
Soumya


> 
> Thanks,
> Soumya
> 
> On 12/6/18 1:52 AM, Anna Schumaker wrote:
>> Hi Soumya,
>>
>> What conditions are you seeing the writes?  It could be the kernel 
>> trying to flush out data to free up memory for other processes.
>>
>> I don't think preemptively writing back data is necessarily a bad 
>> thing though, even with a write delegation.  If the server decides to 
>> recall the delegation, we wouldn't keep it waiting as long while we 
>> write back local data.  Additionally, if the client crashes then any 
>> updates might have already had a chance to sync to the server.
>>
>> I hope this helps!
>> Anna
>>
>> On Mon, Dec 3, 2018 at 2:55 AM Soumya Koduri <skoduri@redhat.com 
>> <mailto:skoduri@redhat.com>> wrote:
>>
>>     Hi,
>>
>>     In our NFSv4.1 Delegations testing we observed that NFS client sends
>>     WRITE requests to the server even when granted WRITE_DELEGATION. I 
>> see
>>     that only OPEN/CLOSE and LOCK operations are served locally by the
>>     client.
>>
>>     Please confirm if this behavior is expected and if yes, can it be
>>     improved to avoid sending WRITEs to the server until the application
>>     does fsync.
>>
>>     Thanks,
>>     Soumya
>>