2014-01-16 11:31:55

by ranjith ruban

[permalink] [raw]
Subject: nfs attribute cache question

Hello

I had set nfs mount actiomeo option for 300 seconds for a share. The
client is an openvz server with container filesystems in nfs . Server
is netapp.

While going through the tcpdump i see below

125 2014-01-15 16:06:01.770522 10.X.0.X 10.X.0.X NFS
190 V3 GETATTR Call (Reply In 126), FH: 0x496fe5c5
126 2014-01-15 16:06:01.770637 10.X.0.X 10.X.0.X NFS
182 V3 GETATTR Reply (Call In 125) Regular File mode: 0644 uid:
0 gid: 0

this call returns file id 2691421

find /vz/private/X/ -inum 2691421 -exec ls -l {} \;
-rw-r--r-- 1 root root 1128 Jul 14 2011 /vz/private/X/fs/root/etc/passwd

Now after a minute i do see the same call again to same file handle.

294 2014-01-15 16:07:01.771152 10.X.0.X 10.X.0.X NFS
190 V3 GETATTR Call (Reply In 295), FH: 0x496fe5c5
295 2014-01-15 16:07:01.771261 10.X.0.X 10.X.0.X NFS
182 V3 GETATTR Reply (Call In 294) Regular File mode: 0644 uid: 0
gid: 0

mount options
# 10.X.0.X:/vol/vz2/private on /vz/private type nfs
(rw,noatime,nodiratime,soft,timeo=60,nfsvers=3,retrans=2,rsize=8192,wsize=8192,actimeo=300,addr=10.X.0.X)

why does this happens with actimeo=300 . The second call in 1 minute
should be answered from cache is it ?. Can some tuning be done to
reduce this?.

Regards
Ranjith Ruban


2014-01-16 15:55:14

by Chuck Lever III

[permalink] [raw]
Subject: Re: nfs attribute cache question


On Jan 16, 2014, at 6:31 AM, ranjith ruban <[email protected]> wrote:

> Hello
>
> I had set nfs mount actiomeo option for 300 seconds for a share. The
> client is an openvz server with container filesystems in nfs . Server
> is netapp.
>
> While going through the tcpdump i see below
>
> 125 2014-01-15 16:06:01.770522 10.X.0.X 10.X.0.X NFS
> 190 V3 GETATTR Call (Reply In 126), FH: 0x496fe5c5
> 126 2014-01-15 16:06:01.770637 10.X.0.X 10.X.0.X NFS
> 182 V3 GETATTR Reply (Call In 125) Regular File mode: 0644 uid:
> 0 gid: 0
>
> this call returns file id 2691421
>
> find /vz/private/X/ -inum 2691421 -exec ls -l {} \;
> -rw-r--r-- 1 root root 1128 Jul 14 2011 /vz/private/X/fs/root/etc/passwd
>
> Now after a minute i do see the same call again to same file handle.
>
> 294 2014-01-15 16:07:01.771152 10.X.0.X 10.X.0.X NFS
> 190 V3 GETATTR Call (Reply In 295), FH: 0x496fe5c5
> 295 2014-01-15 16:07:01.771261 10.X.0.X 10.X.0.X NFS
> 182 V3 GETATTR Reply (Call In 294) Regular File mode: 0644 uid: 0
> gid: 0
>
> mount options
> # 10.X.0.X:/vol/vz2/private on /vz/private type nfs
> (rw,noatime,nodiratime,soft,timeo=60,nfsvers=3,retrans=2,rsize=8192,wsize=8192,actimeo=300,addr=10.X.0.X)
>
> why does this happens with actimeo=300 . The second call in 1 minute
> should be answered from cache is it ?. Can some tuning be done to
> reduce this?.

With NFSv3, open(2) on the client results in an on-the-wire GETATTR, independent of the freshness of the file?s attribute cache. See ?DATA AND METADATA COHERENCE? in the nfs(5) man page.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com




2014-01-16 16:37:12

by Trond Myklebust

[permalink] [raw]
Subject: Re: nfs attribute cache question


On Jan 16, 2014, at 10:55, Chuck Lever <[email protected]> wrote:

>
> On Jan 16, 2014, at 6:31 AM, ranjith ruban <[email protected]> wrote:
>
>> Hello
>>
>> I had set nfs mount actiomeo option for 300 seconds for a share. The
>> client is an openvz server with container filesystems in nfs . Server
>> is netapp.
>>
>> While going through the tcpdump i see below
>>
>> 125 2014-01-15 16:06:01.770522 10.X.0.X 10.X.0.X NFS
>> 190 V3 GETATTR Call (Reply In 126), FH: 0x496fe5c5
>> 126 2014-01-15 16:06:01.770637 10.X.0.X 10.X.0.X NFS
>> 182 V3 GETATTR Reply (Call In 125) Regular File mode: 0644 uid:
>> 0 gid: 0
>>
>> this call returns file id 2691421
>>
>> find /vz/private/X/ -inum 2691421 -exec ls -l {} \;
>> -rw-r--r-- 1 root root 1128 Jul 14 2011 /vz/private/X/fs/root/etc/passwd
>>
>> Now after a minute i do see the same call again to same file handle.
>>
>> 294 2014-01-15 16:07:01.771152 10.X.0.X 10.X.0.X NFS
>> 190 V3 GETATTR Call (Reply In 295), FH: 0x496fe5c5
>> 295 2014-01-15 16:07:01.771261 10.X.0.X 10.X.0.X NFS
>> 182 V3 GETATTR Reply (Call In 294) Regular File mode: 0644 uid: 0
>> gid: 0
>>
>> mount options
>> # 10.X.0.X:/vol/vz2/private on /vz/private type nfs
>> (rw,noatime,nodiratime,soft,timeo=60,nfsvers=3,retrans=2,rsize=8192,wsize=8192,actimeo=300,addr=10.X.0.X)
>>
>> why does this happens with actimeo=300 . The second call in 1 minute
>> should be answered from cache is it ?. Can some tuning be done to
>> reduce this?.
>
> With NFSv3, open(2) on the client results in an on-the-wire GETATTR, independent of the freshness of the file?s attribute cache. See ?DATA AND METADATA COHERENCE? in the nfs(5) man page.

We should probably amend the manpage to make it clear that the ac* mount options control timeout values only, and that the NFS client may still end up sending a GETATTR if cache coherency requires it.

--
Trond Myklebust
Linux NFS client maintainer