Hi,
I'm experiencing really big problems with nfs on 2.5 - and I'm a bit
stuck debugging.
Server:
Pentium II SMP Dual Server with Raid5/dm and nfs running 2.5.7[045][-mm]
Clients:
Athlon, same kernels
P3 800, same kernels and 2.4
Problem:
Accessing the nfs shares on the Server gives lots of 'nfs stale file
handles', making it unusuable. A simple cp from nfs to nfs triggers it
in a matter of seconds.
The shares are mounted with (hard,intr), that used to work with 2.4.20
on the server, but I also tried no option, only hard and only soft,
problem persists. Also I tried to remove nfs_directio from the build and
only compiled in nfs2, all the same.
Being curious whats wrong I set up an export on the P3 800 and mounted
it from the athlon (both running 2.5.75-mm1). This seems to work fine
(just tested for 10 minutes or so, but typically the problem is
triggered much earlier).
I also tried enabling the VERBOSE_DEBUG define in nfs source. But that
doesn't give any more information.
Only one line that gets my attention:
NFS: giant filename in readdir (len 0x2f0a0969)
I'm really lost here. What can I try/do to further narrow this down? Any
specific kernel revision I could try to go back, notice that already
2.5.70 triggered it. With 2.4 on the server nothing of this happens.
Only thing left is to try booting the server without smp support, but I
get some 'hde: lost interrupt' messages and it doesn't boot.
Note that I also tried to export a partition not on dm. Filesystem is
ext3. I also tried the patches you posted some days ago in another thread.
Thanks for any suggestions,
Jan
# grep NFS .config
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
--
Linux rubicon 2.5.75-mm1-jd10 #1 SMP Sat Jul 12 19:40:28 CEST 2003 i686
On Sat, 12 Jul 2003 13:05:13 -0700
[email protected] (Jan Dittmer) wrote:
> I'm experiencing really big problems with nfs on 2.5 - and I'm a bit
> stuck debugging.
I'm going to throw in a 'me too' on this one also. I haven't done
much to narrow it down because at first I thought I fixed it by using
the same kernel version on both client and server. Once the stale
file handle errors started showing up again I noticed that Trond
mentioned on LK the problem with soft versus hard. So I figured some
changes in the NFS code finally caused soft mounts to not work so
well.
I changed over to hard,intr but the stale file handles have shown up
again. It looks like it might be something that shows up over time
because it works fine for awhile after reboots but eventually
starts giving errors.
UP for both, 2.5.75-mm1
I'm willing to help debug also.
> # grep NFS .config
> CONFIG_NFS_FS=m
> CONFIG_NFS_V3=y
> CONFIG_NFS_V4=y
> CONFIG_NFS_DIRECTIO=y
> CONFIG_NFSD=m
> CONFIG_NFSD_V3=y
> CONFIG_NFSD_V4=y
> CONFIG_NFSD_TCP=y
Just in case this will help: (server and client)
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V4 is not set
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V4 is not set
# CONFIG_NFSD_TCP is not set
--
Jeff Lightfoot -- [email protected] -- http://thefoots.com/
"I see the light at the end of the tunnel now ... someone please
tell me it's not a train" -- Cracker
On Saturday July 12, [email protected] wrote:
> Hi,
>
> I'm experiencing really big problems with nfs on 2.5 - and I'm a bit
> stuck debugging.
>
> Server:
> Pentium II SMP Dual Server with Raid5/dm and nfs running 2.5.7[045][-mm]
>
> Clients:
> Athlon, same kernels
> P3 800, same kernels and 2.4
>
> Problem:
> Accessing the nfs shares on the Server gives lots of 'nfs stale file
> handles', making it unusuable. A simple cp from nfs to nfs triggers it
> in a matter of seconds.
> The shares are mounted with (hard,intr), that used to work with 2.4.20
> on the server, but I also tried no option, only hard and only soft,
> problem persists. Also I tried to remove nfs_directio from the build and
> only compiled in nfs2, all the same.
> Being curious whats wrong I set up an export on the P3 800 and mounted
> it from the athlon (both running 2.5.75-mm1). This seems to work fine
> (just tested for 10 minutes or so, but typically the problem is
> triggered much earlier).
> I also tried enabling the VERBOSE_DEBUG define in nfs source. But that
> doesn't give any more information.
> Only one line that gets my attention:
> NFS: giant filename in readdir (len 0x2f0a0969)
This makes me a bit suspicious of hardware, probably networking. It
really looks like data is getting corrupted between client and server.
The fact that two different servers behaved differently while both
running the same kernel, sees to support the hardware theory.
Maybe if you could get a tcpdump (-s 1500 port 2049) on both the server and the
client I could have a look at the filehandles as see if I can see why
they are 'stale', and whether it could be a hardware problem.
NeilBrown
>
> I'm really lost here. What can I try/do to further narrow this down? Any
> specific kernel revision I could try to go back, notice that already
> 2.5.70 triggered it. With 2.4 on the server nothing of this happens.
> Only thing left is to try booting the server without smp support, but I
> get some 'hde: lost interrupt' messages and it doesn't boot.
> Note that I also tried to export a partition not on dm. Filesystem is
> ext3. I also tried the patches you posted some days ago in another thread.
>
> Thanks for any suggestions,
>
> Jan
>
> # grep NFS .config
> CONFIG_NFS_FS=m
> CONFIG_NFS_V3=y
> CONFIG_NFS_V4=y
> CONFIG_NFS_DIRECTIO=y
> CONFIG_NFSD=m
> CONFIG_NFSD_V3=y
> CONFIG_NFSD_V4=y
> CONFIG_NFSD_TCP=y
>
>
> --
> Linux rubicon 2.5.75-mm1-jd10 #1 SMP Sat Jul 12 19:40:28 CEST 2003 i686
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
15:07:41.511454 192.168.1.90.3385710396 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.511744 sfhq.starfleet.nfs > 192.168.1.90.3385710396: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x00000094f (DF)
15:07:41.521710 192.168.1.90.3402487612 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.522012 sfhq.starfleet.nfs > 192.168.1.90.3402487612: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000002e62 (DF)
15:07:41.522225 192.168.1.90.3419264828 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:41.522448 192.168.1.90.3436042044 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.522511 sfhq.starfleet.nfs > 192.168.1.90.3419264828: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:41.522672 192.168.1.90.3452819260 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.522679 sfhq.starfleet.nfs > 192.168.1.90.3436042044: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000002263 (DF)
15:07:41.523198 sfhq.starfleet.nfs > 192.168.1.90.3452819260: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000002897 (DF)
15:07:41.526192 192.168.1.90.3469596476 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.526490 sfhq.starfleet.nfs > 192.168.1.90.3469596476: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000007bd (DF)
15:07:41.534430 192.168.1.90.3486373692 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.534693 192.168.1.90.3503150908 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.534716 sfhq.starfleet.nfs > 192.168.1.90.3486373692: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000000498 (DF)
15:07:41.534883 sfhq.starfleet.nfs > 192.168.1.90.3503150908: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000008f7 (DF)
15:07:41.535037 192.168.1.90.3519928124 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.535397 sfhq.starfleet.nfs > 192.168.1.90.3519928124: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000002dc (DF)
15:07:41.576124 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350208 win 63712 <nop,nop,timestamp 12540625 230724582> (DF)
15:07:41.623818 192.168.1.90.3536705340 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.624111 sfhq.starfleet.nfs > 192.168.1.90.3536705340: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000000326 (DF)
15:07:41.624269 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350324 win 63712 <nop,nop,timestamp 12540674 230724670> (DF)
15:07:41.727778 192.168.1.90.3553482556 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.728089 sfhq.starfleet.nfs > 192.168.1.90.3553482556: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000005f4 (DF)
15:07:41.728246 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350440 win 63712 <nop,nop,timestamp 12540778 230724774> (DF)
15:07:41.833783 192.168.1.90.3570259772 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:41.835488 sfhq.starfleet.nfs > 192.168.1.90.3570259772: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:41.835655 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350472 win 63712 <nop,nop,timestamp 12540885 230724882> (DF)
15:07:41.937094 192.168.1.90.3587036988 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:41.937521 sfhq.starfleet.nfs > 192.168.1.90.3587036988: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:41.937673 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350504 win 63712 <nop,nop,timestamp 12540987 230724984> (DF)
15:07:42.042109 192.168.1.90.3603814204 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.043540 sfhq.starfleet.nfs > 192.168.1.90.3603814204: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.043721 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224350540 win 63712 <nop,nop,timestamp 12541093 230725090> (DF)
15:07:42.074707 192.168.1.90.3620591420 > sfhq.starfleet.nfs: 148 read fh Unknown/1 4096 bytes @ 0x000000000 (DF)
15:07:42.074962 192.168.1.90.3637368636 > sfhq.starfleet.nfs: 148 lookup fh Unknown/1 ".sch_generic.o.d" (DF)
15:07:42.075133 sfhq.starfleet.nfs > 192.168.1.90.3620591420: reply ok 36 read ERROR: Stale NFS file handle (DF)
15:07:42.075270 192.168.1.90.3654145852 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.075366 192.168.1.90.3670923068 > sfhq.starfleet.nfs: 148 read fh Unknown/1 4096 bytes @ 0x000000000 (DF)
15:07:42.075844 sfhq.starfleet.nfs > 192.168.1.90.3637368636: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.076119 sfhq.starfleet.nfs > 192.168.1.90.3654145852: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.080215 192.168.1.90.3687700284 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.080358 sfhq.starfleet.nfs > 192.168.1.90.3670923068: reply ok 36 read ERROR: Stale NFS file handle (DF)
15:07:42.080571 sfhq.starfleet.nfs > 192.168.1.90.3687700284: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:42.080847 192.168.1.90.3704477500 > sfhq.starfleet.nfs: 912 write fh Unknown/1 754 bytes @ 0x000001b9c (DF)
15:07:42.081215 sfhq.starfleet.nfs > 192.168.1.90.3704477500: reply ok 40 write ERROR: Stale NFS file handle (DF)
15:07:42.083373 192.168.1.90.3721254716 > sfhq.starfleet.nfs: 1448 write fh Unknown/1 4096 bytes @ 0x000000040 (DF)
15:07:42.083499 192.168.1.90.4294937437 > sfhq.starfleet.nfs: 1448 proc-3957788427 (DF)
15:07:42.083608 sfhq.starfleet.2049 > 192.168.1.90.798: . ack 307628725 win 63712 <nop,nop,timestamp 230725130 12541133> (DF)
15:07:42.083636 192.168.1.90.3875489300 > sfhq.starfleet.nfs: 1356 proc-2201617012 (DF)
15:07:42.084086 sfhq.starfleet.nfs > 192.168.1.90.3721254716: reply ok 140 write [|nfs] (DF)
15:07:42.084364 192.168.1.90.3738031932 > sfhq.starfleet.nfs: 148 commit fh Unknown/1 4096 bytes @ 0x000000040 (DF)
15:07:42.084880 sfhq.starfleet.nfs > 192.168.1.90.3738031932: reply ok 40 commit ERROR: Stale NFS file handle (DF)
15:07:42.085265 192.168.1.90.3754809148 > sfhq.starfleet.nfs: 152 remove fh Unknown/1 ".tmp_sch_generic.o" (DF)
15:07:42.085697 sfhq.starfleet.nfs > 192.168.1.90.3754809148: reply ok 124 remove (DF)
15:07:42.096810 192.168.1.90.3771586364 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0000 (DF)
15:07:42.097106 192.168.1.90.3788363580 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "tcp_diag.o" (DF)
15:07:42.097304 sfhq.starfleet.nfs > 192.168.1.90.3771586364: reply ok 124 access c 0000 (DF)
15:07:42.097830 sfhq.starfleet.nfs > 192.168.1.90.3788363580: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.099906 192.168.1.90.3805140796 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "tcp_diag.c" (DF)
15:07:42.100143 sfhq.starfleet.nfs > 192.168.1.90.3805140796: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.100445 192.168.1.90.3821918012 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.100664 sfhq.starfleet.nfs > 192.168.1.90.3821918012: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.100874 192.168.1.90.3838695228 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "tcp_diag.c" (DF)
15:07:42.101088 sfhq.starfleet.nfs > 192.168.1.90.3838695228: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.104453 192.168.1.90.3855472444 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.104670 sfhq.starfleet.nfs > 192.168.1.90.3855472444: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.107106 192.168.1.90.3872249660 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.107330 sfhq.starfleet.nfs > 192.168.1.90.3872249660: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.107430 192.168.1.90.3889026876 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.107636 sfhq.starfleet.nfs > 192.168.1.90.3889026876: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.109834 192.168.1.90.3905804092 > sfhq.starfleet.nfs: 152 lookup fh Unknown/1 ".tmp_tcp_minisocks.o" (DF)
15:07:42.109994 192.168.1.90.3922581308 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "mach-default" (DF)
15:07:42.110075 sfhq.starfleet.nfs > 192.168.1.90.3905804092: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.110257 sfhq.starfleet.nfs > 192.168.1.90.3922581308: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.110310 192.168.1.90.3939358524 > sfhq.starfleet.nfs: 152 lookup fh Unknown/1 ".tmp_tcp_minisocks.o" (DF)
15:07:42.110518 sfhq.starfleet.nfs > 192.168.1.90.3939358524: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.118187 192.168.1.90.3956135740 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.118419 sfhq.starfleet.nfs > 192.168.1.90.3956135740: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000078bf (DF)
15:07:42.119631 192.168.1.90.3972912956 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.119762 192.168.1.90.3989690172 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.120183 sfhq.starfleet.nfs > 192.168.1.90.3972912956: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.120351 sfhq.starfleet.nfs > 192.168.1.90.3989690172: reply ok 124 access c 0002 (DF)
15:07:42.121037 192.168.1.90.4006467388 > sfhq.starfleet.nfs: 152 lookup fh Unknown/1 ".tcp_minisocks.o.d" (DF)
15:07:42.121168 192.168.1.90.4023244604 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.122442 sfhq.starfleet.nfs > 192.168.1.90.4006467388: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.122703 sfhq.starfleet.nfs > 192.168.1.90.4023244604: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:42.123787 192.168.1.90.4040021820 > sfhq.starfleet.nfs: 152 lookup fh Unknown/1 ".tmp_tcp_minisocks.o" (DF)
15:07:42.124165 sfhq.starfleet.nfs > 192.168.1.90.4040021820: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.125933 192.168.1.90.4056799036 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.126314 sfhq.starfleet.nfs > 192.168.1.90.4056799036: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.126533 192.168.1.90.4073576252 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.126848 sfhq.starfleet.nfs > 192.168.1.90.4073576252: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.127609 192.168.1.90.4090353468 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.127993 sfhq.starfleet.nfs > 192.168.1.90.4090353468: reply ok 124 access c 0002 (DF)
15:07:42.128171 192.168.1.90.4107130684 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "timer.h" (DF)
15:07:42.128545 sfhq.starfleet.nfs > 192.168.1.90.4107130684: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.130263 192.168.1.90.4123907900 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "aio.h" (DF)
15:07:42.130498 sfhq.starfleet.nfs > 192.168.1.90.4123907900: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.130808 192.168.1.90.4140685116 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "workqueue.h" (DF)
15:07:42.131046 sfhq.starfleet.nfs > 192.168.1.90.4140685116: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.131904 192.168.1.90.4157462332 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "aio_abi.h" (DF)
15:07:42.132135 sfhq.starfleet.nfs > 192.168.1.90.4157462332: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.137453 192.168.1.90.4174239548 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.137662 sfhq.starfleet.nfs > 192.168.1.90.4174239548: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.137857 192.168.1.90.4191016764 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.138753 sfhq.starfleet.nfs > 192.168.1.90.4191016764: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.138924 192.168.1.90.4207793980 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.139323 sfhq.starfleet.nfs > 192.168.1.90.4207793980: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.139460 192.168.1.90.4224571196 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "current.h" (DF)
15:07:42.139747 sfhq.starfleet.nfs > 192.168.1.90.4224571196: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.140468 192.168.1.90.4241348412 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.140751 sfhq.starfleet.nfs > 192.168.1.90.4241348412: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.140912 192.168.1.90.4258125628 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.141194 sfhq.starfleet.nfs > 192.168.1.90.4258125628: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.167394 192.168.1.90.4274902844 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.167813 sfhq.starfleet.nfs > 192.168.1.90.4274902844: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.168025 192.168.1.90.4291680060 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.168322 sfhq.starfleet.nfs > 192.168.1.90.4291680060: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.168499 192.168.1.90.13555516 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.168776 sfhq.starfleet.nfs > 192.168.1.90.13555516: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.168930 192.168.1.90.30332732 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.169210 sfhq.starfleet.nfs > 192.168.1.90.30332732: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.170933 192.168.1.90.47109948 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.171253 sfhq.starfleet.nfs > 192.168.1.90.47109948: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.171453 192.168.1.90.63887164 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.171733 sfhq.starfleet.nfs > 192.168.1.90.63887164: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.171899 192.168.1.90.80664380 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.172177 sfhq.starfleet.nfs > 192.168.1.90.80664380: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.172325 192.168.1.90.97441596 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.172606 sfhq.starfleet.nfs > 192.168.1.90.97441596: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.173343 192.168.1.90.114218812 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.173622 sfhq.starfleet.nfs > 192.168.1.90.114218812: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.173839 192.168.1.90.130996028 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.174229 sfhq.starfleet.nfs > 192.168.1.90.130996028: reply ok 124 access c 0002 (DF)
15:07:42.178548 192.168.1.90.147773244 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.178771 sfhq.starfleet.nfs > 192.168.1.90.147773244: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.178975 192.168.1.90.164550460 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "mmzone.h" (DF)
15:07:42.179270 sfhq.starfleet.nfs > 192.168.1.90.164550460: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.179461 192.168.1.90.181327676 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.179746 sfhq.starfleet.nfs > 192.168.1.90.181327676: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.180682 192.168.1.90.198104892 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "fs.h" (DF)
15:07:42.180969 sfhq.starfleet.nfs > 192.168.1.90.198104892: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.181171 192.168.1.90.214882108 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.181469 sfhq.starfleet.nfs > 192.168.1.90.214882108: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.182362 192.168.1.90.231659324 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "asm" (DF)
15:07:42.182843 sfhq.starfleet.nfs > 192.168.1.90.231659324: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.183049 192.168.1.90.248436540 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.183309 sfhq.starfleet.nfs > 192.168.1.90.248436540: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.183495 192.168.1.90.265213756 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.183701 sfhq.starfleet.nfs > 192.168.1.90.265213756: reply ok 124 access c 0002 (DF)
15:07:42.183849 192.168.1.90.281990972 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "mach-default" (DF)
15:07:42.184141 sfhq.starfleet.nfs > 192.168.1.90.281990972: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.188122 192.168.1.90.298768188 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "page-flags.h" (DF)
15:07:42.188413 sfhq.starfleet.nfs > 192.168.1.90.298768188: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.188617 192.168.1.90.315545404 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "mach-default" (DF)
15:07:42.188829 sfhq.starfleet.nfs > 192.168.1.90.315545404: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.189016 192.168.1.90.332322620 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.189221 sfhq.starfleet.nfs > 192.168.1.90.332322620: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.207301 192.168.1.90.349099836 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "highmem.h" (DF)
15:07:42.207565 sfhq.starfleet.nfs > 192.168.1.90.349099836: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.209149 192.168.1.90.365877052 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "asm" (DF)
15:07:42.209484 sfhq.starfleet.nfs > 192.168.1.90.365877052: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.209708 192.168.1.90.382654268 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.209991 sfhq.starfleet.nfs > 192.168.1.90.382654268: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.210228 192.168.1.90.399431484 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "cacheflush.h" (DF)
15:07:42.210530 sfhq.starfleet.nfs > 192.168.1.90.399431484: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.212823 192.168.1.90.416208700 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "poll.h" (DF)
15:07:42.213069 sfhq.starfleet.nfs > 192.168.1.90.416208700: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.213394 192.168.1.90.432985916 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.213621 sfhq.starfleet.nfs > 192.168.1.90.432985916: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.213774 192.168.1.90.449763132 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "poll.h" (DF)
15:07:42.214012 sfhq.starfleet.nfs > 192.168.1.90.449763132: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.214474 192.168.1.90.466540348 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.214701 sfhq.starfleet.nfs > 192.168.1.90.466540348: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.214869 192.168.1.90.483317564 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "uaccess.h" (DF)
15:07:42.215112 sfhq.starfleet.nfs > 192.168.1.90.483317564: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.226051 192.168.1.90.500094780 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "net.h" (DF)
15:07:42.226356 sfhq.starfleet.nfs > 192.168.1.90.500094780: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.229408 192.168.1.90.516871996 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "slab.h" (DF)
15:07:42.229702 sfhq.starfleet.nfs > 192.168.1.90.516871996: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.232034 192.168.1.90.533649212 > sfhq.starfleet.nfs: 148 lookup fh Unknown/1 "kmalloc_sizes.h" (DF)
15:07:42.232338 sfhq.starfleet.nfs > 192.168.1.90.533649212: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.248272 192.168.1.90.550426428 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "net" (DF)
15:07:42.248580 sfhq.starfleet.nfs > 192.168.1.90.550426428: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.248836 192.168.1.90.567203644 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.249254 sfhq.starfleet.nfs > 192.168.1.90.567203644: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000007b86 (DF)
15:07:42.249532 192.168.1.90.583980860 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "module.h" (DF)
15:07:42.249825 sfhq.starfleet.nfs > 192.168.1.90.583980860: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.250075 192.168.1.90.600758076 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "stat.h" (DF)
15:07:42.250374 sfhq.starfleet.nfs > 192.168.1.90.600758076: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.250738 192.168.1.90.617535292 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.251018 sfhq.starfleet.nfs > 192.168.1.90.617535292: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.251175 192.168.1.90.634312508 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "stat.h" (DF)
15:07:42.251477 sfhq.starfleet.nfs > 192.168.1.90.634312508: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.252618 192.168.1.90.651089724 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "kmod.h" (DF)
15:07:42.252910 sfhq.starfleet.nfs > 192.168.1.90.651089724: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.254779 192.168.1.90.667866940 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "elf.h" (DF)
15:07:42.255073 sfhq.starfleet.nfs > 192.168.1.90.667866940: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.255451 192.168.1.90.684644156 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.255736 sfhq.starfleet.nfs > 192.168.1.90.684644156: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.255896 192.168.1.90.701421372 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "elf.h" (DF)
15:07:42.256197 sfhq.starfleet.nfs > 192.168.1.90.701421372: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.256672 192.168.1.90.718198588 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.256960 sfhq.starfleet.nfs > 192.168.1.90.718198588: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.258695 192.168.1.90.734975804 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "user.h" (DF)
15:07:42.258969 sfhq.starfleet.nfs > 192.168.1.90.734975804: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.259197 192.168.1.90.751753020 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "asm" (DF)
15:07:42.259765 sfhq.starfleet.nfs > 192.168.1.90.751753020: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.261460 192.168.1.90.768530236 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "utsname.h" (DF)
15:07:42.261681 sfhq.starfleet.nfs > 192.168.1.90.768530236: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.261893 192.168.1.90.785307452 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.262118 sfhq.starfleet.nfs > 192.168.1.90.785307452: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.266156 192.168.1.90.802084668 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.266471 sfhq.starfleet.nfs > 192.168.1.90.802084668: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.266679 192.168.1.90.818861884 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.266961 sfhq.starfleet.nfs > 192.168.1.90.818861884: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.267113 192.168.1.90.835639100 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.267402 sfhq.starfleet.nfs > 192.168.1.90.835639100: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.267565 192.168.1.90.852416316 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "asm-i386" (DF)
15:07:42.267855 sfhq.starfleet.nfs > 192.168.1.90.852416316: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.269078 192.168.1.90.869193532 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.269364 sfhq.starfleet.nfs > 192.168.1.90.869193532: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.276421 192.168.1.90.885970748 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "netdevice.h" (DF)
15:07:42.276709 sfhq.starfleet.nfs > 192.168.1.90.885970748: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.276915 192.168.1.90.902747964 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.277209 sfhq.starfleet.nfs > 192.168.1.90.902747964: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.277541 192.168.1.90.919525180 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "security.h" (DF)
15:07:42.277825 sfhq.starfleet.nfs > 192.168.1.90.919525180: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.279268 192.168.1.90.936302396 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.279554 sfhq.starfleet.nfs > 192.168.1.90.936302396: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.280097 192.168.1.90.953079612 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "filter.h" (DF)
15:07:42.280400 sfhq.starfleet.nfs > 192.168.1.90.953079612: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.283954 192.168.1.90.969856828 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.284586 sfhq.starfleet.nfs > 192.168.1.90.969856828: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.284799 192.168.1.90.986634044 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.285084 sfhq.starfleet.nfs > 192.168.1.90.986634044: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.318521 192.168.1.90.1003411260 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.319127 sfhq.starfleet.nfs > 192.168.1.90.1003411260: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.319326 192.168.1.90.1020188476 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.319611 sfhq.starfleet.nfs > 192.168.1.90.1020188476: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.320783 192.168.1.90.1036965692 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.321345 sfhq.starfleet.nfs > 192.168.1.90.1036965692: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.321536 192.168.1.90.1053742908 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.321820 sfhq.starfleet.nfs > 192.168.1.90.1053742908: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.322541 192.168.1.90.1070520124 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "ip.h" (DF)
15:07:42.322827 sfhq.starfleet.nfs > 192.168.1.90.1070520124: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.323031 192.168.1.90.1087297340 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.323321 sfhq.starfleet.nfs > 192.168.1.90.1087297340: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.327240 192.168.1.90.1104074556 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.327813 sfhq.starfleet.nfs > 192.168.1.90.1104074556: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.328028 192.168.1.90.1120851772 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:42.328324 sfhq.starfleet.nfs > 192.168.1.90.1120851772: reply ok 124 access c 0002 (DF)
15:07:42.328475 192.168.1.90.1137628988 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "mach-default" (DF)
15:07:42.328755 sfhq.starfleet.nfs > 192.168.1.90.1137628988: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.329562 192.168.1.90.1154406204 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.329844 sfhq.starfleet.nfs > 192.168.1.90.1154406204: reply ok 124 access c 0001 (DF)
15:07:42.330201 192.168.1.90.1171183420 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "ipv6.h" (DF)
15:07:42.330488 sfhq.starfleet.nfs > 192.168.1.90.1171183420: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.330653 192.168.1.90.1187960636 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "mach-default" (DF)
15:07:42.330948 sfhq.starfleet.nfs > 192.168.1.90.1187960636: reply ok 232 lookup fh Unknown/1 (DF)
15:07:42.331127 192.168.1.90.1204737852 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.331399 sfhq.starfleet.nfs > 192.168.1.90.1204737852: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.332553 192.168.1.90.1221515068 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:42.332829 sfhq.starfleet.nfs > 192.168.1.90.1221515068: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:42.333034 192.168.1.90.1238292284 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "hardirq.h" (DF)
15:07:42.333348 sfhq.starfleet.nfs > 192.168.1.90.1238292284: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.333515 192.168.1.90.1255069500 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "asm" (DF)
15:07:42.333826 sfhq.starfleet.nfs > 192.168.1.90.1255069500: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:42.334646 192.168.1.90.1271846716 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.335213 sfhq.starfleet.nfs > 192.168.1.90.1271846716: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.335414 192.168.1.90.1288623932 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "net" (DF)
15:07:42.335725 sfhq.starfleet.nfs > 192.168.1.90.1288623932: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:42.336509 192.168.1.90.1305401148 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.336740 sfhq.starfleet.nfs > 192.168.1.90.1305401148: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x0000006f4 (DF)
15:07:42.337067 192.168.1.90.1322178364 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "in6.h" (DF)
15:07:42.337294 sfhq.starfleet.nfs > 192.168.1.90.1322178364: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.349785 192.168.1.90.1338955580 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.350042 sfhq.starfleet.nfs > 192.168.1.90.1338955580: reply ok 116 getattr REG 100644 ids 1000/1000 sz 0x000000745 (DF)
15:07:42.350906 192.168.1.90.1355732796 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.351132 sfhq.starfleet.nfs > 192.168.1.90.1355732796: reply ok 124 access c 0001 (DF)
15:07:42.351386 192.168.1.90.1372510012 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "xfrm.h" (DF)
15:07:42.352450 sfhq.starfleet.nfs > 192.168.1.90.1372510012: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.352637 192.168.1.90.1389287228 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.352854 sfhq.starfleet.nfs > 192.168.1.90.1389287228: reply ok 124 access c 0001 (DF)
15:07:42.355403 192.168.1.90.1406064444 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "crypto.h" (DF)
15:07:42.355619 sfhq.starfleet.nfs > 192.168.1.90.1406064444: reply ok 240 lookup fh Unknown/1 (DF)
15:07:42.355844 192.168.1.90.1422841660 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.357265 sfhq.starfleet.nfs > 192.168.1.90.1422841660: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.357440 192.168.1.90.1439618876 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.357730 sfhq.starfleet.nfs > 192.168.1.90.1439618876: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.358482 192.168.1.90.1456396092 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "pfkeyv2.h" (DF)
15:07:42.358767 sfhq.starfleet.nfs > 192.168.1.90.1456396092: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.358963 192.168.1.90.1473173308 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.359256 sfhq.starfleet.nfs > 192.168.1.90.1473173308: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.360348 192.168.1.90.1489950524 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:42.360916 sfhq.starfleet.nfs > 192.168.1.90.1489950524: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:42.361796 192.168.1.90.1506727740 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0001 (DF)
15:07:42.362361 sfhq.starfleet.nfs > 192.168.1.90.1506727740: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:42.394732 192.168.1.90.1523504956 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "inet.h" (DF)
15:07:42.395058 sfhq.starfleet.nfs > 192.168.1.90.1523504956: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.395268 192.168.1.90.1540282172 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.395555 sfhq.starfleet.nfs > 192.168.1.90.1540282172: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.396889 192.168.1.90.1557059388 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "proc_fs.h" (DF)
15:07:42.397186 sfhq.starfleet.nfs > 192.168.1.90.1557059388: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.397384 192.168.1.90.1573836604 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.397702 sfhq.starfleet.nfs > 192.168.1.90.1573836604: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.398457 192.168.1.90.1590613820 > sfhq.starfleet.nfs: 144 lookup fh Unknown/1 "seq_file.h" (DF)
15:07:42.398743 sfhq.starfleet.nfs > 192.168.1.90.1590613820: reply ok 36 lookup ERROR: Stale NFS file handle (DF)
15:07:42.398944 192.168.1.90.1607391036 > sfhq.starfleet.nfs: 140 lookup fh Unknown/1 "linux" (DF)
15:07:42.399253 sfhq.starfleet.nfs > 192.168.1.90.1607391036: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:42.439772 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224364368 win 63712 <nop,nop,timestamp 12541489 230725446> (DF)
15:07:43.076736 192.168.1.90.1624168252 > sfhq.starfleet.nfs: 256 write fh Unknown/1 98 bytes @ 0x000000040 (DF)
15:07:43.076950 192.168.1.90.1640945468 > sfhq.starfleet.nfs: 148 lookup fh Unknown/1 ".tcp_ipv4.o.d" (DF)
15:07:43.077751 sfhq.starfleet.nfs > 192.168.1.90.1640945468: reply ok 120 lookup ERROR: No such file or directory (DF)
15:07:43.077926 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224364488 win 63712 <nop,nop,timestamp 12542128 230726124> (DF)
15:07:43.077996 192.168.1.90.1657722684 > sfhq.starfleet.nfs: 180 create fh Unknown/1 ".tcp_ipv4.o.d" (DF)
15:07:43.078075 sfhq.starfleet.nfs > 192.168.1.90.1624168252: reply ok 40 write ERROR: Stale NFS file handle (DF)
15:07:43.078245 192.168.1.90.1674499900 > sfhq.starfleet.nfs: 208 write fh Unknown/1 52 bytes @ 0x000000000 (DF)
15:07:43.078551 sfhq.starfleet.nfs > 192.168.1.90.1657722684: reply ok 272 create fh Unknown/1 (DF)
15:07:43.078753 192.168.1.90.1691277116 > sfhq.starfleet.nfs: 140 access fh Unknown/1 0000 (DF)
15:07:43.078941 sfhq.starfleet.nfs > 192.168.1.90.1674499900: reply ok 40 write ERROR: Stale NFS file handle (DF)
15:07:43.079081 192.168.1.90.1708054332 > sfhq.starfleet.nfs: 148 read fh Unknown/1 4096 bytes @ 0x000000000 (DF)
15:07:43.079483 sfhq.starfleet.nfs > 192.168.1.90.1691277116: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:43.079942 sfhq.starfleet.nfs > 192.168.1.90.1708054332: reply ok 36 read ERROR: Stale NFS file handle (DF)
15:07:43.084753 192.168.1.90.1724831548 > sfhq.starfleet.nfs: 148 remove fh Unknown/1 ".tmp_tcp_ipv4.o" (DF)
15:07:43.084987 sfhq.starfleet.nfs > 192.168.1.90.1724831548: reply ok 40 remove ERROR: Stale NFS file handle (DF)
15:07:43.087044 192.168.1.90.1741608764 > sfhq.starfleet.nfs: 124 access fh Unknown/1 0002 (DF)
15:07:43.087285 sfhq.starfleet.nfs > 192.168.1.90.1741608764: reply ok 124 access c 0002 (DF)
15:07:43.116026 192.168.1.90.1758385980 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:43.116269 sfhq.starfleet.nfs > 192.168.1.90.1758385980: reply ok 116 getattr DIR 40755 ids 1000/1000 sz 0x000001000 (DF)
15:07:43.116484 192.168.1.90.1775163196 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:43.116780 sfhq.starfleet.nfs > 192.168.1.90.1775163196: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:43.117466 192.168.1.90.1791940412 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:43.117747 sfhq.starfleet.nfs > 192.168.1.90.1791940412: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:43.117949 192.168.1.90.1808717628 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:43.118233 sfhq.starfleet.nfs > 192.168.1.90.1808717628: reply ok 32 getattr ERROR: Stale NFS file handle (DF)
15:07:43.118401 192.168.1.90.1825494844 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:43.118682 sfhq.starfleet.nfs > 192.168.1.90.1825494844: reply ok 36 access ERROR: Stale NFS file handle (DF)
15:07:43.150516 192.168.1.90.1842272060 > sfhq.starfleet.nfs: 128 getattr fh Unknown/1 (DF)
15:07:43.150824 sfhq.starfleet.nfs > 192.168.1.90.1842272060: reply ok 116 getattr DIR 40700 ids 1000/1000 sz 0x000001000 (DF)
15:07:43.151013 192.168.1.90.1859049276 > sfhq.starfleet.nfs: 136 lookup fh Unknown/1 "2.5" (DF)
15:07:43.151440 sfhq.starfleet.nfs > 192.168.1.90.1859049276: reply ok 232 lookup fh Unknown/1 (DF)
15:07:43.151635 192.168.1.90.1875826492 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:43.152238 sfhq.starfleet.nfs > 192.168.1.90.1875826492: reply ok 124 access c 0002 (DF)
15:07:43.152386 192.168.1.90.1892603708 > sfhq.starfleet.nfs: 148 lookup fh Unknown/1 "2.6.0-test1-jd3" (DF)
15:07:43.152674 sfhq.starfleet.nfs > 192.168.1.90.1892603708: reply ok 232 lookup fh Unknown/1 (DF)
15:07:43.153521 192.168.1.90.1909380924 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:43.153801 sfhq.starfleet.nfs > 192.168.1.90.1909380924: reply ok 124 access c 0002 (DF)
15:07:43.155650 192.168.1.90.1926158140 > sfhq.starfleet.nfs: 136 getattr fh Unknown/1 (DF)
15:07:43.155930 sfhq.starfleet.nfs > 192.168.1.90.1926158140: reply ok 116 getattr REG 100755 ids 1000/1000 sz 0x000000135 (DF)
15:07:43.164775 192.168.1.90.1942935356 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0001 (DF)
15:07:43.165084 sfhq.starfleet.nfs > 192.168.1.90.1942935356: reply ok 124 access c 0001 (DF)
15:07:43.165403 192.168.1.90.1959712572 > sfhq.starfleet.nfs: 152 readdirplus fh Unknown/1 512 bytes @ 0x000000000 (DF)
15:07:43.166074 sfhq.starfleet.nfs > 192.168.1.90.1959712572: reply ok 1448 readdirplus (DF)
15:07:43.166168 sfhq.starfleet.nfs > 192.168.1.90.1: reply ERR 1448 (DF)
15:07:43.166258 sfhq.starfleet.nfs > 192.168.1.90.20: reply ERR 1180 (DF)
15:07:43.166493 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224369292 win 63712 <nop,nop,timestamp 12542216 230726213> (DF)
15:07:43.166600 192.168.1.90.1976489788 > sfhq.starfleet.nfs: 152 readdirplus fh Unknown/1 512 bytes @ 0x0570bf97b (DF)
15:07:43.166962 sfhq.starfleet.nfs > 192.168.1.90.1976489788: reply ok 1448 readdirplus (DF)
15:07:43.167035 sfhq.starfleet.nfs > 192.168.1.90.0: reply ok 104 (DF)
15:07:43.167215 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224372024 win 63712 <nop,nop,timestamp 12542217 230726214> (DF)
15:07:43.168121 192.168.1.90.1993267004 > sfhq.starfleet.nfs: 132 access fh Unknown/1 0002 (DF)
15:07:43.168378 sfhq.starfleet.nfs > 192.168.1.90.1993267004: reply ok 124 access c 0002 (DF)
15:07:43.208478 192.168.1.90.798 > sfhq.starfleet.2049: . ack 224372148 win 63712 <nop,nop,timestamp 12542258 230726215> (DF)
Neil Brown wrote:
> On Saturday July 12, [email protected] wrote:
>
>>Hi,
>>
>>I'm experiencing really big problems with nfs on 2.5 - and I'm a bit
>>stuck debugging.
>>
>
> This makes me a bit suspicious of hardware, probably networking. It
> really looks like data is getting corrupted between client and server.
>
Ok, to rule out hardware I mounted the filesystem on the same machine I
exported it. So no real network hardware is involved only lo.
I ran nhfsstone (which btw is a great tool to reproduce it, just type nhfsrun)
and got this nearly immediatly:
17:16:32.706611 localhost.3772543495 > localhost.nfs: 148 remove fh Unknown/1
"012abcdefghijklmn" (DF) [ttl 0]
17:16:32.707305 localhost.3789320711 > localhost.nfs: 116 fsstat fh Unknown/1
(DF) [ttl 0]
17:16:32.707602 localhost.3806097927 > localhost.nfs: 136 access fh Unknown/1
000d (DF) [ttl 0]
17:16:32.709633 localhost.3822875143 > localhost.nfs: 116 fsstat fh Unknown/1
(DF) [ttl 0]
17:16:32.709822 localhost.3839652359 > localhost.nfs: 144 commit fh Unknown/1
2048 bytes @ 0x000000000 (DF) [ttl 0]
17:16:32.725623 localhost.3806097927 > localhost.nfs: 136 access fh Unknown/1
000d (DF) [ttl 0]
17:16:32.733637 localhost.nfs > localhost.3772543495: reply ok 120 remove (DF)
[ttl 0]
17:16:32.735074 localhost.nfs > localhost.3789320711: reply ok 84 fsstat
tbytes 0x189b948000 fbytes 0x173616c000 abytes 0x15f616c000 (DF) [ttl
0]
17:16:32.743608 localhost.3806097927 > localhost.nfs: 136 access fh Unknown/1
000d (DF) [ttl 0]
17:16:32.750618 localhost.3839652359 > localhost.nfs: 144 commit fh Unknown/1
2048 bytes @ 0x000000000 (DF) [ttl 0]
17:16:32.756246 localhost.nfs > localhost.3302781447: reply ok 144 setattr
(DF) [ttl 0]
17:16:32.756467 localhost.nfs > localhost.3806097927: reply ok 120 access c
000d (DF) [ttl 0]
17:16:32.759073 localhost.3856429575 > localhost.nfs: 168 setattr fh Unknown/1
(DF) [ttl 0]
17:16:32.759632 localhost.nfs > localhost.3822875143: reply ok 84 fsstat
tbytes 0x189b948000 fbytes 0x1736172000 abytes 0x15f6172000 (DF) [ttl
0]
17:16:32.759815 localhost.3873206791 > localhost.nfs: 148 lookup fh Unknown/1
"012abcdefghijklmn" (DF) [ttl 0]
17:16:32.760182 localhost.nfs > localhost.3839652359: reply ok 128 commit (DF)
[ttl 0]
17:16:32.760574 localhost.3889984007 > localhost.nfs: 148 lookup fh Unknown/1
"032abcdefghijklmn" (DF) [ttl 0]
17:16:32.760703 localhost.nfs > localhost.3806097927: reply ok 120 access c
000d (DF) [ttl 0]
17:16:32.760942 localhost.3906761223 > localhost.nfs: 148 lookup fh Unknown/1
"007abcdefghijklmn" (DF) [ttl 0]
17:16:32.761071 localhost.nfs > localhost.3806097927: reply ok 120 access c
000d (DF) [ttl 0]
17:16:32.769785 localhost.3873206791 > localhost.nfs: 148 lookup fh Unknown/1
"012abcdefghijklmn" (DF) [ttl 0]
17:16:32.778592 localhost.3889984007 > localhost.nfs: 148 lookup fh Unknown/1
"032abcdefghijklmn" (DF) [ttl 0]
17:16:32.797611 localhost.3906761223 > localhost.nfs: 148 lookup fh Unknown/1
"007abcdefghijklmn" (DF) [ttl 0]
17:16:32.830371 localhost.nfs > localhost.3839652359: reply ok 128 commit (DF)
[ttl 0]
17:16:32.831099 localhost.nfs > localhost.3856429575: reply ok 36 setattr
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.831728 localhost.nfs > localhost.3873206791: reply ok 32 lookup
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.832699 localhost.nfs > localhost.3889984007: reply ok 32 lookup
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.832862 localhost.3923538439 > localhost.nfs: 132 getattr fh Unknown/1
(DF) [ttl 0]
17:16:32.833612 localhost.nfs > localhost.3906761223: reply ok 32 lookup
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.834058 localhost.nfs > localhost.3873206791: reply ok 32 lookup
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.834416 localhost.3940315655 > localhost.nfs: 164 setattr fh Unknown/1
(DF) [ttl 0]
17:16:32.834940 localhost.nfs > localhost.3889984007: reply ok 32 lookup
ERROR: Stale NFS file handle (DF) [ttl 0]
17:16:32.835896 localhost.nfs > localhost.3906761223: reply ok 236 lookup fh
Unknown/1 (DF) [ttl 0]
17:16:32.836910 localhost.nfs > localhost.3923538439: reply ok 112 getattr REG
100666 ids 1000/1000 sz 0x000000000 (DF) [ttl 0]
17:16:32.837481 localhost.nfs > localhost.3940315655: reply ok 144 setattr
(DF) [ttl 0]
17:16:32.841279 localhost.3957092871 > localhost.nfs: 132 getattr fh Unknown/1
(DF) [ttl 0]
17:16:32.843429 localhost.nfs > localhost.3957092871: reply ok 112 getattr REG
100666 ids 1000/1000 sz 0x000000000 (DF) [ttl 0]
17:16:32.844561 localhost.3973870087 > localhost.nfs: 132 getattr fh Unknown/1
(DF) [ttl 0]
17:16:32.844918 localhost.3990647303 > localhost.nfs: 148 lookup fh Unknown/1
"012abcdefghijklmn" (DF) [ttl 0]
I'll try to reproduce this on my other (UP) machines.
Thanks,
Jan
--
Linux rubicon 2.6.0-test1-jd2 #1 SMP Mon Jul 14 17:37:41 CEST 2003 i686
Neil Brown wrote:
> On Saturday July 12, [email protected] wrote:
>>Problem:
>>Accessing the nfs shares on the Server gives lots of 'nfs stale file
>>handles', making it unusuable. A simple cp from nfs to nfs triggers it
>>in a matter of seconds.
> This makes me a bit suspicious of hardware, probably networking. It
> really looks like data is getting corrupted between client and server.
>
just as a data point 2.4.22pre6aa1 is working rock solid on the same machine -
not one timeout yet in 2 days testing (an infinite loop of kernel compiles).
Thanks,
Jan
--
Linux rubicon 2.6.0-test1-mm1-jd1 #1 SMP Wed Jul 16 10:14:18 CEST 2003 i686