Hi,everyone. I found a big bug abount nfs cache.
when server delete the file ,nfs client can read file also.
the following is the reproduce.
ip: 192.168.0.19 nfs-client
ip: 192.168.0.20 nfs-client
ip: 192.168.0.21 nfs-server
############################################################################
/usr/bin/ssh -n 192.168.0.21 service nfs start
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot
/usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.19 umount /nfsroot
/usr/bin/ssh -n 192.168.0.20 umount /nfsroot
cmd="echo \"hello world\" > /nfsroot/tmpfile"
/usr/bin/ssh -n 192.168.0.21 $cmd
/usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir
/usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile
/usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot
/usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot
/usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile
/usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir
/usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile
/usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir
/usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile
/usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir
/usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null
/usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null
/usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile
echo -e "sleep 60~~~~~~~~\n"
sleep 60
#############################################################################
last: In 192.168.0.19 I do:
#cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
hello world
I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.
[email protected] 写道:
> On Fri, Nov 23, 2012 at 02:00:13PM +0800, fanchaoting wrote:
>> Myklebust, Trond 写道:
>>> On Fri, 2012-11-23 at 13:23 +0800, fanchaoting wrote:
>>>> Myklebust, Trond 写道:
>>>>> On Fri, 2012-11-23 at 12:18 +0800, fanchaoting wrote:
>>>>>> I think when the nfs server delete the file ,
>>>>>> the server should notice the nfs client,
>>>>>> but the upstream kernel does't this.
>>>>> So is this a problem with the client or the server? In other words, if
>>>>> you use a different server/client combination, do you see a different
>>>>> result?
>>>>>
>>>> I think that the server has the problem .when the server deletes the file ,
>>>> it should notice the client immediately.
>>> There is no notification mechanism in NFS; on open(), the client is
>>> supposed to revalidate its cached information and the server is supposed
>>> to return an ESTALE error if the filehandle is no longer valid. Either
>>> one of these 2 mechanisms (client revalidation or server reply) could be
>>> going wrong here, which is why I'm asking.
>> I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem .
>> But I did't found it in the upstream kernel.
>
> Yes, still working on getting that merged. Next step is probably to
> rebase it on top of jlayton's ESTALE patches, assuming those will go in
> first. In theory they could make 3.8 but that may be optimistic.
>
Yes, when I applyed the patch, this problem was solved.
I found when server deleted the file , it can return
"DELEGRETURN" to the nfs client.
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
T24gRnJpLCAyMDEyLTExLTIzIGF0IDEzOjIzICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4g
TXlrbGVidXN0LCBUcm9uZCDlhpnpgZM6DQo+ID4gT24gRnJpLCAyMDEyLTExLTIzIGF0IDEyOjE4
ICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4gPj4gSGksZXZlcnlvbmUuIEkgZm91bmQgYSBi
aWcgYnVnIGFib3VudCBuZnMgY2FjaGUuDQo+ID4+DQo+ID4+IHdoZW4gc2VydmVyIGRlbGV0ZSB0
aGUgZmlsZSAsbmZzIGNsaWVudCBjYW4gcmVhZCBmaWxlIGFsc28uDQo+ID4+DQo+ID4+IHRoZSBm
b2xsb3dpbmcgaXMgdGhlIHJlcHJvZHVjZS4NCj4gPj4NCj4gPj4gaXA6IDE5Mi4xNjguMC4xOSAg
bmZzLWNsaWVudA0KPiA+PiBpcDogMTkyLjE2OC4wLjIwICBuZnMtY2xpZW50DQo+ID4+IGlwOiAx
OTIuMTY4LjAuMjEgIG5mcy1zZXJ2ZXINCj4gPj4NCj4gPj4gIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIw0K
PiA+Pg0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHNlcnZpY2UgbmZzIHN0YXJ0
DQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmluL3JwYy5pZG1hcGQN
Cj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4xOSAvdXNyL3NiaW4vcnBjLmlkbWFwZA0K
PiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIC91c3Ivc2Jpbi9ycGMuaWRtYXBkDQo+
ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgInJtIC1yZiAvbmZzcm9vdDsgbWtkaXIg
LXAgL25mc3Jvb3QiDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmlu
L2V4cG9ydGZzIC1hdQ0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIC91c3Ivc2Jp
bi9leHBvcnRmcyAtaSAtbyBpbnNlY3VyZSxub19yb290X3NxdWFzaCxydyxmc2lkPTAgKjovbmZz
cm9vdA0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IHRlc3QgLWQgL25mc3Jvb3Qg
fHwgKHJtIC1yZiAvbmZzcm9vdDsgbWtkaXIgLXAgL25mc3Jvb3QpDQo+ID4+IC91c3IvYmluL3Nz
aCAtbiAxOTIuMTY4LjAuMjAgdGVzdCAtZCAvbmZzcm9vdCB8fCAocm0gLXJmIC9uZnNyb290OyBt
a2RpciAtcCAvbmZzcm9vdCkNCj4gPj4NCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4x
OSB1bW91bnQgL25mc3Jvb3QNCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCB1bW91
bnQgL25mc3Jvb3QNCj4gPj4gY21kPSJlY2hvIFwiaGVsbG8gd29ybGRcIiA+IC9uZnNyb290L3Rt
cGZpbGUiDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgJGNtZA0KPiA+PiAvdXNy
L2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIG1rZGlyIC9uZnNyb290L3RtcGRpcg0KPiA+PiAvdXNy
L2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHRvdWNoIC9uZnNyb290L3RtcGRpci90bXBkZmlsZQ0K
PiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IG1vdW50IC10IG5mczQgMTkyLjE2OC4w
LjIxOi8gL25mc3Jvb3QNCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCBtb3VudCAt
dCBuZnM0IDE5Mi4xNjguMC4yMTovIC9uZnNyb290DQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu
MTY4LjAuMjEgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu
MTY4LjAuMjEgbHMgLWwgL25mc3Jvb3QvdG1wZGlyDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu
MTY4LjAuMTkgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu
MTY4LjAuMTkgbHMgLWwgIC9uZnNyb290L3RtcGRpcg0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTky
LjE2OC4wLjIwIGNhdCAgL25mc3Jvb3QvdG1wZmlsZQ0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTky
LjE2OC4wLjIwIGxzIC1sICAvbmZzcm9vdC90bXBkaXINCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5
Mi4xNjguMC4xOSBjYXQgL25mc3Jvb3QvdG1wZmlsZSA+IC9kZXYvbnVsbA0KPiA+PiAvdXNyL2Jp
bi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNhdCAvbmZzcm9vdC90bXBmaWxlID4gL2Rldi9udWxsDQo+
ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgcm0gLXJmIC9uZnNyb290L3RtcGZpbGUN
Cj4gPj4gZWNobyAtZSAic2xlZXAgNjB+fn5+fn5+flxuIg0KPiA+PiBzbGVlcCA2MA0KPiA+Pg0K
PiA+PiAjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIw0KPiA+Pg0KPiA+PiBsYXN0OiBJbiAxOTIuMTY4LjAu
MTkgSSBkbzoNCj4gPj4NCj4gPj4gI2NhdCAvbmZzcm9vdC90bXBmaWxlICAgICAgICAgICA8LS10
aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUsYnV0IG5mcyBjbGllbnQgY2FuIHJlYWQgdGhl
IGZpbGUNCj4gPj4gIGhlbGxvIHdvcmxkDQo+ID4+DQo+ID4+DQo+ID4+IEkgdGhpbmsgIHdoZW4g
dGhlIG5mcyBzZXJ2ZXIgZGVsZXRlIHRoZSBmaWxlICwNCj4gPj4gdGhlICBzZXJ2ZXIgc2hvdWxk
IG5vdGljZSB0aGUgbmZzIGNsaWVudCwNCj4gPj4gYnV0IHRoZSB1cHN0cmVhbSBrZXJuZWwgZG9l
cyd0IHRoaXMuDQo+ID4gDQo+ID4gU28gaXMgdGhpcyBhIHByb2JsZW0gd2l0aCB0aGUgY2xpZW50
IG9yIHRoZSBzZXJ2ZXI/IEluIG90aGVyIHdvcmRzLCBpZg0KPiA+IHlvdSB1c2UgYSBkaWZmZXJl
bnQgc2VydmVyL2NsaWVudCBjb21iaW5hdGlvbiwgZG8geW91IHNlZSBhIGRpZmZlcmVudA0KPiA+
IHJlc3VsdD8NCj4gPiANCj4gDQo+IEkgdGhpbmsgdGhhdCB0aGUgc2VydmVyIGhhcyB0aGUgcHJv
YmxlbSAud2hlbiB0aGUgc2VydmVyIGRlbGV0ZXMgdGhlIGZpbGUgLA0KPiBpdCBzaG91bGQgbm90
aWNlIHRoZSBjbGllbnQgaW1tZWRpYXRlbHkuDQoNClRoZXJlIGlzIG5vIG5vdGlmaWNhdGlvbiBt
ZWNoYW5pc20gaW4gTkZTOyBvbiBvcGVuKCksIHRoZSBjbGllbnQgaXMNCnN1cHBvc2VkIHRvIHJl
dmFsaWRhdGUgaXRzIGNhY2hlZCBpbmZvcm1hdGlvbiBhbmQgdGhlIHNlcnZlciBpcyBzdXBwb3Nl
ZA0KdG8gcmV0dXJuIGFuIEVTVEFMRSBlcnJvciBpZiB0aGUgZmlsZWhhbmRsZSBpcyBubyBsb25n
ZXIgdmFsaWQuIEVpdGhlcg0Kb25lIG9mIHRoZXNlIDIgbWVjaGFuaXNtcyAoY2xpZW50IHJldmFs
aWRhdGlvbiBvciBzZXJ2ZXIgcmVwbHkpIGNvdWxkIGJlDQpnb2luZyB3cm9uZyBoZXJlLCB3aGlj
aCBpcyB3aHkgSSdtIGFza2luZy4NCg0KPiA+IFdoaWNoIGtlcm5lbHMgYXJlIHlvdSB1c2luZyBp
biB5b3VyIHRlc3RzIGZvciB0aGUgY2xpZW50IGFuZCBzZXJ2ZXI/DQo+ID4gDQo+IA0KPiBJIGZv
dW5kIHRoZSBwcm9ibGVtIGluIGtlcm5lbCAgMy43LjAtcmM2DQoNCk9LLiBEbyB5b3UgaGF2ZSBh
IG5vbi1MaW51eCBzZXJ2ZXIgb3IgY2xpZW50IGF2YWlsYWJsZSB0aGF0IHlvdSBjYW4gdXNlDQpm
b3IgdGVzdGluZz8gQWx0ZXJuYXRpdmVseSwgY291bGQgeW91IHVzZSB3aXJlc2hhcmsgdG8gY2Fw
dHVyZSBhIGR1bXAgb2YNCnRoZSBORlMgdHJhZmZpYyBiZXR3ZWVuIHRoZSBjbGllbnQgYW5kIHNl
cnZlcj8NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5l
cg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K
Myklebust, Trond 写道:
> On Fri, 2012-11-23 at 12:18 +0800, fanchaoting wrote:
>> Hi,everyone. I found a big bug abount nfs cache.
>>
>> when server delete the file ,nfs client can read file also.
>>
>> the following is the reproduce.
>>
>> ip: 192.168.0.19 nfs-client
>> ip: 192.168.0.20 nfs-client
>> ip: 192.168.0.21 nfs-server
>>
>> ############################################################################
>>
>> /usr/bin/ssh -n 192.168.0.21 service nfs start
>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
>> /usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
>> /usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
>> /usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot
>> /usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
>> /usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
>>
>> /usr/bin/ssh -n 192.168.0.19 umount /nfsroot
>> /usr/bin/ssh -n 192.168.0.20 umount /nfsroot
>> cmd="echo \"hello world\" > /nfsroot/tmpfile"
>> /usr/bin/ssh -n 192.168.0.21 $cmd
>> /usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir
>> /usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile
>> /usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot
>> /usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot
>> /usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile
>> /usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir
>> /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile
>> /usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir
>> /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile
>> /usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir
>> /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null
>> /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null
>> /usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile
>> echo -e "sleep 60~~~~~~~~\n"
>> sleep 60
>>
>> #############################################################################
>>
>> last: In 192.168.0.19 I do:
>>
>> #cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
>> hello world
>>
>>
>> I think when the nfs server delete the file ,
>> the server should notice the nfs client,
>> but the upstream kernel does't this.
>
> So is this a problem with the client or the server? In other words, if
> you use a different server/client combination, do you see a different
> result?
>
I think that the server has the problem .when the server deletes the file ,
it should notice the client immediately.
> Which kernels are you using in your tests for the client and server?
>
I found the problem in kernel 3.7.0-rc6
T24gRnJpLCAyMDEyLTExLTIzIGF0IDEyOjE4ICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4g
SGksZXZlcnlvbmUuIEkgZm91bmQgYSBiaWcgYnVnIGFib3VudCBuZnMgY2FjaGUuDQo+IA0KPiB3
aGVuIHNlcnZlciBkZWxldGUgdGhlIGZpbGUgLG5mcyBjbGllbnQgY2FuIHJlYWQgZmlsZSBhbHNv
Lg0KPiANCj4gdGhlIGZvbGxvd2luZyBpcyB0aGUgcmVwcm9kdWNlLg0KPiANCj4gaXA6IDE5Mi4x
NjguMC4xOSAgbmZzLWNsaWVudA0KPiBpcDogMTkyLjE2OC4wLjIwICBuZnMtY2xpZW50DQo+IGlw
OiAxOTIuMTY4LjAuMjEgIG5mcy1zZXJ2ZXINCj4gDQo+ICMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCj4g
DQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgc2VydmljZSBuZnMgc3RhcnQNCj4gL3Vz
ci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMSAvdXNyL3NiaW4vcnBjLmlkbWFwZA0KPiAvdXNyL2Jp
bi9zc2ggLW4gMTkyLjE2OC4wLjE5IC91c3Ivc2Jpbi9ycGMuaWRtYXBkDQo+IC91c3IvYmluL3Nz
aCAtbiAxOTIuMTY4LjAuMjAgL3Vzci9zYmluL3JwYy5pZG1hcGQNCj4gL3Vzci9iaW4vc3NoIC1u
IDE5Mi4xNjguMC4yMSAicm0gLXJmIC9uZnNyb290OyBta2RpciAtcCAvbmZzcm9vdCINCj4gL3Vz
ci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMSAvdXNyL3NiaW4vZXhwb3J0ZnMgLWF1DQo+IC91c3Iv
YmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmluL2V4cG9ydGZzIC1pIC1vIGluc2VjdXJl
LG5vX3Jvb3Rfc3F1YXNoLHJ3LGZzaWQ9MCAqOi9uZnNyb290DQo+IC91c3IvYmluL3NzaCAtbiAx
OTIuMTY4LjAuMTkgdGVzdCAtZCAvbmZzcm9vdCB8fCAocm0gLXJmIC9uZnNyb290OyBta2RpciAt
cCAvbmZzcm9vdCkNCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCB0ZXN0IC1kIC9uZnNy
b290IHx8IChybSAtcmYgL25mc3Jvb3Q7IG1rZGlyIC1wIC9uZnNyb290KQ0KPiANCj4gL3Vzci9i
aW4vc3NoIC1uIDE5Mi4xNjguMC4xOSB1bW91bnQgL25mc3Jvb3QNCj4gL3Vzci9iaW4vc3NoIC1u
IDE5Mi4xNjguMC4yMCB1bW91bnQgL25mc3Jvb3QNCj4gY21kPSJlY2hvIFwiaGVsbG8gd29ybGRc
IiA+IC9uZnNyb290L3RtcGZpbGUiDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgJGNt
ZA0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIG1rZGlyIC9uZnNyb290L3RtcGRpcg0K
PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHRvdWNoIC9uZnNyb290L3RtcGRpci90bXBk
ZmlsZQ0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IG1vdW50IC10IG5mczQgMTkyLjE2
OC4wLjIxOi8gL25mc3Jvb3QNCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCBtb3VudCAt
dCBuZnM0IDE5Mi4xNjguMC4yMTovIC9uZnNyb290DQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4
LjAuMjEgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAu
MjEgbHMgLWwgL25mc3Jvb3QvdG1wZGlyDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMTkg
Y2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMTkgbHMg
LWwgIC9uZnNyb290L3RtcGRpcg0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNhdCAg
L25mc3Jvb3QvdG1wZmlsZQ0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGxzIC1sICAv
bmZzcm9vdC90bXBkaXINCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4xOSBjYXQgL25mc3Jv
b3QvdG1wZmlsZSA+IC9kZXYvbnVsbA0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNh
dCAvbmZzcm9vdC90bXBmaWxlID4gL2Rldi9udWxsDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4
LjAuMjEgcm0gLXJmIC9uZnNyb290L3RtcGZpbGUNCj4gZWNobyAtZSAic2xlZXAgNjB+fn5+fn5+
flxuIg0KPiBzbGVlcCA2MA0KPiANCj4gIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCj4gDQo+IGxhc3Q6
IEluIDE5Mi4xNjguMC4xOSBJIGRvOg0KPiANCj4gI2NhdCAvbmZzcm9vdC90bXBmaWxlICAgICAg
ICAgICA8LS10aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUsYnV0IG5mcyBjbGllbnQgY2Fu
IHJlYWQgdGhlIGZpbGUNCj4gIGhlbGxvIHdvcmxkDQo+IA0KPiANCj4gSSB0aGluayAgd2hlbiB0
aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUgLA0KPiB0aGUgIHNlcnZlciBzaG91bGQgbm90
aWNlIHRoZSBuZnMgY2xpZW50LA0KPiBidXQgdGhlIHVwc3RyZWFtIGtlcm5lbCBkb2VzJ3QgdGhp
cy4NCg0KU28gaXMgdGhpcyBhIHByb2JsZW0gd2l0aCB0aGUgY2xpZW50IG9yIHRoZSBzZXJ2ZXI/
IEluIG90aGVyIHdvcmRzLCBpZg0KeW91IHVzZSBhIGRpZmZlcmVudCBzZXJ2ZXIvY2xpZW50IGNv
bWJpbmF0aW9uLCBkbyB5b3Ugc2VlIGEgZGlmZmVyZW50DQpyZXN1bHQ/DQoNCldoaWNoIGtlcm5l
bHMgYXJlIHlvdSB1c2luZyBpbiB5b3VyIHRlc3RzIGZvciB0aGUgY2xpZW50IGFuZCBzZXJ2ZXI/
DQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0K
TmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==
On Fri, Nov 23, 2012 at 02:00:13PM +0800, fanchaoting wrote:
> Myklebust, Trond 写道:
> > On Fri, 2012-11-23 at 13:23 +0800, fanchaoting wrote:
> >> Myklebust, Trond 写道:
> >>> On Fri, 2012-11-23 at 12:18 +0800, fanchaoting wrote:
> >>>> I think when the nfs server delete the file ,
> >>>> the server should notice the nfs client,
> >>>> but the upstream kernel does't this.
> >>> So is this a problem with the client or the server? In other words, if
> >>> you use a different server/client combination, do you see a different
> >>> result?
> >>>
> >> I think that the server has the problem .when the server deletes the file ,
> >> it should notice the client immediately.
> >
> > There is no notification mechanism in NFS; on open(), the client is
> > supposed to revalidate its cached information and the server is supposed
> > to return an ESTALE error if the filehandle is no longer valid. Either
> > one of these 2 mechanisms (client revalidation or server reply) could be
> > going wrong here, which is why I'm asking.
>
> I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem .
> But I did't found it in the upstream kernel.
Yes, still working on getting that merged. Next step is probably to
rebase it on top of jlayton's ESTALE patches, assuming those will go in
first. In theory they could make 3.8 but that may be optimistic.
--b.
Myklebust, Trond 写道:
> On Fri, 2012-11-23 at 13:23 +0800, fanchaoting wrote:
>> Myklebust, Trond 写道:
>>> On Fri, 2012-11-23 at 12:18 +0800, fanchaoting wrote:
>>>> Hi,everyone. I found a big bug abount nfs cache.
>>>>
>>>> when server delete the file ,nfs client can read file also.
>>>>
>>>> the following is the reproduce.
>>>>
>>>> ip: 192.168.0.19 nfs-client
>>>> ip: 192.168.0.20 nfs-client
>>>> ip: 192.168.0.21 nfs-server
>>>>
>>>> ############################################################################
>>>>
>>>> /usr/bin/ssh -n 192.168.0.21 service nfs start
>>>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
>>>> /usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
>>>> /usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
>>>> /usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
>>>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
>>>> /usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot
>>>> /usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
>>>> /usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
>>>>
>>>> /usr/bin/ssh -n 192.168.0.19 umount /nfsroot
>>>> /usr/bin/ssh -n 192.168.0.20 umount /nfsroot
>>>> cmd="echo \"hello world\" > /nfsroot/tmpfile"
>>>> /usr/bin/ssh -n 192.168.0.21 $cmd
>>>> /usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir
>>>> /usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile
>>>> /usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot
>>>> /usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot
>>>> /usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile
>>>> /usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir
>>>> /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile
>>>> /usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir
>>>> /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile
>>>> /usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir
>>>> /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null
>>>> /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null
>>>> /usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile
>>>> echo -e "sleep 60~~~~~~~~\n"
>>>> sleep 60
>>>>
>>>> #############################################################################
>>>>
>>>> last: In 192.168.0.19 I do:
>>>>
>>>> #cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
>>>> hello world
>>>>
>>>>
>>>> I think when the nfs server delete the file ,
>>>> the server should notice the nfs client,
>>>> but the upstream kernel does't this.
>>> So is this a problem with the client or the server? In other words, if
>>> you use a different server/client combination, do you see a different
>>> result?
>>>
>> I think that the server has the problem .when the server deletes the file ,
>> it should notice the client immediately.
>
> There is no notification mechanism in NFS; on open(), the client is
> supposed to revalidate its cached information and the server is supposed
> to return an ESTALE error if the filehandle is no longer valid. Either
> one of these 2 mechanisms (client revalidation or server reply) could be
> going wrong here, which is why I'm asking.
I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem .
But I did't found it in the upstream kernel.
I think when the server delete the file, it should reply a DELEGRETURN to the nfs client.
>
>>> Which kernels are you using in your tests for the client and server?
>>>
>> I found the problem in kernel 3.7.0-rc6
>
> OK. Do you have a non-Linux server or client available that you can use
> for testing? Alternatively, could you use wireshark to capture a dump of
> the NFS traffic between the client and server?
In the wireshark, I found the right traffic should have open operation,
but now i can't find the open operation.
I found the function nfs4_open_prepare has problem , in the old kernel ,
it can run
...snip...
rpc_call_start(task);
...snip...
>