2003-06-23 22:14:05

by Christian Robottom Reis

[permalink] [raw]
Subject: Stale fcntl lock never released


Just saw this from one of the diskless boxes on our system (clients
using vanilla 2.4.21, server running 2.4.20-rc3). The client was running
mutt and was editing a message when it hung. We shut down the box and
started another one up, but when the user tried to read his mail, mutt
was hanging upon fcntl(); strace shows it was returning Resource
temporarily unavailable (ENOLCK IIRC).

On the server side, we have:

kiko@anthem:~$ cat /proc/locks | grep 09:06
1: POSIX ADVISORY WRITE 11771 09:06:60316 0 EOF d0d51300 c02d1048
d0d51424 00000000 d0d5130c
2: POSIX ADVISORY WRITE 11752 09:06:60311 0 EOF d0d51420 d0d51304
f7c367e4 00000000 d0d5142c
4: POSIX ADVISORY WRITE 10560 09:06:60312 0 EOF f7c36840 f7c367e4
f7c36f64 00000000 f7c3684c
5: POSIX ADVISORY WRITE 599 09:06:30193 0 EOF f7c36f60 f7c36844
f7c36964 00000000 f7c36eac
5: -> POSIX ADVISORY WRITE 10569 09:06:30193 191252 EOF f7c36ea0
f7c36604 c02d1050 f7c36f60 f7c3660c
5: -> POSIX ADVISORY WRITE 11787 09:06:30193 191252 EOF f7c36600
c02d1050 f7c36ea4 f7c36f60 f7c36f6c

(All these locks were held only by the crashed workstation)

Client mount options are:

anthem:/var/spool/mail on /var/mail type nfs
(rw,sync,v3,rsize=8192,wsize=8192,acregmin=0,acregmax=0,
acdirmin=0,acdirmax=0,hard,udp,noac,lock,addr=anthem)

Restarting the nfs service frees the locks, but shouldn't it be done
automatically? I'm not sure if it's a case of the server not clearing
the lock, or the client not tickling the server into it.. any ideas?

(Trond, do any of the lockd patches in
http://www.fys.uio.no/~trondmy/src/2.4.21/ fix something like this?)

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2003-06-24 19:02:10

by Trond Myklebust

[permalink] [raw]
Subject: Re: Stale fcntl lock never released

>>>>> " " == Christian Reis <[email protected]> writes:

> (Trond, do any of the lockd patches in
> http://www.fys.uio.no/~trondmy/src/2.4.21/ fix something like
> this?)

No. This is what rpc.statd is supposed to do. When a client reboots,
it is supposed to notify the server that it should remove all locks
belonging to that client.

Cheers,
Trond


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-06-24 19:07:23

by Christian Robottom Reis

[permalink] [raw]
Subject: Re: Stale fcntl lock never released

On Tue, Jun 24, 2003 at 09:02:10PM +0200, Trond Myklebust wrote:
> >>>>> " " == Christian Reis <[email protected]> writes:
>
> > (Trond, do any of the lockd patches in
> > http://www.fys.uio.no/~trondmy/src/2.4.21/ fix something like
> > this?)
>
> No. This is what rpc.statd is supposed to do. When a client reboots,
> it is supposed to notify the server that it should remove all locks
> belonging to that client.

What if the client never reboots, but instead is turned off and another
client turned on? Should the lock persist? For how long?

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-06-24 19:14:10

by Trond Myklebust

[permalink] [raw]
Subject: Re: Stale fcntl lock never released

>>>>> " " == Christian Reis <[email protected]> writes:

> What if the client never reboots, but instead is turned off and
> another client turned on? Should the lock persist? For how
> long?

It will persist forever.

Cheers,
Trond


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-06-24 19:18:30

by Christian Robottom Reis

[permalink] [raw]
Subject: Re: Stale fcntl lock never released

On Tue, Jun 24, 2003 at 09:14:10PM +0200, Trond Myklebust wrote:
> >>>>> " " == Christian Reis <[email protected]> writes:
>
> > What if the client never reboots, but instead is turned off and
> > another client turned on? Should the lock persist? For how
> > long?
>
> It will persist forever.

How odd.

Is the only solution is to restart the server-side statd, or knfsd?

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-06-24 22:03:10

by Spencer Shepler

[permalink] [raw]
Subject: Re: Stale fcntl lock never released


Some server implementations provide a administrative command
to release NFS locks held at the server for a specific client
for this scenario where the client is turned off or will never
contact the server again.

Spencer

Christian Reis wrote:
>
> On Tue, Jun 24, 2003 at 09:14:10PM +0200, Trond Myklebust wrote:
> > >>>>> " " == Christian Reis <[email protected]> writes:
> >
> > > What if the client never reboots, but instead is turned off and
> > > another client turned on? Should the lock persist? For how
> > > long?
> >
> > It will persist forever.
>
> How odd.
>
> Is the only solution is to restart the server-side statd, or knfsd?
>
> Take care,
> --
> Christian Reis, Senior Engineer, Async Open Source, Brazil.
> http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: INetU
> Attention Web Developers & Consultants: Become An INetU Hosting Partner.
> Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
> INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs


-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-06-24 21:28:36

by Steve Dickson

[permalink] [raw]
Subject: Re: Stale fcntl lock never released

The statd in nfs-utils-1.0.3 has a new -N flag that might work...

-N Causes statd to run in the notify-only mode. When
started in
this mode, the statd program will check its state
directory,
send notifications to any monitored nodes, and exit
once the
notifications have been sent. This mode is used to enable
Highly
Available NFS implementations (i.e. HA-NFS).

SteveD.

Christian Reis wrote:

>On Tue, Jun 24, 2003 at 09:14:10PM +0200, Trond Myklebust wrote:
>
>
>>>>>>>" " == Christian Reis <[email protected]> writes:
>>>>>>>
>>>>>>>
>> > What if the client never reboots, but instead is turned off and
>> > another client turned on? Should the lock persist? For how
>> > long?
>>
>>It will persist forever.
>>
>>
>
>How odd.
>
>Is the only solution is to restart the server-side statd, or knfsd?
>
>Take care,
>--
>Christian Reis, Senior Engineer, Async Open Source, Brazil.
>http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: INetU
>Attention Web Developers & Consultants: Become An INetU Hosting Partner.
>Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
>INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
>_______________________________________________
>NFS maillist - [email protected]
>https://lists.sourceforge.net/lists/listinfo/nfs
>
>



-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs