2011-12-01 17:07:23

by Pavel A

[permalink] [raw]
Subject: NFS - lock failover

Hi everyone!

I was trying not to create new topics, but it seems that posting to an
old one doesn't bring it up. Here is the original topic I'm referring
to: http://comments.gmane.org/gmane.linux.nfs/13108

I'm building an A/A cluster using NFS v3 and local file systems, and
looking for
efficient ways for failover (for now I have to restart nfs-kernel-server on
Takeover node to be able to initiate grace period), so the discussed solutions
are very interesting to me.

Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
see that the ability to release locks was really implemented and is
working well
(I mean interfaces /proc/fs/nfsd/unlock_ip and
/proc/fs/nfsd/unlock_filesystem),
but how about reacquiring locks on the node, share migrates to? - I've been
going through various mailing lists and found a lot of discussions on the topic
(also dated mainly 2007), but don't seem to find any rpc-based mechanism or
interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?

Any clarification would be really nice. Thank you!


2011-12-12 19:56:14

by J. Bruce Fields

[permalink] [raw]
Subject: Re: NFS - lock failover

On Fri, Dec 09, 2011 at 11:36:46AM +0100, Frank van Maarseveen wrote:
> On Thu, Dec 01, 2011 at 06:24:16PM +0100, Frank van Maarseveen wrote:
> > On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
> > > Hi everyone!
> > >
> > > I was trying not to create new topics, but it seems that posting to an
> > > old one doesn't bring it up. Here is the original topic I'm referring
> > > to: http://comments.gmane.org/gmane.linux.nfs/13108
> > >
> > > I'm building an A/A cluster using NFS v3 and local file systems, and
> > > looking for
> > > efficient ways for failover (for now I have to restart nfs-kernel-server on
> > > Takeover node to be able to initiate grace period), so the discussed solutions
> > > are very interesting to me.
> > >
> > > Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
> > > see that the ability to release locks was really implemented and is
> > > working well
> > > (I mean interfaces /proc/fs/nfsd/unlock_ip and
> > > /proc/fs/nfsd/unlock_filesystem),
> > > but how about reacquiring locks on the node, share migrates to? - I've been
> > > going through various mailing lists and found a lot of discussions on the topic
> > > (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
> > > interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
> >
> > I've posted a patch some time ago implementing
> > /proc/fs/nfsd/relock_filesystem:
> >
> > http://permalink.gmane.org/gmane.linux.nfs/42360
> > http://permalink.gmane.org/gmane.linux.nfs/42361
> > http://permalink.gmane.org/gmane.linux.nfs/42362
>
> Can this patch be scheduled for inclusion in mainline (3.3)?

Could you resend and I'll take another look?

But this should really be fixed to handle v4 locks as well.

--b.

2011-12-09 10:36:48

by Frank van Maarseveen

[permalink] [raw]
Subject: Re: NFS - lock failover

On Thu, Dec 01, 2011 at 06:24:16PM +0100, Frank van Maarseveen wrote:
> On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
> > Hi everyone!
> >
> > I was trying not to create new topics, but it seems that posting to an
> > old one doesn't bring it up. Here is the original topic I'm referring
> > to: http://comments.gmane.org/gmane.linux.nfs/13108
> >
> > I'm building an A/A cluster using NFS v3 and local file systems, and
> > looking for
> > efficient ways for failover (for now I have to restart nfs-kernel-server on
> > Takeover node to be able to initiate grace period), so the discussed solutions
> > are very interesting to me.
> >
> > Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
> > see that the ability to release locks was really implemented and is
> > working well
> > (I mean interfaces /proc/fs/nfsd/unlock_ip and
> > /proc/fs/nfsd/unlock_filesystem),
> > but how about reacquiring locks on the node, share migrates to? - I've been
> > going through various mailing lists and found a lot of discussions on the topic
> > (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
> > interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
>
> I've posted a patch some time ago implementing
> /proc/fs/nfsd/relock_filesystem:
>
> http://permalink.gmane.org/gmane.linux.nfs/42360
> http://permalink.gmane.org/gmane.linux.nfs/42361
> http://permalink.gmane.org/gmane.linux.nfs/42362

Can this patch be scheduled for inclusion in mainline (3.3)?

--
Frank

2011-12-01 17:34:37

by Pavel A

[permalink] [raw]
Subject: Re: NFS - lock failover

Thank you very much! This is something I was looking for.
I only wonder, why isn't it included in packages? - I will be of great
help to many other people as well.

2011/12/1 Frank van Maarseveen <[email protected]>:
> On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
>> Hi everyone!
>>
>> I was trying not to create new topics, but it seems that posting to an
>> old one doesn't bring it up. Here is the original topic I'm referring
>> to: http://comments.gmane.org/gmane.linux.nfs/13108
>>
>> I'm building an A/A cluster using NFS v3 and local file systems, and
>> looking for
>> efficient ways for failover (for now I have to restart nfs-kernel-server on
>> Takeover node to be able to initiate grace period), so the discussed solutions
>> are very interesting to me.
>>
>> Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
>> see that the ability to release locks was really implemented and is
>> working well
>> (I mean interfaces /proc/fs/nfsd/unlock_ip and
>> /proc/fs/nfsd/unlock_filesystem),
>> but how about reacquiring locks on the node, share migrates to? - I've been
>> going through various mailing lists and found a lot of discussions on the topic
>> (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
>> interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
>
> I've posted a patch some time ago implementing
> /proc/fs/nfsd/relock_filesystem:
>
>        http://permalink.gmane.org/gmane.linux.nfs/42360
>        http://permalink.gmane.org/gmane.linux.nfs/42361
>        http://permalink.gmane.org/gmane.linux.nfs/42362
>
> --
> Frank

2011-12-12 20:30:29

by Frank van Maarseveen

[permalink] [raw]
Subject: Re: NFS - lock failover

On Mon, Dec 12, 2011 at 02:56:13PM -0500, J. Bruce Fields wrote:
> On Fri, Dec 09, 2011 at 11:36:46AM +0100, Frank van Maarseveen wrote:
> > On Thu, Dec 01, 2011 at 06:24:16PM +0100, Frank van Maarseveen wrote:
> > > On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
> > > > Hi everyone!
> > > >
> > > > I was trying not to create new topics, but it seems that posting to an
> > > > old one doesn't bring it up. Here is the original topic I'm referring
> > > > to: http://comments.gmane.org/gmane.linux.nfs/13108
> > > >
> > > > I'm building an A/A cluster using NFS v3 and local file systems, and
> > > > looking for
> > > > efficient ways for failover (for now I have to restart nfs-kernel-server on
> > > > Takeover node to be able to initiate grace period), so the discussed solutions
> > > > are very interesting to me.
> > > >
> > > > Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
> > > > see that the ability to release locks was really implemented and is
> > > > working well
> > > > (I mean interfaces /proc/fs/nfsd/unlock_ip and
> > > > /proc/fs/nfsd/unlock_filesystem),
> > > > but how about reacquiring locks on the node, share migrates to? - I've been
> > > > going through various mailing lists and found a lot of discussions on the topic
> > > > (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
> > > > interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
> > >
> > > I've posted a patch some time ago implementing
> > > /proc/fs/nfsd/relock_filesystem:
> > >
> > > http://permalink.gmane.org/gmane.linux.nfs/42360
> > > http://permalink.gmane.org/gmane.linux.nfs/42361
> > > http://permalink.gmane.org/gmane.linux.nfs/42362
> >
> > Can this patch be scheduled for inclusion in mainline (3.3)?
>
> Could you resend and I'll take another look?

I'll check it again but except for one fuzz it should still apply on
3.2-rc. Patch offsets and an incidental fuzz is sometimes informative
about possible conflict areas. It has been verified on 3.0 and 3.1

>
> But this should really be fixed to handle v4 locks as well.

Why do you think it doesn't?

--
Frank

2011-12-12 20:48:39

by J. Bruce Fields

[permalink] [raw]
Subject: Re: NFS - lock failover

On Mon, Dec 12, 2011 at 09:30:27PM +0100, Frank van Maarseveen wrote:
> On Mon, Dec 12, 2011 at 02:56:13PM -0500, J. Bruce Fields wrote:
> > On Fri, Dec 09, 2011 at 11:36:46AM +0100, Frank van Maarseveen wrote:
> > > On Thu, Dec 01, 2011 at 06:24:16PM +0100, Frank van Maarseveen wrote:
> > > > On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
> > > > > Hi everyone!
> > > > >
> > > > > I was trying not to create new topics, but it seems that posting to an
> > > > > old one doesn't bring it up. Here is the original topic I'm referring
> > > > > to: http://comments.gmane.org/gmane.linux.nfs/13108
> > > > >
> > > > > I'm building an A/A cluster using NFS v3 and local file systems, and
> > > > > looking for
> > > > > efficient ways for failover (for now I have to restart nfs-kernel-server on
> > > > > Takeover node to be able to initiate grace period), so the discussed solutions
> > > > > are very interesting to me.
> > > > >
> > > > > Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
> > > > > see that the ability to release locks was really implemented and is
> > > > > working well
> > > > > (I mean interfaces /proc/fs/nfsd/unlock_ip and
> > > > > /proc/fs/nfsd/unlock_filesystem),
> > > > > but how about reacquiring locks on the node, share migrates to? - I've been
> > > > > going through various mailing lists and found a lot of discussions on the topic
> > > > > (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
> > > > > interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
> > > >
> > > > I've posted a patch some time ago implementing
> > > > /proc/fs/nfsd/relock_filesystem:
> > > >
> > > > http://permalink.gmane.org/gmane.linux.nfs/42360
> > > > http://permalink.gmane.org/gmane.linux.nfs/42361
> > > > http://permalink.gmane.org/gmane.linux.nfs/42362
> > >
> > > Can this patch be scheduled for inclusion in mainline (3.3)?
> >
> > Could you resend and I'll take another look?
>
> I'll check it again but except for one fuzz it should still apply on
> 3.2-rc. Patch offsets and an incidental fuzz is sometimes informative
> about possible conflict areas. It has been verified on 3.0 and 3.1

Thanks. It would be helpful to have them resent in any case.

> > But this should really be fixed to handle v4 locks as well.
>
> Why do you think it doesn't?

The fact that unlock_* doesn't. Looking at it.... I suppose that
doesn't matter as much on the grace-period starting side. How much use
does that have on its own?

--b.

2011-12-01 17:30:39

by Frank van Maarseveen

[permalink] [raw]
Subject: Re: NFS - lock failover

On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
> Hi everyone!
>
> I was trying not to create new topics, but it seems that posting to an
> old one doesn't bring it up. Here is the original topic I'm referring
> to: http://comments.gmane.org/gmane.linux.nfs/13108
>
> I'm building an A/A cluster using NFS v3 and local file systems, and
> looking for
> efficient ways for failover (for now I have to restart nfs-kernel-server on
> Takeover node to be able to initiate grace period), so the discussed solutions
> are very interesting to me.
>
> Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
> see that the ability to release locks was really implemented and is
> working well
> (I mean interfaces /proc/fs/nfsd/unlock_ip and
> /proc/fs/nfsd/unlock_filesystem),
> but how about reacquiring locks on the node, share migrates to? - I've been
> going through various mailing lists and found a lot of discussions on the topic
> (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
> interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?

I've posted a patch some time ago implementing
/proc/fs/nfsd/relock_filesystem:

http://permalink.gmane.org/gmane.linux.nfs/42360
http://permalink.gmane.org/gmane.linux.nfs/42361
http://permalink.gmane.org/gmane.linux.nfs/42362

--
Frank

2011-12-01 17:36:04

by Pavel A

[permalink] [raw]
Subject: Re: NFS - lock failover

2011/12/1 Pavel A <[email protected]>:
> Thank you very much! This is something I was looking for.
> I only wonder, why isn't it included in packages? - I will be of great
> help to many other people as well.
sorry, _it_ will be of great help to many other people as well.

>
> 2011/12/1 Frank van Maarseveen <[email protected]>:
>> On Thu, Dec 01, 2011 at 07:07:22PM +0200, Pavel A wrote:
>>> Hi everyone!
>>>
>>> I was trying not to create new topics, but it seems that posting to an
>>> old one doesn't bring it up. Here is the original topic I'm referring
>>> to: http://comments.gmane.org/gmane.linux.nfs/13108
>>>
>>> I'm building an A/A cluster using NFS v3 and local file systems, and
>>> looking for
>>> efficient ways for failover (for now I have to restart nfs-kernel-server on
>>> Takeover node to be able to initiate grace period), so the discussed solutions
>>> are very interesting to me.
>>>
>>> Now (4 years after) in current nfs-utils packages (v. 1.2.2-4 and later) I can
>>> see that the ability to release locks was really implemented and is
>>> working well
>>> (I mean interfaces /proc/fs/nfsd/unlock_ip and
>>> /proc/fs/nfsd/unlock_filesystem),
>>> but how about reacquiring locks on the node, share migrates to? - I've been
>>> going through various mailing lists and found a lot of discussions on the topic
>>> (also dated mainly 2007), but don't seem to find any rpc-based mechanism or
>>> interface like /proc/fs/nfsd/nlm_set_grace to do that, was it ever made?
>>
>> I've posted a patch some time ago implementing
>> /proc/fs/nfsd/relock_filesystem:
>>
>>        http://permalink.gmane.org/gmane.linux.nfs/42360
>>        http://permalink.gmane.org/gmane.linux.nfs/42361
>>        http://permalink.gmane.org/gmane.linux.nfs/42362
>>
>> --
>> Frank