Hi all,
Apparently lockd does not expect statd to be used with -n
switch: statd is expected to bind loopback, always. Attached
patches show one (IPv4 specific) way of fixing it. Comments?
--
// Janne
Janne Karhunen wrote:
>
>> I think you mis-understand our approach. With our patch, you do *not* need
>> to do anything if you have only one single (floating) interface to do nfs
>> export (and you can have the machine configured to use other hostname). The
>> statd's "my_name" is correctly filled (from kernel by our patch) - it is no
>> longer bound to 127.0.0.1.
>>
>
> What happens when it fails over to node that has
> different name? Mon_name in NOTIFY changes
> while address it's coming from does not. Which
> one should client believe - the address or the
> name?
>
> But OK, I probably need to dig out the patch to
> have a look.
>
In the middle of re-doing our old patch (it should be very small this
time, with Frank's patch already in mainline and Chuck taking over IPV6
issues) so the logic can be clear. I also have a feel that you never
look into "sm-notify" ? Get nfs-util git tree source and take a look at
the "README" file in the top directory. Either Neil Brown or Steve
Dickson had added a nice "DAEMON STARTUP ORDER" there. Basically the
whole issue is a two-steps thing:
1. statd is the process responsible for recording client address into
the nsm directory
2. sm-notify is the process responsible for notifying clients about
server reboot activities.
The "sm-notify" design allows the flexibility of notifying clients with
different cluster configurations, including yours. I think you already
know this stuff very well... just do a "man sm-notify" ... it should
clear up your confusions.
-- Wendy
On Sat, May 3, 2008 at 11:29 AM, Wendy Cheng <[email protected]> wrote:
> > > I think you mis-understand our approach. With our patch, you do *not*
> need
> > > to do anything if you have only one single (floating) interface to do
> nfs
> > > export (and you can have the machine configured to use other hostname).
> The
> > > statd's "my_name" is correctly filled (from kernel by our patch) - it is
> no
> > > longer bound to 127.0.0.1.
> > >
> > >
> >
> > What happens when it fails over to node that has
> > different name? Mon_name in NOTIFY changes
> > while address it's coming from does not. Which
> > one should client believe - the address or the
> > name?
> >
> > But OK, I probably need to dig out the patch to
> > have a look.
> >
> >
> In the middle of re-doing our old patch (it should be very small this time,
> with Frank's patch already in mainline and Chuck taking over IPV6 issues) so
> the logic can be clear. I also have a feel that you never look into
> "sm-notify" ?
I didn't, as I was under a (apparently false) impression
that this indeed is a separate, manual way to trigger
the notification for whatever purpose. My apologies,
--
// Janne
On Fri, May 02, 2008 at 08:24:25PM -0400, Janne Karhunen wrote:
> On Fri, May 2, 2008 at 6:33 PM, Wendy Cheng <[email protected]> wrote:
>
> > Maybe I mislead you in previous post. The "-H" is only an "example" - to
> > show people how to selectively move an ip address around without affecting
> > other co-existing nfs ip interface on the same server. I did plan to submit
> > a complete user mode patch so the program after "-H" will have a default
> > executable (but user still can use their own if they don't like our nlm
> > directory structure).
>
> Even if you ship the default executable user would
> still need to go and edit it (add the address). Well,
> I guess it would not be that much harder than
> specifying the -n, but somehow this does seem
> more arcane (for simple setups).
In any case, if there's still a legimate use case for -n, even if it's
not great, we should err on the side of fixing it just to save problems
for anyone with an existing working setup.
If -n has never worked at all for anyone, then OK, let's get rid of it.
--b.
J. Bruce Fields wrote:
> On Fri, May 02, 2008 at 08:24:25PM -0400, Janne Karhunen wrote:
>
>> On Fri, May 2, 2008 at 6:33 PM, Wendy Cheng <[email protected]> wrote:
>>
>>
>>> Maybe I mislead you in previous post. The "-H" is only an "example" - to
>>> show people how to selectively move an ip address around without affecting
>>> other co-existing nfs ip interface on the same server. I did plan to submit
>>> a complete user mode patch so the program after "-H" will have a default
>>> executable (but user still can use their own if they don't like our nlm
>>> directory structure).
>>>
>> Even if you ship the default executable user would
>> still need to go and edit it (add the address). Well,
>> I guess it would not be that much harder than
>> specifying the -n, but somehow this does seem
>> more arcane (for simple setups).
>>
>
> In any case, if there's still a legimate use case for -n, even if it's
> not great, we should err on the side of fixing it just to save problems
> for anyone with an existing working setup.
>
> If -n has never worked at all for anyone, then OK, let's get rid of it.
>
> --b.
>
Simpler code and consolidated logic flow are always better - would like
to vote for its deletion.
I had a very bad experience with a filesystem (*cough*) that accepted
few mis-understood boundary condition fixes that ended up disturbing the
core logic. It still couldn't recover from it and the code churning
keeps going on until today.
-- Wendy
On Mon, May 5, 2008 at 10:59 AM, Wendy Cheng <[email protected]> wrote:
> > In any case, if there's still a legimate use case for -n, even if it's
> > not great, we should err on the side of fixing it just to save problems
> > for anyone with an existing working setup.
> >
> Simpler code and consolidated logic flow are always better - would like to
> vote for its deletion.
So the solution would be to use -H with a script that
does sm-notify with user specified name (-v)? Kernel
name can't be used directly as it might migrate to
node that has different name.
--
// Janne
Janne Karhunen wrote:
> On Mon, May 5, 2008 at 10:59 AM, Wendy Cheng <[email protected]> wrote:
>
>
>>> In any case, if there's still a legimate use case for -n, even if it's
>>> not great, we should err on the side of fixing it just to save problems
>>> for anyone with an existing working setup.
>>>
>>>
>> Simpler code and consolidated logic flow are always better - would like to
>> vote for its deletion.
>>
>
> So the solution would be to use -H with a script that
> does sm-notify with user specified name (-v)? Kernel
> name can't be used directly as it might migrate to
> node that has different name.
>
>
>
No. "-H" is only for different export (server IP) interfaces. Say server
has 10.1.1.1 and 10.1.1.2 as NFS export interfaces. If 10.1.1.1 is
migrated but 10.1.1.2 stays, you would like to notify the nfs clients
coming in from 10.1.1.1 but not the clients from 10.1.1.2, you need to
structure your sm directory seperately. That is the usage of "-H" (to
allow admin to organize client IPs).
For a single export IP interface like you have (even you have a seperate
hostname), ignore "-H". I believe "sm-notify -v" should be enough -
otherwise we have bugs - i.e., sm-notify should be part of your nfsd
bring-up script.
-- Wendy
On Mon, May 5, 2008 at 11:21 AM, Wendy Cheng <[email protected]> wrote:
> No. "-H" is only for different export (server IP) interfaces. Say server
> has 10.1.1.1 and 10.1.1.2 as NFS export interfaces. If 10.1.1.1 is migrated
> but 10.1.1.2 stays, you would like to notify the nfs clients coming in from
> 10.1.1.1 but not the clients from 10.1.1.2, you need to structure your sm
> directory seperately. That is the usage of "-H" (to allow admin to organize
> client IPs).
Right.
> For a single export IP interface like you have (even you have a seperate
> hostname), ignore "-H". I believe "sm-notify -v" should be enough -
> otherwise we have bugs - i.e., sm-notify should be part of your nfsd
> bring-up script.
As statd runs this directly and reads name for -v from
MY_NAME, how exactly would you pass it MY_NAME
given that -n is removed?
--
// Janne
On Mon, May 5, 2008 at 11:23 AM, Janne Karhunen
<[email protected]> wrote:
> > For a single export IP interface like you have (even you have a seperate
> > hostname), ignore "-H". I believe "sm-notify -v" should be enough -
> > otherwise we have bugs - i.e., sm-notify should be part of your nfsd
> > bring-up script.
>
> As statd runs this directly and reads name for -v from
> MY_NAME, how exactly would you pass it MY_NAME
> given that -n is removed?
So users should add this themselves into their scripts?
--
// Janne
On Mon, May 05, 2008 at 11:23:37AM -0400, Janne Karhunen wrote:
> On Mon, May 5, 2008 at 11:21 AM, Wendy Cheng <[email protected]> wrote:
>
> > No. "-H" is only for different export (server IP) interfaces. Say server
> > has 10.1.1.1 and 10.1.1.2 as NFS export interfaces. If 10.1.1.1 is migrated
> > but 10.1.1.2 stays, you would like to notify the nfs clients coming in from
> > 10.1.1.1 but not the clients from 10.1.1.2, you need to structure your sm
> > directory seperately. That is the usage of "-H" (to allow admin to organize
> > client IPs).
>
> Right.
>
>
> > For a single export IP interface like you have (even you have a seperate
> > hostname), ignore "-H". I believe "sm-notify -v" should be enough -
> > otherwise we have bugs - i.e., sm-notify should be part of your nfsd
> > bring-up script.
>
> As statd runs this directly and reads name for -v from
> MY_NAME,
Nope; see the -L option to statd.
--b.
> how exactly would you pass it MY_NAME
> given that -n is removed?
On Mon, May 5, 2008 at 11:25 AM, J. Bruce Fields <[email protected]> wrote:
> > As statd runs this directly and reads name for -v from
> > MY_NAME,
>
> Nope; see the -L option to statd.
So the let the user worry approach. Statd does not
notify anyone and user is supposed to do it with
sm-notify manually. This will work, but -n looks
cleaner me..
--
// Janne
Janne Karhunen wrote:
> On Mon, May 5, 2008 at 11:25 AM, J. Bruce Fields <[email protected]> wrote:
>
>
>> > As statd runs this directly and reads name for -v from
>> > MY_NAME,
>>
>> Nope; see the -L option to statd.
>>
>
> So the let the user worry approach. Statd does not
> notify anyone and user is supposed to do it with
> sm-notify manually. This will work, but -n looks
> cleaner me..
>
>
Or we can do this ...
If user specified "statd -n", we run "sm-notify -v" command for them ?
So we don't need to fork the code but also keep someone happy ?
Give me one day to see whether it is doable ?
-- Wendy
On Mon, May 05, 2008 at 11:28:03AM -0400, Janne Karhunen wrote:
> On Mon, May 5, 2008 at 11:25 AM, J. Bruce Fields <[email protected]> wrote:
>
> > > As statd runs this directly and reads name for -v from
> > > MY_NAME,
> >
> > Nope; see the -L option to statd.
>
> So the let the user worry approach. Statd does not
> notify anyone and user is supposed to do it with
> sm-notify manually.
We should modify distro startup scripts to run sm-notify and add -L to
statd, following the recommendations in nfs-utils/README.
That will leave the -v option to sm-notify, which will need to be added
somehow to handle the highly-available-nfs case.
> This will work, but -n looks cleaner me..
I've lost the thread here a little. Let me try to summarize, and tell
me what I've forgotten:
So we've got multiple nfs servers that may serve the same export. Only
one serves a given export at a time.
To keep the clients from having to understand this, we tell them to
mount a single "floating" ip address, and when we want to use a
different server, we move that floating ip address, and the clients
automatically follow it. The only problem is that the new server
doesn't know about locks held on the old server. We solve that problem
by telling clients that the server has rebooted, and that they must
reclaim locks.
To make that work, we need to ensure that statd notifies the clients
that "floating-ip" has rebooted. (They will get confused if they are
notified that "some-other-ip" has rebooted, where "some-other-ip" is
just some random ip that the new server happens to use on a different
interface.)
In the case that you care about, a given server only has one floating ip
address pointing at it at a time. For your case, you could start statd
either with:
sm-notify -v <floating ip>
statd -L
or (assuming it was fixed)
statd -n <floating ip>
and either would do the job of notifying all monitored clients that
<floating ip> has "rebooted".
Wendy also worries about the case of a server with multiple floating
ip's that may move independently of each other, or with both floating
ip's and ip's that don't move. In that case we want to notify only
some subset of the clients.
So she uses 'statd -H' to modify the way statd stores its records, so
that it can easily distinguish between clients that are associated with
a floating ip that's moving (and need to be notified), and clients that
aren't (and don't need to be notified right now).
The statd -n option has actually *never* worked, in any previous version
of nfs-utils--with it specified, statd can no longer communicate with
the nfs server on the same host. That means the server won't find out
when a client reboots. (But I guess even the broken -n may still have
been sufficient to allow clients to monitor the server and find out when
the server reboots.)
--b.
On Mon, May 5, 2008 at 12:00 PM, Wendy Cheng <[email protected]> wrote:
> > > > As statd runs this directly and reads name for -v from
> > > > MY_NAME,
> > >
> > > Nope; see the -L option to statd.
> > >
> >
> > So the let the user worry approach. Statd does not
> > notify anyone and user is supposed to do it with
> > sm-notify manually. This will work, but -n looks
> > cleaner me..
>
> Or we can do this ...
> If user specified "statd -n", we run "sm-notify -v" command for them ? So
> we don't need to fork the code but also keep someone happy ?
Ok, so -n remains..
> Give me one day to see whether it is doable ?
Given above 1.1.* probably works already, but
it does not address the ANY bind - but do you
even wish that to be addressed?
--
// Janne
On Mon, May 5, 2008 at 11:58 AM, J. Bruce Fields <[email protected]> wrote:
> > So the let the user worry approach. Statd does not
> > notify anyone and user is supposed to do it with
> > sm-notify manually.
>
> We should modify distro startup scripts to run sm-notify and add -L to
> statd, following the recommendations in nfs-utils/README.
>
> That will leave the -v option to sm-notify, which will need to be added
> somehow to handle the highly-available-nfs case.
Humm ... imho this does not add anything to current
1.1.* tree. Exactly same things happen when -n is
being used.
> > This will work, but -n looks cleaner me..
>
> I've lost the thread here a little. Let me try to summarize, and tell
> me what I've forgotten:
>
> So we've got multiple nfs servers that may serve the same export. Only
> one serves a given export at a time.
>
> To keep the clients from having to understand this, we tell them to
> mount a single "floating" ip address, and when we want to use a
> different server, we move that floating ip address, and the clients
> automatically follow it. The only problem is that the new server
> doesn't know about locks held on the old server. We solve that problem
> by telling clients that the server has rebooted, and that they must
> reclaim locks.
>
> To make that work, we need to ensure that statd notifies the clients
> that "floating-ip" has rebooted. (They will get confused if they are
> notified that "some-other-ip" has rebooted, where "some-other-ip" is
> just some random ip that the new server happens to use on a different
> interface.)
Exactly right.
> The statd -n option has actually *never* worked, in any previous version
> of nfs-utils--with it specified, statd can no longer communicate with
> the nfs server on the same host. That means the server won't find out
> when a client reboots. (But I guess even the broken -n may still have
> been sufficient to allow clients to monitor the server and find out when
> the server reboots.)
-n may work as is in 1.1.* tree:
a) It will bind to ANY, so kernel lockd will see packets coming
in from LO. 1.0.x bound the -n interface which will break.
b) sm-notify is run with -v and correct name (that will bind to
correct IP and use the right name)
So the only thing missing would be to limit the port visibility
of long-standing sockets; but this should probably be
discussed in another thread if you think it's worth it?
--
// Janne
On Mon, May 05, 2008 at 12:42:49PM -0400, Janne Karhunen wrote:
> On Mon, May 5, 2008 at 11:58 AM, J. Bruce Fields <[email protected]> wrote:
> > The statd -n option has actually *never* worked, in any previous version
> > of nfs-utils--with it specified, statd can no longer communicate with
> > the nfs server on the same host. That means the server won't find out
> > when a client reboots. (But I guess even the broken -n may still have
> > been sufficient to allow clients to monitor the server and find out when
> > the server reboots.)
>
> -n may work as is in 1.1.* tree:
Oh, right, I'd forgotten about that change.
> a) It will bind to ANY, so kernel lockd will see packets coming
> in from LO. 1.0.x bound the -n interface which will break.
>
> b) sm-notify is run with -v and correct name (that will bind to
> correct IP and use the right name)
>
> So the only thing missing would be to limit the port visibility
> of long-standing sockets; but this should probably be
> discussed in another thread if you think it's worth it?
Is the only justification just to limit the consequences if a remote
exploit is found in statd?
If so, iptables seems the better solution, since it provides a uniform
way of limiting the exposure of the ports for all the various
nfs-related (and other) services, as opposed to just being a one-off
thing for statd.
--b.
On Mon, May 5, 2008 at 1:02 PM, J. Bruce Fields <[email protected]> wrote:
> > So the only thing missing would be to limit the port visibility
> > of long-standing sockets; but this should probably be
> > discussed in another thread if you think it's worth it?
>
> Is the only justification just to limit the consequences if a remote
> exploit is found in statd?
It will also make it a LOT easier to debug and understand.
Discussions like this would have never existed given that
binds would have been specific.
--
// Janne
Hi,
So effectively, it makes me sleep better. With it:
- I can rely on clients identifying the server correctly,
- I'm not exposing out anything that is not needed,
- Can tell by the address what this traffic is,
- Can be sure that packets are sent out via right interface
It might be even better if it would exit if -n is used when
no such interface is actually available. As I did it, it still
gambles here just as before.
--
// Janne
On Thu, May 1, 2008 at 8:57 AM, Janne Karhunen <[email protected]> wrote:
> On Tue, Apr 29, 2008 at 12:16 PM, J. Bruce Fields <[email protected]> wrote:
>
> > > Do we really have to add so many lines of the code just to fix "statd
> > > -n"
>
> Which is why I offered the small patch initially; it was
> mentioned that intrusiveness does not matter if it
> can be kept in userspace.
>
>
>
> > > ? Maybe we should go back to the basics by understanding the
> > > requirement of this command ? So why do we need it (i.e. what kind of
> > > bad things we'll see if we don't fix this) ? Some short description
> > > would help.
> >
> > I recall two reasons for -n given in this thread; I think one was just
> > security (maybe you don't want statd listening on some ports, for
> > whatever reason. The other was a code comment quoted here:
>
> That being one..
>
>
>
> > http://marc.info/?l=linux-nfs&m=120854237320424&w=2
> >
> >
> > "This is required to support clients that ignore the mon_name in
> > the statd protocol but use the source address from the request
> > packet."
>
> This another, and the third the fact that this way mon_name
> stays the same on server failover to node that has different
> name. It identifies the server instance..
>
>
>
> > Which I don't completely understand. I guess it was meant as a way to
> > ensure that *outgoing* packets are sent from the correct (floating) ip
> > address?
>
> Right.
>
>
> --
> // Janne
>
Janne Karhunen wrote:
> Hi,
>
> So effectively, it makes me sleep better. With it:
> - I can rely on clients identifying the server correctly,
> - I'm not exposing out anything that is not needed,
> - Can tell by the address what this traffic is,
> - Can be sure that packets are sent out via right interface
>
> It might be even better if it would exit if -n is used when
> no such interface is actually available. As I did it, it still
> gambles here just as before.
>
>
>
Yep, understood. Linux NFS server failover was one of my painful
projects (actually it was "the" one - so much work and so little result).
But the flow you proposed confuses me. Reading the code right now (to
refresh my memory).. Will get back to the list soon.
-- Wendy
On Thu, May 1, 2008 at 9:50 AM, Wendy Cheng <[email protected]> wrote:
> > It might be even better if it would exit if -n is used when
> > no such interface is actually available. As I did it, it still
> > gambles here just as before.
> >
> Yep, understood. Linux NFS server failover was one of my painful projects
> (actually it was "the" one - so much work and so little result).
Heh
> But the flow you proposed confuses me. Reading the code right now (to
> refresh my memory).. Will get back to the list soon.
Patch no.1 was better imho..
To be frank, given that it has to be done this way
I would probably make it even more intrusive by
having 'clean looking' socket pool. Not that it
matters much in real life terms as there are only
two sockets, but at least it would look more
readable..
--
// Janne
Janne Karhunen wrote:
> Hi,
>
> So effectively, it makes me sleep better. With it:
> - I can rely on clients identifying the server correctly,
> - I'm not exposing out anything that is not needed,
> - Can tell by the address what this traffic is,
> - Can be sure that packets are sent out via right interface
>
> It might be even better if it would exit if -n is used when
> no such interface is actually available. As I did it, it still
> gambles here just as before.
>
>
After browsing thru "statd -n" flow, it is still not clear what will
happen if there are more than 2 interfaces used to export NFS shares ?
Using "statd -H", together with patches described in:
https://www.redhat.com/archives/cluster-devel/2007-April/msg00028.html ,
our cluster failover (with 4 IP interfaces per server) seemed to run
well without troubles. Note that 2/3 of the patch in 4-3 can be removed
*now* since it deals with moving server address from network header into
lockd internal structures - another similar patch (by Frank van
Maarseveen) was accepted into mainline kernel after our patch that has
the required functionality:
http://lkml.org/lkml/2007/7/10/553 .
So the following is our (-H) flow:
* Server dispatches statd with "-N" option that has a user mode script
(sample program fotest.c enclosed). It is expected the user mode script
could structure its nlm directory accordingly.
* Upon failover, the take-over server notifies clients with:
"/usr/sbin/sm-notify -f -v floating_ip_address -P an_sm_directory"
The advantages of "-H" approach over "-n" are (I think ?):
* It can handle multiple NFS export network interfaces.
* It knows which clients coming from which interfaces to allow selective
grace period for each interface.
In many ways, I would think "-n" should be obsolete ?
-- Wendy
Wendy Cheng wrote:
> Janne Karhunen wrote:
>> Hi,
>>
>> So effectively, it makes me sleep better. With it:
>> - I can rely on clients identifying the server correctly,
>> - I'm not exposing out anything that is not needed,
>> - Can tell by the address what this traffic is,
>> - Can be sure that packets are sent out via right interface
>>
>> It might be even better if it would exit if -n is used when
>> no such interface is actually available. As I did it, it still
>> gambles here just as before.
>>
>>
>
> After browsing thru "statd -n" flow, it is still not clear what will
> happen if there are more than 2 interfaces used to export NFS shares ?
> Using "statd -H", together with patches described in:
> https://www.redhat.com/archives/cluster-devel/2007-April/msg00028.html ,
>
> our cluster failover (with 4 IP interfaces per server) seemed to run
> well without troubles. Note that 2/3 of the patch in 4-3 can be
> removed *now* since it deals with moving server address from network
> header into lockd internal structures - another similar patch (by
> Frank van Maarseveen) was accepted into mainline kernel after our
> patch that has the required functionality:
> http://lkml.org/lkml/2007/7/10/553 .
>
> So the following is our (-H) flow:
> * Server dispatches statd with "-N" option that has a user mode script
obvious typo here .. "Server dispatches statd with "-H" option that has
a uwer mode script"
-- Wendy
On Fri, May 2, 2008 at 11:21 AM, Wendy Cheng <[email protected]> wrote:
> After browsing thru "statd -n" flow, it is still not clear what will happen
> if there are more than 2 interfaces used to export NFS shares ?
How come? It will use the interface specified with -n.
> Using "statd -H", together with patches described in:
> https://www.redhat.com/archives/cluster-devel/2007-April/msg00028.html ,
So this basically makes it a users problem, I don't
like it. Statd's standard notifications are just fine and
I don't want to have anything to do with the process.
It should work as is, out of the box, without writing
separate programs to handle stuff that statd doesn't
do.
> our cluster failover (with 4 IP interfaces per server) seemed to run well
> without troubles.
Your idea was to serve traffic via all of these interfaces?
One specific segment is just enough for us. Our servers
can have anything from 5 - 100 floating addresses and
it would be just great if we could keep each service
bound in it's own address. It's just better that way.
> Note that 2/3 of the patch in 4-3 can be removed *now*
> since it deals with moving server address from network header into lockd
> internal structures - another similar patch (by Frank van Maarseveen) was
> accepted into mainline kernel after our patch that has the required
> functionality:
> http://lkml.org/lkml/2007/7/10/553 .
>
> So the following is our (-H) flow:
> * Server dispatches statd with "-N" option that has a user mode script
> (sample program fotest.c enclosed). It is expected the user mode script
> could structure its nlm directory accordingly.
> * Upon failover, the take-over server notifies clients with:
> "/usr/sbin/sm-notify -f -v floating_ip_address -P an_sm_directory"
>
> The advantages of "-H" approach over "-n" are (I think ?):
> * It can handle multiple NFS export network interfaces.
> * It knows which clients coming from which interfaces to allow selective
> grace period for each interface.
>
> In many ways, I would think "-n" should be obsolete ?
To me these use cases are clearly different. You're
trying to serve traffic to multiple segments and need
stuff that 'user have to worry about' to accomplish
this. -n works as is for just one segment. And how
many users really need interface specific selective
grace anyway?
--
// Janne
On Fri, May 2, 2008 at 5:13 PM, Janne Karhunen <[email protected]> wrote:
> > After browsing thru "statd -n" flow, it is still not clear what will happen
> > if there are more than 2 interfaces used to export NFS shares ?
>
> How come? It will use the interface specified with -n.
Ah, you were talking about rest of the services. Yeah,
we would need to go this through one patch at a time.
--
// Janne
Janne Karhunen wrote:
> On Fri, May 2, 2008 at 11:21 AM, Wendy Cheng <[email protected]> wrote:
>
>
>> After browsing thru "statd -n" flow, it is still not clear what will happen
>> if there are more than 2 interfaces used to export NFS shares ?
>>
>
> How come? It will use the interface specified with -n.
>
ok, just read your follow-on email .. so we can cross out this one.
>
>
>> Using "statd -H", together with patches described in:
>> https://www.redhat.com/archives/cluster-devel/2007-April/msg00028.html ,
>>
>
> So this basically makes it a users problem, I don't
> like it. Statd's standard notifications are just fine and
> I don't want to have anything to do with the process.
> It should work as is, out of the box, without writing
> separate programs to handle stuff that statd doesn't
> do.
>
I think you mis-understand our approach. With our patch, you do *not*
need to do anything if you have only one single (floating) interface to
do nfs export (and you can have the machine configured to use other
hostname). The statd's "my_name" is correctly filled (from kernel by our
patch) - it is no longer bound to 127.0.0.1.
Maybe I mislead you in previous post. The "-H" is only an "example" - to
show people how to selectively move an ip address around without
affecting other co-existing nfs ip interface on the same server. I did
plan to submit a complete user mode patch so the program after "-H" will
have a default executable (but user still can use their own if they
don't like our nlm directory structure).
>
>
>> our cluster failover (with 4 IP interfaces per server) seemed to run well
>> without troubles.
>>
>
> Your idea was to serve traffic via all of these interfaces?
>
yes ...
> One specific segment is just enough for us.
I know .. but there *are* users and live systems *today* that have
multiple nfs interfaces. For big-fat server, I normally saw 4.
> Our servers
> can have anything from 5 - 100 floating addresses and
> it would be just great if we could keep each service
> bound in it's own address. It's just better that way.
>
> To me these use cases are clearly different. You're
> trying to serve traffic to multiple segments and need
> stuff that 'user have to worry about' to accomplish
> this. -n works as is for just one segment. And how
> many users really need interface specific selective
> grace anyway?
>
>
>
I hope above clears the confusion ? Again, admin(s) do *not* need to do
anything if there is only one single (floating) nfs export IP address.
On the other hand, we did see systems had many nfs interfaces (I was
surprised too). And that was exactly the rationale to kick off this work
three years ago .
-- Wendy
On Fri, May 2, 2008 at 6:33 PM, Wendy Cheng <[email protected]> wrote:
> > So this basically makes it a users problem, I don't
> > like it. Statd's standard notifications are just fine and
> > I don't want to have anything to do with the process.
> > It should work as is, out of the box, without writing
> > separate programs to handle stuff that statd doesn't
> > do.
> >
> I think you mis-understand our approach. With our patch, you do *not* need
> to do anything if you have only one single (floating) interface to do nfs
> export (and you can have the machine configured to use other hostname). The
> statd's "my_name" is correctly filled (from kernel by our patch) - it is no
> longer bound to 127.0.0.1.
What happens when it fails over to node that has
different name? Mon_name in NOTIFY changes
while address it's coming from does not. Which
one should client believe - the address or the
name?
But OK, I probably need to dig out the patch to
have a look.
> > One specific segment is just enough for us.
>
> I know .. but there *are* users and live systems *today* that have multiple
> nfs interfaces. For big-fat server, I normally saw 4.
I'm still a bit confused - does your patch make it
sure that all the extra N+1 IP addresses in the
node have nothing to do with NFS?
If not, these still seem like two different use cases,
ie. making the service tightly bound to one service
address instead of making it work right with many.
--
// Janne
On Fri, May 2, 2008 at 6:33 PM, Wendy Cheng <[email protected]> wrote:
> Maybe I mislead you in previous post. The "-H" is only an "example" - to
> show people how to selectively move an ip address around without affecting
> other co-existing nfs ip interface on the same server. I did plan to submit
> a complete user mode patch so the program after "-H" will have a default
> executable (but user still can use their own if they don't like our nlm
> directory structure).
Even if you ship the default executable user would
still need to go and edit it (add the address). Well,
I guess it would not be that much harder than
specifying the -n, but somehow this does seem
more arcane (for simple setups).
--
// Janne