2003-11-12 09:53:44

by Erik Thiele

[permalink] [raw]
Subject: trouble with file locking

Hi.

I am using as a fileserver

ii nfs-kernel-ser 1.0-2woody1 Kernel NFS server support
ii kernel-image-2 2.4.18-5 Linux kernel image for version 2.4.18 on AMD
ii nfs-common 1.0-2woody1 NFS support files common to client and serve

on a AMD K6-2/400 with 128MB Ram.

the server has / mounted as ext2.

i am exporting to old hp-ux machines a subdirectory of / like this:

/Archiv xxx.unix.yyy-intern(rw,no_root_squash)

i also export my homedirectory to my linux woody debian machines like this:

/home/erik str.blafoo-intern(rw,no_root_squash)


i have written a test-program like this:

#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>

int main()
{
int h=open("delmelockfile", O_WRONLY|O_CREAT, 0700);
struct flock l;
memset(&l,0,sizeof l);
l.l_type=F_WRLCK;
l.l_whence=SEEK_SET;
l.l_start=0;
l.l_len=0;
fcntl(h,F_SETLK,&l);
sleep(100);
return 0;
}

i strace it, and it gets a no lock availiable error on the fcntl.
i use ethereal to analyze traffic and get a V4 LOCK CALL with the right inode,
and a V4 xxxxxx (i forgot to write) which has a NLM_DENIED or NLM_NO_LOCKS or something like that.

using rpcinfo i found nlockmgr running.
now if i restart the rpc.statd (!!!) suddenly it works and i get locks granted packages (NLM_GRANTED)

someone please enlighten me :)

cu
erik


-------------------------------------------------------
This SF.Net email sponsored by: ApacheCon 2003,
16-19 November in Las Vegas. Learn firsthand the latest
developments in Apache, PHP, Perl, XML, Java, MySQL,
WebDAV, and more! http://www.apachecon.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2003-11-19 20:02:54

by David Dougall

[permalink] [raw]
Subject: Re: trouble with file locking

Is this test program run on a HP-UX client or a linux client? My
experience with HP-UX nfs client is that locking is very flaky if it even
works at all. You might try flock instead of fcntl. That seems to work
better. You might also, if you have a support contract, check out HP-UX
nfs patches. They might suddenly make it work.
--David Dougall


On Wed, 12 Nov 2003, Erik Thiele wrote:

> Hi.
>
> I am using as a fileserver
>
> ii nfs-kernel-ser 1.0-2woody1 Kernel NFS server support
> ii kernel-image-2 2.4.18-5 Linux kernel image for version 2.4.18 on AMD
> ii nfs-common 1.0-2woody1 NFS support files common to client and serve
>
> on a AMD K6-2/400 with 128MB Ram.
>
> the server has / mounted as ext2.
>
> i am exporting to old hp-ux machines a subdirectory of / like this:
>
> /Archiv xxx.unix.yyy-intern(rw,no_root_squash)
>
> i also export my homedirectory to my linux woody debian machines like this:
>
> /home/erik str.blafoo-intern(rw,no_root_squash)
>
>
> i have written a test-program like this:
>
> #include <unistd.h>
> #include <stdio.h>
> #include <fcntl.h>
> #include <string.h>
>
> int main()
> {
> int h=open("delmelockfile", O_WRONLY|O_CREAT, 0700);
> struct flock l;
> memset(&l,0,sizeof l);
> l.l_type=F_WRLCK;
> l.l_whence=SEEK_SET;
> l.l_start=0;
> l.l_len=0;
> fcntl(h,F_SETLK,&l);
> sleep(100);
> return 0;
> }
>
> i strace it, and it gets a no lock availiable error on the fcntl.
> i use ethereal to analyze traffic and get a V4 LOCK CALL with the right inode,
> and a V4 xxxxxx (i forgot to write) which has a NLM_DENIED or NLM_NO_LOCKS or something like that.
>
> using rpcinfo i found nlockmgr running.
> now if i restart the rpc.statd (!!!) suddenly it works and i get locks granted packages (NLM_GRANTED)
>
> someone please enlighten me :)
>
> cu
> erik
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: ApacheCon 2003,
> 16-19 November in Las Vegas. Learn firsthand the latest
> developments in Apache, PHP, Perl, XML, Java, MySQL,
> WebDAV, and more! http://www.apachecon.com/
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>
>
>


______________________________________
Inflex Virus Scanner - installed on mailserver for domain @et.byu.edu
Queries to: [email protected]


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-19 20:21:38

by Trond Myklebust

[permalink] [raw]
Subject: Re: trouble with file locking

>>>>> " " == David Dougall <[email protected]> writes:

> Is this test program run on a HP-UX client or a linux client?
> My experience with HP-UX nfs client is that locking is very
> flaky if it even works at all. You might try flock instead of
> fcntl. That seems to work better. You might also, if you have
> a support contract, check out HP-UX nfs patches. They might
> suddenly make it work. --David Dougall

flock() doesn't work at all over NFS in Linux. It just does local
locking.

Cheers,
Trond


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-20 06:06:15

by Erik Thiele

[permalink] [raw]
Subject: Re: trouble with file locking

On 19 Nov 2003 15:21:31 -0500
Trond Myklebust <[email protected]> wrote:

> >>>>> " " == David Dougall <[email protected]> writes:
>
> > Is this test program run on a HP-UX client or a linux client?
> > My experience with HP-UX nfs client is that locking is very
> > flaky if it even works at all. You might try flock instead of
> > fcntl. That seems to work better. You might also, if you have
> > a support contract, check out HP-UX nfs patches. They might
> > suddenly make it work. --David Dougall
>

the program is run on a linux client. the hp-ux clients i only told you
about because they _might_ be disturbing the system in a way that the
other clients don't work anymore.

> flock() doesn't work at all over NFS in Linux. It just does local
> locking.

well, but i saw ethereal showimg me the nfs locking packets.


cu
erik

--
Erik Thiele


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-20 06:15:15

by Trond Myklebust

[permalink] [raw]
Subject: Re: trouble with file locking

>>>>> " " == Erik Thiele <[email protected]> writes:

>> flock() doesn't work at all over NFS in Linux. It just does
>> local locking.

> well, but i saw ethereal showimg me the nfs locking packets.

Then you have been using POSIX locks (i.e. fcntl()/lockf()). They
should work fine.


I agree, though, with David's assertion that the fact you are running
against HP-UX may be significant. They have historically had a
reputation for a poor implementation of the NFS lock manager protocol.

Cheers,
Trond


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-20 06:50:43

by Erik Thiele

[permalink] [raw]
Subject: Re: trouble with file locking

On Thu, 20 Nov 2003 01:14:26 -0500
Trond Myklebust <[email protected]> wrote:

> >>>>> " " == Erik Thiele <[email protected]> writes:
>
> >> flock() doesn't work at all over NFS in Linux. It just does
> >> local locking.
>
> > well, but i saw ethereal showimg me the nfs locking packets.
>
> Then you have been using POSIX locks (i.e. fcntl()/lockf()). They
> should work fine.

from my original mail:


int main()
{
int h=open("delmelockfile", O_WRONLY|O_CREAT, 0700);
struct flock l;
memset(&l,0,sizeof l);
l.l_type=F_WRLCK;
l.l_whence=SEEK_SET;
l.l_start=0;
l.l_len=0;
fcntl(h,F_SETLK,&l);
sleep(100);
return 0;
}

>
>
> I agree, though, with David's assertion that the fact you are running
> against HP-UX may be significant. They have historically had a
> reputation for a poor implementation of the NFS lock manager protocol.
>

from my original mail:

i am exporting to old hp-ux machines a subdirectory of / like this:
/Archiv xxx.unix.yyy-intern(rw,no_root_squash)

i also export my homedirectory to my linux woody debian machines like
this:
/home/erik str.blafoo-intern(rw,no_root_squash)


.. so the question is, can different exports interfere with each other
with locking???

can one lock files on a readonly export? (then this would be a security
problem, because if i mount an anonymous readonly export with hp-ux, the
other exports will fail to lock.


again, what i don't understand is that a restart of the statd daemon
fixes the problem forever.



cu
erik

--
Erik Thiele


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-20 07:37:37

by Trond Myklebust

[permalink] [raw]
Subject: Re: trouble with file locking

>>>>> " " == Erik Thiele <[email protected]> writes:

> from my original mail:


I know... I was responding to the suggestion that you use flock()
instead. That is a bad idea.

> from my original mail:

> i am exporting to old hp-ux machines a subdirectory of / like
> this: /Archiv xxx.unix.yyy-intern(rw,no_root_squash)

> i also export my homedirectory to my linux woody debian
> machines like this: /home/erik
> str.blafoo-intern(rw,no_root_squash)


> .. so the question is, can different exports interfere with
> each other with locking???

If the server is a linux box, then yes. I have a patch from Patrice
Dumas that fixes a bug w.r.t. 2 clients waiting on a lock. The current
code allows those waits to interfere with one another.
See

http://www.fys.uio.no/~trondmy/src/Linux-2.4.x/2.4.23-rc1/linux-2.4.23-04-fix_lockd3.dif

> can one lock files on a readonly export? (then this would be a
> security problem, because if i mount an anonymous readonly
> export with hp-ux, the other exports will fail to lock.


> again, what i don't understand is that a restart of the statd
> daemon fixes the problem forever.

That signals to the server that the client has rebooted (or vice
versa) and tells it to clear all locks (start lock recovery).

Cheers,
Trond


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-24 14:24:37

by Matt Bernstein

[permalink] [raw]
Subject: Re: trouble with file locking

On Nov 20 Trond Myklebust wrote:

>Then you have been using POSIX locks (i.e. fcntl()/lockf()). They
>should work fine.

Possibly unrelated [hence CC trimmed]:

I can't get Debian testing to log in to GNOME when home directories are
mounted over NFS (Linux 2.6-mm client -> Linux 2.4-ac server). It claims
not to be able to lock a file..

I tried writing a trivial program to test lockf() and it returns ENOLCK
over NFS, but succeeds locally. The client kernel offers some grumbles:

RPC: Can't bind to reserved port (13).
RPC: can't bind to reserved port.
nsm_mon_unmon: rpc failed, status=-5
lockd: cannot monitor a.b.c.d
lockd: failed to monitor a.b.c.d

[where a.b.c.d is our NFS server]

/sbin/rpc.statd is running on both client and server, and with a 2.4
kernel on the client (as the only change) GNOME logins work just fine.

Any ideas? I'm stumped at this point.

Cheers,

Matt


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-11-27 15:28:16

by Matt Bernstein

[permalink] [raw]
Subject: Re: trouble with file locking

On Nov 24 mb/[email protected] wrote:

>I tried writing a trivial program to test lockf() and it returns ENOLCK
>over NFS, but succeeds locally. The client kernel offers some grumbles:
>
>RPC: Can't bind to reserved port (13).
>RPC: can't bind to reserved port.
>nsm_mon_unmon: rpc failed, status=-5

solved over on LKML; solution: "modprobe capability"


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs