2003-05-07 15:47:47

by Jeremy Sanders

[permalink] [raw]
Subject: Permission denied

I've seen this problem a couple of times (I think I reported it to the
list before but I can find the posting). Using the automounter to mount
home directories (rw,v3,rsize=8192,wsize=8192,hard,intr,udp,lock), it
seems that rebooting a Linux server can give a "Permission denied" on
the client. Further attempts at accessing the mountpoint give this error.
It can be fixed my umounting it (if nothing is using it), and reaccessing
the mountpoint.

The server is running RedHat's 2.4.18-27.7.x kernel, and the client is
using 2.4.18-17.7.x (unfortunately).

dmesg on the client shows:

nfs: server server.ast.cam.ac.uk not responding, still trying
nfs: task 57889 can't get a request slot
nfs: server server.ast.cam.ac.uk OK
nfs: server server.ast.cam.ac.uk OK

sendmail gave errors such as:

May 7 16:07:39 xserv1 sendmail[22255]: h47EwYl22254: forward
/home/gmorris: transient error: Timeout on file open
May 7 16:07:39 xserv1 sendmail[22195]: h47Eljl22194: forward
/home/gmorris: transient error: Timeout on file open
May 7 16:07:39 xserv1 sendmail[22255]: h47EwYl22254: forward
/home/gmorris/.forward: Permission denied
May 7 16:07:39 xserv1 sendmail[22195]: h47Eljl22194: forward
/home/gmorris/.forward: Permission denied

Any ideas? I suppose it's hard to diagnose this without a network traffic
analysis, and we're not using the standard kernel either. I suppose it
could be an automounter bug.


Jeremy

--
Jeremy Sanders <[email protected]> http://www-xray.ast.cam.ac.uk/~jss/
X-Ray Group, Institute of Astronomy, University of Cambridge, UK.
Public Key Server PGP Key ID: E1AAE053


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
http://www.enterpriselinuxforum.com

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2003-05-07 16:22:34

by James Pearson

[permalink] [raw]
Subject: Re: Permission denied

What version of nfs-utils are you running on the server?

I had a problem if the server crashed before a client mount could be
recorded to /var/lib/nfs/rmtab, the client got permission denied on the
mount point after the server rebooted - see:

http://marc.theaimsgroup.com/?l=linux-nfs&m=101828003630443&w=2

This is fixed in the more recent versions of nfs-utils.

Also, if you manually attempt to umount an NFS file system (automounted
or not), but the umount fails because the file system is busy - this
will cause a problem as rpc.mountd on the server has already removed the
client from /var/lib/nfs/rmtab - which means if the server reboots, the
server has no record of the client mount, so you get permission denied
on client when it accesses the mount point ... see:

http://marc.theaimsgroup.com/?l=linux-nfs&m=104558814532107&w=2

I had this problem - my work round is not to manually umount automounted
file systems ... (or, if I have to, use a hacked version of umount that
doesn't contact rpc.mountd on the server).

James Pearson

Jeremy Sanders wrote:
>
> I've seen this problem a couple of times (I think I reported it to the
> list before but I can find the posting). Using the automounter to mount
> home directories (rw,v3,rsize=8192,wsize=8192,hard,intr,udp,lock), it
> seems that rebooting a Linux server can give a "Permission denied" on
> the client. Further attempts at accessing the mountpoint give this error.
> It can be fixed my umounting it (if nothing is using it), and reaccessing
> the mountpoint.
>
> The server is running RedHat's 2.4.18-27.7.x kernel, and the client is
> using 2.4.18-17.7.x (unfortunately).
>
> dmesg on the client shows:
>
> nfs: server server.ast.cam.ac.uk not responding, still trying
> nfs: task 57889 can't get a request slot
> nfs: server server.ast.cam.ac.uk OK
> nfs: server server.ast.cam.ac.uk OK
>
> sendmail gave errors such as:
>
> May 7 16:07:39 xserv1 sendmail[22255]: h47EwYl22254: forward
> /home/gmorris: transient error: Timeout on file open
> May 7 16:07:39 xserv1 sendmail[22195]: h47Eljl22194: forward
> /home/gmorris: transient error: Timeout on file open
> May 7 16:07:39 xserv1 sendmail[22255]: h47EwYl22254: forward
> /home/gmorris/.forward: Permission denied
> May 7 16:07:39 xserv1 sendmail[22195]: h47Eljl22194: forward
> /home/gmorris/.forward: Permission denied
>
> Any ideas? I suppose it's hard to diagnose this without a network traffic
> analysis, and we're not using the standard kernel either. I suppose it
> could be an automounter bug.
>
> Jeremy
>
> --
> Jeremy Sanders <[email protected]> http://www-xray.ast.cam.ac.uk/~jss/
> X-Ray Group, Institute of Astronomy, University of Cambridge, UK.
> Public Key Server PGP Key ID: E1AAE053
>
> -------------------------------------------------------
> Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
> The only event dedicated to issues related to Linux enterprise solutions
> http://www.enterpriselinuxforum.com
>
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
http://www.enterpriselinuxforum.com

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-05-07 16:32:41

by Jeremy Sanders

[permalink] [raw]
Subject: Re: Permission denied

On Wed, 7 May 2003, James Pearson wrote:

> What version of nfs-utils are you running on the server?

nfs-utils-0.3.3 on both. From the thread this looks like it has the bug
(unless RedHat patched it in their rpm). I'll have a go updating it and
see whether we still get the bug.

> Also, if you manually attempt to umount an NFS file system (automounted
> or not), but the umount fails because the file system is busy - this
> will cause a problem as rpc.mountd on the server has already removed the
> client from /var/lib/nfs/rmtab - which means if the server reboots, the
> server has no record of the client mount, so you get permission denied
> on client when it accesses the mount point ... see:

That's interesting. Thanks!

Jeremy

--
Jeremy Sanders <[email protected]> http://www-xray.ast.cam.ac.uk/~jss/
X-Ray Group, Institute of Astronomy, University of Cambridge, UK.
Public Key Server PGP Key ID: E1AAE053


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
http://www.enterpriselinuxforum.com

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-05-07 16:31:50

by Trond Myklebust

[permalink] [raw]
Subject: Re: Permission denied

>>>>> " " == Jeremy Sanders <[email protected]> writes:

> I've seen this problem a couple of times (I think I reported it
> to the list before but I can find the posting). Using the
> automounter to mount home directories
> (rw,v3,rsize=8192,wsize=8192,hard,intr,udp,lock), it seems that
> rebooting a Linux server can give a "Permission denied" on the
> client.

Can it be duplicated on newer kernels?

Which automounter BTW?

Cheers,
Trond


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
http://www.enterpriselinuxforum.com

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2003-05-07 16:35:03

by Jeremy Sanders

[permalink] [raw]
Subject: Re: Permission denied

On Wed, 7 May 2003, Trond Myklebust wrote:

> Can it be duplicated on newer kernels?

I'm not sure. I couldn't reproduce the problem when rebooting machines by
hand, and I don't think I can do much testing with the production
machines as I can't easily upgrade the kernel. I'll invesitgate this.

> Which automounter BTW?

autofs-3.1.7

Thanks, Jeremy

--
Jeremy Sanders <[email protected]> http://www-xray.ast.cam.ac.uk/~jss/
X-Ray Group, Institute of Astronomy, University of Cambridge, UK.
Public Key Server PGP Key ID: E1AAE053


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
http://www.enterpriselinuxforum.com

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-02-20 12:16:47

by Daniel Bramkamp

[permalink] [raw]
Subject: Re: Permission denied

Quoting Frank van Maarseveen <[email protected]>:

> On Tue, Feb 07, 2006 at 12:34:48PM +0100, Daniel Bramkamp wrote:
>> Hi list,
>>
>> I am experiencing some issues with a NFS server running on Linux Kernel
>> 2.4.26 with nfs-utils-1.0.6 installed. Basically it works great,
>> however, sometimes access to files and directories stored on the server
>> is denied for a certain user account. I am not sure if it affects other
>> users as well - only heard complaints from that particular user so far.
>
> A bit late but.. is that user a member of more than 16 groups?

Hi,

yesterday I reinstalled the server. The server is now running Gentoo
instead of Slackware. The installed software is up to date now. Kernel
is 2.6.14, nfs-utils is still 1.0.6, it seems 1.0.8 is currently a
realease candidate. Half an hour ago I got a call from the user - the
problem still exists. The user is a member of 13 groups. I really hoped
the update would solve the problem since I have no idea how to fix this.

--
Daniel Bramkamp

team! datentechnik GmbH & Co.KG
Werner von Siemens Stra=DFe 12a
49124 Georgsmarienh=FCtte
Tel.: 05401-8226-0
Fax : 05401-8226-20
eMail: [email protected]


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-02-20 14:32:01

by Trond Myklebust

[permalink] [raw]
Subject: Re: Permission denied

On Mon, 2006-02-20 at 13:16 +0100, Daniel Bramkamp wrote:
> Quoting Frank van Maarseveen <[email protected]>:
>
> > On Tue, Feb 07, 2006 at 12:34:48PM +0100, Daniel Bramkamp wrote:
> >> Hi list,
> >>
> >> I am experiencing some issues with a NFS server running on Linux Kernel
> >> 2.4.26 with nfs-utils-1.0.6 installed. Basically it works great,
> >> however, sometimes access to files and directories stored on the server
> >> is denied for a certain user account. I am not sure if it affects other
> >> users as well - only heard complaints from that particular user so far.
> >
> > A bit late but.. is that user a member of more than 16 groups?
>
> Hi,
>
> yesterday I reinstalled the server. The server is now running Gentoo
> instead of Slackware. The installed software is up to date now. Kernel
> is 2.6.14, nfs-utils is still 1.0.6, it seems 1.0.8 is currently a
> realease candidate. Half an hour ago I got a call from the user - the
> problem still exists. The user is a member of 13 groups. I really hoped
> the update would solve the problem since I have no idea how to fix this.

You have to look at the client machine too.

Is the user still limited to 13 groups on the client, and do the
numerical uid/gid values match on the client and server?

Cheers,
Trond



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-02-20 15:43:32

by Daniel Bramkamp

[permalink] [raw]
Subject: Re: Permission denied

Quoting Trond Myklebust <[email protected]>:


> You have to look at the client machine too.

yeah, the client in this case is a terminal server.

> Is the user still limited to 13 groups on the client, and do the
> numerical uid/gid values match on the client and server?

Since I use LDAP to store user accounts the uid and gid is the same on
all servers. The user is not a member of any local group on any server
either so the total number of groups the particular user is in is 13 on
all servers. I just double checked that.

--
Daniel Bramkamp

team! datentechnik GmbH & Co.KG
Werner von Siemens Stra=DFe 12a
49124 Georgsmarienh=FCtte
Tel.: 05401-8226-0
Fax : 05401-8226-20
eMail: [email protected]


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs