2007-12-07 02:20:58

by Erez Zadok

[permalink] [raw]
Subject: NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

I get a "permission denied" when trying to mount a localhost nfsv2/3
exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:

# mount -t ext2 /dev/hdb1 /n/lower/b0
# exportfs -o no_root_squash,rw localhost:/n/lower/b0
# mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt

Erez.


2007-12-07 02:55:21

by J. Bruce Fields

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

On Thu, Dec 06, 2007 at 09:20:41PM -0500, Erez Zadok wrote:
> I get a "permission denied" when trying to mount a localhost nfsv2/3
> exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
>
> # mount -t ext2 /dev/hdb1 /n/lower/b0
> # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt

What do you see if you watch the network traffic in ethereal?

--b.

2007-12-07 10:55:14

by Mikael Pettersson

[permalink] [raw]
Subject: Re: NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

On Thu, 6 Dec 2007 21:20:41 -0500, Erez Zadok wrote:
> I get a "permission denied" when trying to mount a localhost nfsv2/3
> exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
>
> # mount -t ext2 /dev/hdb1 /n/lower/b0
> # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt

I'm seeing something similar too. NFSv3 export of an ext3 partition
to another machine in my lan fails (client gets permission denied)
when the server runs 2.6.24-rc4. It worked fine in 2.6.24-rc3.

There's no NFSv4 of any kind on either client or server.

2007-12-07 15:32:48

by J. Bruce Fields

[permalink] [raw]
Subject: Re: NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

On Fri, Dec 07, 2007 at 11:54:38AM +0100, Mikael Pettersson wrote:
> On Thu, 6 Dec 2007 21:20:41 -0500, Erez Zadok wrote:
> > I get a "permission denied" when trying to mount a localhost nfsv2/3
> > exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> > It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
> >
> > # mount -t ext2 /dev/hdb1 /n/lower/b0
> > # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> > # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt
>
> I'm seeing something similar too. NFSv3 export of an ext3 partition
> to another machine in my lan fails (client gets permission denied)
> when the server runs 2.6.24-rc4. It worked fine in 2.6.24-rc3.
>
> There's no NFSv4 of any kind on either client or server.

And you're not varying the client at all, you're only changing the
kernel version on the server?

There are literally no commits between v2.6.24-rc3 and v2.6.24-rc4 which
touch fs/nfsd/. What filesystem are you exporting? Are the nfs-utils
versions the same in both cases?

Also, could you get a network trace showing the failure?

--b.

2007-12-07 20:01:45

by Erez Zadok

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

In message <[email protected]>, "J. Bruce Fields" writes:
> On Thu, Dec 06, 2007 at 09:20:41PM -0500, Erez Zadok wrote:
> > I get a "permission denied" when trying to mount a localhost nfsv2/3
> > exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> > It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
> >
> > # mount -t ext2 /dev/hdb1 /n/lower/b0
> > # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> > # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt
>
> What do you see if you watch the network traffic in ethereal?
>
> --b.

Bruce, I'm using nfs-utils-1.0.10-14.fc6 on an FC6 system with all latest
FC6 patches. Using git-bisect I was able to find the patch which broke it:

commit 2b1e300a9dfc3196ccddf6f1d74b91b7af55e416
Author: Eric W. Biederman <[email protected]>
Date: Sun Dec 2 00:33:17 2007 +1100

[NETNS]: Fix /proc/net breakage

Well I clearly goofed when I added the initial network namespace support
for /proc/net. Currently things work but there are odd details visible to
user space, even when we have a single network namespace.

Since we do not cache proc_dir_entry dentries at the moment we can just
modify ->lookup to return a different directory inode depending on the
network namespace of the process looking at /proc/net, replacing the
current technique of using a magic and fragile follow_link method.

To accomplish that this patch:
- introduces a shadow_proc method to allow different dentries to
be returned from proc_lookup.
- Removes the old /proc/net follow_link magic
- Fixes a weakness in our not caching of proc generic dentries.

As shadow_proc uses a task struct to decided which dentry to return we can
go back later and fix the proc generic caching without modifying any code
that uses the shadow_proc method.

Signed-off-by: Eric W. Biederman <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Pavel Machek <[email protected]>
Cc: Pavel Emelyanov <[email protected]>
Cc: "David S. Miller" <[email protected]>
Cc: Ingo Molnar <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>

With the above patch, rpc.mountd is unable to open /proc/fs/nfsd/filehandle.
Strace shows:

open("/proc/fs/nfsd/filehandle", O_RDWR|O_LARGEFILE) = -1 ENOENT (No such file or directory)

Without the above patch, /proc/fs/nfsd is populated with a number of files,
including "filehandle".

Erez.

2007-12-07 20:05:39

by Erez Zadok

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

In message <[email protected]>, "J. Bruce Fields" writes:
> On Thu, Dec 06, 2007 at 09:20:41PM -0500, Erez Zadok wrote:
> > I get a "permission denied" when trying to mount a localhost nfsv2/3
> > exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> > It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
> >
> > # mount -t ext2 /dev/hdb1 /n/lower/b0
> > # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> > # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt
>
> What do you see if you watch the network traffic in ethereal?
>
> --b.

Bruce, I'm using nfs-utils-1.0.10-14.fc6 on an FC6 system with all latest
FC6 patches. Using git-bisect I was able to find the patch which broke it:

commit 2b1e300a9dfc3196ccddf6f1d74b91b7af55e416
Author: Eric W. Biederman <[email protected]>
Date: Sun Dec 2 00:33:17 2007 +1100

[NETNS]: Fix /proc/net breakage

Well I clearly goofed when I added the initial network namespace support
for /proc/net. Currently things work but there are odd details visible to
user space, even when we have a single network namespace.

Since we do not cache proc_dir_entry dentries at the moment we can just
modify ->lookup to return a different directory inode depending on the
network namespace of the process looking at /proc/net, replacing the
current technique of using a magic and fragile follow_link method.

To accomplish that this patch:
- introduces a shadow_proc method to allow different dentries to
be returned from proc_lookup.
- Removes the old /proc/net follow_link magic
- Fixes a weakness in our not caching of proc generic dentries.

As shadow_proc uses a task struct to decided which dentry to return we can
go back later and fix the proc generic caching without modifying any code
that uses the shadow_proc method.

Signed-off-by: Eric W. Biederman <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Pavel Machek <[email protected]>
Cc: Pavel Emelyanov <[email protected]>
Cc: "David S. Miller" <[email protected]>
Cc: Ingo Molnar <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>

With the above patch, rpc.mountd is unable to open /proc/fs/nfsd/filehandle.
Strace shows:

open("/proc/fs/nfsd/filehandle", O_RDWR|O_LARGEFILE) = -1 ENOENT (No such file or directory)

Without the above patch, /proc/fs/nfsd is populated with a number of files,
including "filehandle".

Erez.

2007-12-07 20:07:22

by J. Bruce Fields

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

On Fri, Dec 07, 2007 at 03:00:13PM -0500, Erez Zadok wrote:
> In message <[email protected]>, "J. Bruce Fields" writes:
> > On Thu, Dec 06, 2007 at 09:20:41PM -0500, Erez Zadok wrote:
> > > I get a "permission denied" when trying to mount a localhost nfsv2/3
> > > exported volume, on v2.6.24-rc4-124-gf194d13. It works w/ nfsv4 mounting.
> > > It worked fine in 2.6.24-rc3. Here's a sequence of ops I tried:
> > >
> > > # mount -t ext2 /dev/hdb1 /n/lower/b0
> > > # exportfs -o no_root_squash,rw localhost:/n/lower/b0
> > > # mount -t nfs -o nfsvers=3 localhost:/n/lower/b0 /mnt
> >
> > What do you see if you watch the network traffic in ethereal?
> >
> > --b.
>
> Bruce, I'm using nfs-utils-1.0.10-14.fc6 on an FC6 system with all latest
> FC6 patches. Using git-bisect I was able to find the patch which broke it:

Wow, thanks for your work finding that.

>
> commit 2b1e300a9dfc3196ccddf6f1d74b91b7af55e416
> Author: Eric W. Biederman <[email protected]>
> Date: Sun Dec 2 00:33:17 2007 +1100
>
> [NETNS]: Fix /proc/net breakage
>
> Well I clearly goofed when I added the initial network namespace support
> for /proc/net. Currently things work but there are odd details visible to
> user space, even when we have a single network namespace.
>
> Since we do not cache proc_dir_entry dentries at the moment we can just
> modify ->lookup to return a different directory inode depending on the
> network namespace of the process looking at /proc/net, replacing the
> current technique of using a magic and fragile follow_link method.
>
> To accomplish that this patch:
> - introduces a shadow_proc method to allow different dentries to
> be returned from proc_lookup.
> - Removes the old /proc/net follow_link magic
> - Fixes a weakness in our not caching of proc generic dentries.
>
> As shadow_proc uses a task struct to decided which dentry to return we can
> go back later and fix the proc generic caching without modifying any code
> that uses the shadow_proc method.
>
> Signed-off-by: Eric W. Biederman <[email protected]>
> Cc: "Rafael J. Wysocki" <[email protected]>
> Cc: Pavel Machek <[email protected]>
> Cc: Pavel Emelyanov <[email protected]>
> Cc: "David S. Miller" <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Signed-off-by: Andrew Morton <[email protected]>
> Signed-off-by: Herbert Xu <[email protected]>
>
> With the above patch, rpc.mountd is unable to open /proc/fs/nfsd/filehandle.
> Strace shows:
>
> open("/proc/fs/nfsd/filehandle", O_RDWR|O_LARGEFILE) = -1 ENOENT (No such file or directory)
>
> Without the above patch, /proc/fs/nfsd is populated with a number of files,
> including "filehandle".

Those files are actually in a separate filesystem (of type "nfsd") which
is supposed to be mounted on /proc/fs/nfsd/. So that mount must have
failed in the bad case? It's not immediately obvious to me what this
patch has to do with that. Hm.

--b.

2007-12-07 20:33:06

by Erez Zadok

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

In message <[email protected]>, "J. Bruce Fields" writes:
> On Fri, Dec 07, 2007 at 03:00:13PM -0500, Erez Zadok wrote:
[...]
> Those files are actually in a separate filesystem (of type "nfsd") which
> is supposed to be mounted on /proc/fs/nfsd/. So that mount must have
> failed in the bad case? It's not immediately obvious to me what this
> patch has to do with that. Hm.

Yes, it is indeed a separate mount in both cases, but in the broken case,
/proc/fs/nfsd is empty.

The patch in question introduces a proc ->d_revalidate method which does
this:

static int proc_revalidate_dentry(struct dentry *dentry, struct nameidata *nd)
{
d_drop(dentry);
return 0;
}

I'm not sure why it drops the dentry and then returns OK to the VFS; is it
to force the VFS to revalidate the dentry? In that case, I think it should
return -ESTALE. I also don't know why /proc needs a ->d_revalidate in the
first place (it was fine up until now). Perhaps what proc does now is
correct, but its behavior has changed such that nfsd's /proc/fs/nfsd needs
to do something different (like grab an extra dentry ref?).

Anyway, if I comment out the d_drop line in proc_revalidate_dentry, or
remove proc's ->d_revalidate method, nfs exporting works again.

Someone more familiar with this patch and /proc should investigate. Until
then, nfsv2/3 exporting are broken in 2.6.24-rc4.

> --b.

Cheers,
Erez.

2007-12-07 20:39:37

by Valdis Klētnieks

[permalink] [raw]
Subject: Re: [NFS] NFSv2/3 broken exporting/mounting (permission denied) in 2.6.24-rc4

On Fri, 07 Dec 2007 15:06:55 EST, "J. Bruce Fields" said:
> On Fri, Dec 07, 2007 at 03:00:13PM -0500, Erez Zadok wrote:

> > commit 2b1e300a9dfc3196ccddf6f1d74b91b7af55e416
> > Author: Eric W. Biederman <[email protected]>
> > Date: Sun Dec 2 00:33:17 2007 +1100

> Those files are actually in a separate filesystem (of type "nfsd") which
> is supposed to be mounted on /proc/fs/nfsd/. So that mount must have
> failed in the bad case? It's not immediately obvious to me what this
> patch has to do with that. Hm.

For what it's worth, the startup scripts for 'wine' and 'jexec' fail as well,
and at shutdown, I get complaints about being unable to dismount
/proc/sys/fs/binfmt_misc. 'strace' says:

open("/proc/sys/fs/binfmt_misc/windows", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 EN
OENT (No such file or directory)

# grep binfmt /proc/mounts
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0

I have to wonder if the fact there's 2 of them mounted is related to the
fact that 2 startup scripts tried to touch files under fs/binfmt_misc/ ?


Attachments:
(No filename) (226.00 B)