2006-05-26 18:46:15

by Frank Filz

[permalink] [raw]
Subject: Problems with fh_fsid_type=3 exports (device minor id > 255)

I having a problem with the 2.6.9 kernel and mounting file systems with
a minor device id > 255 where the mount fails. It seems like the file
handle is never cached. We are able to proceed if fsid is specified
in /etc/exports. nfs-utils version is 1.0.6.

Have others seen this problem and is there a fix in a later kernel?

I've been trying to follow the fh_compose and fh_verify code to
understand this, but I don't see where there's a difference between type
3 file handles and type 1 file handles (fsid specified in /etc/exports)
that could cause a problem. When I add a little bit of debug messages, I
see that expkey_parse is being called as a result of a system call for
the type 1 handles, but not for the type 3 handles.

Thanks

Frank Filz




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2006-05-31 22:46:06

by Frank Filz

[permalink] [raw]
Subject: Re: Problems with fh_fsid_type=3 exports (device minor id > 255)

On Fri, 2006-05-26 at 11:48 -0700, Frank Filz wrote:
> I having a problem with the 2.6.9 kernel and mounting file systems with
> a minor device id > 255 where the mount fails. It seems like the file
> handle is never cached. We are able to proceed if fsid is specified
> in /etc/exports. nfs-utils version is 1.0.6.
>
> Have others seen this problem and is there a fix in a later kernel?
>
> I've been trying to follow the fh_compose and fh_verify code to
> understand this, but I don't see where there's a difference between type
> 3 file handles and type 1 file handles (fsid specified in /etc/exports)
> that could cause a problem. When I add a little bit of debug messages, I
> see that expkey_parse is being called as a result of a system call for
> the type 1 handles, but not for the type 3 handles.

I've trudged my way through the code, and I think I've found the
problem. The problem seems to be the way the kernel and nfs-utils expect
the fsid to be coded. The kernel uses this code to encode:

unsigned major = MAJOR(dev);
unsigned minor = MINOR(dev);
return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);

The user space uses this code to decode:

memcpy(&dev, fsid, 4);
memcpy(&inode, fsid+4, 4);
major = (dev & 0xfff00) >> 8;
minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);

I think when the entry is exported into the kernel, the fsid is also
going to be wrong.

The question I have is should I fix this in the kernel or in nfs-utils?

Thanks

Frank Filz




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-05-31 23:12:13

by Frank Filz

[permalink] [raw]
Subject: Re: Problems with fh_fsid_type=3 exports (device minor id > 255)

On Wed, 2006-05-31 at 15:48 -0700, Frank Filz wrote:
> On Fri, 2006-05-26 at 11:48 -0700, Frank Filz wrote:
> > I having a problem with the 2.6.9 kernel and mounting file systems with
> > a minor device id > 255 where the mount fails. It seems like the file
> > handle is never cached. We are able to proceed if fsid is specified
> > in /etc/exports. nfs-utils version is 1.0.6.
> >
> > Have others seen this problem and is there a fix in a later kernel?
> >
> > I've been trying to follow the fh_compose and fh_verify code to
> > understand this, but I don't see where there's a difference between type
> > 3 file handles and type 1 file handles (fsid specified in /etc/exports)
> > that could cause a problem. When I add a little bit of debug messages, I
> > see that expkey_parse is being called as a result of a system call for
> > the type 1 handles, but not for the type 3 handles.
>
> I've trudged my way through the code, and I think I've found the
> problem. The problem seems to be the way the kernel and nfs-utils expect
> the fsid to be coded. The kernel uses this code to encode:
>
> unsigned major = MAJOR(dev);
> unsigned minor = MINOR(dev);
> return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
>
> The user space uses this code to decode:
>
> memcpy(&dev, fsid, 4);
> memcpy(&inode, fsid+4, 4);
> major = (dev & 0xfff00) >> 8;
> minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);

Oops, I'm horribly confused... The above code actually is right, well
mostly, I think it actually should be:

> major = (dev & 0xff00) >> 8;
> minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);

The bug I'm chasing is because the install is using nfs-utils 1.0.6
which does not handle type 3 file handles at all.

Frank




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-06-01 03:34:05

by NeilBrown

[permalink] [raw]
Subject: Re: Problems with fh_fsid_type=3 exports (device minor id > 255)

On Wednesday May 31, [email protected] wrote:
>
> Oops, I'm horribly confused... The above code actually is right, well

:-) I'm glad you got your problem sorted!

> mostly, I think it actually should be:
>
> > major = (dev & 0xff00) >> 8;
> > minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
>

No, major is 12 bits, minor is 20 bits. So the original code is
correct.

NeilBrown


> The bug I'm chasing is because the install is using nfs-utils 1.0.6
> which does not handle type 3 file handles at all.
>
> Frank
>
>
>
>
> -------------------------------------------------------
> All the advantages of Linux Managed Hosting--Without the Cost and Risk!
> Fully trained technicians. The highest number of Red Hat certifications in
> the hosting industry. Fanatical Support. Click to learn more
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-06-01 15:20:37

by Frank Filz

[permalink] [raw]
Subject: Re: Problems with fh_fsid_type=3 exports (device minor id > 255)

On Thu, 2006-06-01 at 13:33 +1000, Neil Brown wrote:
> On Wednesday May 31, [email protected] wrote:
> >
> > Oops, I'm horribly confused... The above code actually is right, well
>
> :-) I'm glad you got your problem sorted!
>
> > mostly, I think it actually should be:
> >
> > > major = (dev & 0xff00) >> 8;
> > > minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
> >
>
> No, major is 12 bits, minor is 20 bits. So the original code is
> correct.

With a bit more sleep, I see that now. Thanks.

What version of nfs-utils did this show up in? It looks like the
situation is just a mis-match of nfs-utils and kernel.

Thanks

Frank Filz




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2006-06-01 17:48:31

by Frank Filz

[permalink] [raw]
Subject: [PATCH] expkey_parse doesn't handle type 3 exports (was Re: [NFS] Problems with fh_fsid_type=3 exports (device minor id > 255))

On Thu, 2006-06-01 at 08:23 -0700, Frank Filz wrote:
> On Thu, 2006-06-01 at 13:33 +1000, Neil Brown wrote:
> > On Wednesday May 31, [email protected] wrote:
> > >
> > > Oops, I'm horribly confused... The above code actually is right, well
> >
> > :-) I'm glad you got your problem sorted!
> >
> > > mostly, I think it actually should be:
> > >
> > > > major = (dev & 0xff00) >> 8;
> > > > minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
> > >
> >
> > No, major is 12 bits, minor is 20 bits. So the original code is
> > correct.
>
> With a bit more sleep, I see that now. Thanks.
>
> What version of nfs-utils did this show up in? It looks like the
> situation is just a mis-match of nfs-utils and kernel.

I tried the 1.0.8 version of rpc.mountd, and things almost worked, but
the mount still failed. I looked at expkey_parse, and discovered it
doesn't handle type 3 exports. The following patch fixes the problem:

Signed-Off-By: Frank Filz <[email protected]>

diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -132,7 +132,7 @@ static int expkey_parse(struct cache_det
if (*ep)
goto out;
dprintk("found fsidtype %d\n", fsidtype);
- if (fsidtype > 2)
+ if (fsidtype > 3)
goto out;
if ((len=qword_get(&mesg, buf, PAGE_SIZE)) <= 0)
goto out;




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs