The following simple program creates files with un-initialized mtime values
on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
mtimes are wildly different each time the program is run. The problem is
reproducible but does not happen under any of the following circumstances:
1) Drop O_EXCL from open() call.
2) NFS mount using v3.
3) Switch NFS v4 client from Linux to Solaris.
This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
and nfs-utils 1.0.9-16 backported from CentOS 5.
Thanks.
> cat badmtime.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
main() {
int fd;
fd = open("badtime.out", O_CREAT|O_EXCL, 0664);
close(fd);
}
> vi badmtime.c
> gcc badmtime.c -o badmtime
> ./badmtime
> ls -l badmtime.out
-rw-r--r-- 1 anderson anderson 0 Oct 9 1979 badmtime.out
> stat badmtime.out
File: `badmtime.out'
Size: 0 Blocks: 1 IO Block: 1048576 regular empty file
Device: 20h/32d Inode: 107016 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 5026/anderson) Gid: ( 5026/anderson)
Access: 2007-06-27 16:07:24.184162655 -0700
Modify: 1979-10-09 12:54:39.549017000 -0700
Change: 2007-06-27 16:07:24.192792428 -0700
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Unfortunately this patch did not have any observable affects.
On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
> On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
> > The following simple program creates files with un-initialized mtime values
> > on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
> > mtimes are wildly different each time the program is run. The problem is
> > reproducible but does not happen under any of the following circumstances:
> >
> > 1) Drop O_EXCL from open() call.
> > 2) NFS mount using v3.
> > 3) Switch NFS v4 client from Linux to Solaris.
> >
> > This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
> > revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
> > and nfs-utils 1.0.9-16 backported from CentOS 5.
> >
> > Thanks.
>
> Doesn't Jeff's patch fix it?
>
> Trond
>
>
> From: Jeff Layton <[email protected]>
> Date: Tue, 5 Jun 2007 14:49:03 -0400
> NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
> verifier
> Subject: No Subject
>
> The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
> call and so it doesn't bother to reset any fields that may be holding
> the verifier. This patch has us save the first two words of the bitmask
> (which is all the current client has #defines for). The client then
> later checks this bitmask and turns on the appropriate flags in the
> sattr->ia_verify field for the following SETATTR call.
>
> This patch only currently checks to see if the server used the atime
> and mtime slots for the verifier (which is what the Linux server uses
> for this). I'm not sure of what other fields the server could
> reasonably use, but adding checks for others should be trivial.
>
> Signed-off-by: Jeff Layton <[email protected]>
> Signed-off-by: Trond Myklebust <[email protected]>
> ---
>
> fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
> fs/nfs/nfs4xdr.c | 9 +++++++--
> include/linux/nfs4.h | 1 +
> include/linux/nfs_xdr.h | 1 +
> 4 files changed, 29 insertions(+), 2 deletions(-)
>
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index 3cc7544..fee2d14 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
> }
>
> /*
> + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
> + * fields corresponding to attributes that were used to store the verifier.
> + * Make sure we clobber those fields in the later setattr call
> + */
> +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
> +{
> + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
> + !(sattr->ia_valid & ATTR_ATIME_SET))
> + sattr->ia_valid |= ATTR_ATIME;
> +
> + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
> + !(sattr->ia_valid & ATTR_MTIME_SET))
> + sattr->ia_valid |= ATTR_MTIME;
> +}
> +
> +/*
> * Returns a referenced nfs4_state
> */
> static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
> @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
> if (status != 0)
> goto err_opendata_free;
>
> + if (opendata->o_arg.open_flags & O_EXCL)
> + nfs4_exclusive_attrset(opendata, sattr);
> +
> status = -ENOMEM;
> state = nfs4_opendata_to_nfs4_state(opendata);
> if (state == NULL)
> @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
> status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
> if (status == 0)
> nfs_setattr_update_inode(state->inode, sattr);
> + nfs_post_op_update_inode(state->inode, &fattr);
> }
> if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
> status = nfs4_intent_set_file(nd, &path, state);
> diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> index 8003c91..5efd314 100644
> --- a/fs/nfs/nfs4xdr.c
> +++ b/fs/nfs/nfs4xdr.c
> @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
> static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> {
> __be32 *p;
> - uint32_t bmlen;
> + uint32_t savewords, bmlen, i;
> int status;
>
> status = decode_op_hdr(xdr, OP_OPEN);
> @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> goto xdr_error;
>
> READ_BUF(bmlen << 2);
> - p += bmlen;
> + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
> + for (i = 0; i < savewords; ++i)
> + READ32(res->attrset[i]);
> +
> + p += (bmlen - savewords);
> +
> return decode_delegation(xdr, res);
> xdr_error:
> dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
> diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> index 7e7f33a..8726491 100644
> --- a/include/linux/nfs4.h
> +++ b/include/linux/nfs4.h
> @@ -15,6 +15,7 @@
>
> #include <linux/types.h>
>
> +#define NFS4_BITMAP_SIZE 2
> #define NFS4_VERIFIER_SIZE 8
> #define NFS4_STATEID_SIZE 16
> #define NFS4_FHSIZE 128
> diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> index 10c26ed..f7100df 100644
> --- a/include/linux/nfs_xdr.h
> +++ b/include/linux/nfs_xdr.h
> @@ -144,6 +144,7 @@ struct nfs_openres {
> nfs4_stateid delegation;
> __u32 do_recall;
> __u64 maxsize;
> + __u32 attrset[NFS4_BITMAP_SIZE];
> };
>
> /*
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
More precisely, applying this patch to the 2.6.20.14 kernel plus the
revalidate-the-fsid patch did not result in any changes. Does the
O_EXCL patch require some other supporting patches relative to 2.6.20.14?
or perhaps it is necessary to rebase to a newer kernel before applying it?
even tough it applies, builds and runs with 2.6.20.14?
Thanks.
On Wed, Jun 27, 2007 at 05:49:04PM -0700, Stuart Anderson wrote:
> Unfortunately this patch did not have any observable affects.
>
> On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
> > On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
> > > The following simple program creates files with un-initialized mtime values
> > > on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
> > > mtimes are wildly different each time the program is run. The problem is
> > > reproducible but does not happen under any of the following circumstances:
> > >
> > > 1) Drop O_EXCL from open() call.
> > > 2) NFS mount using v3.
> > > 3) Switch NFS v4 client from Linux to Solaris.
> > >
> > > This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
> > > revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
> > > and nfs-utils 1.0.9-16 backported from CentOS 5.
> > >
> > > Thanks.
> >
> > Doesn't Jeff's patch fix it?
> >
> > Trond
> >
> >
>
> > From: Jeff Layton <[email protected]>
> > Date: Tue, 5 Jun 2007 14:49:03 -0400
> > NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
> > verifier
> > Subject: No Subject
> >
> > The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
> > call and so it doesn't bother to reset any fields that may be holding
> > the verifier. This patch has us save the first two words of the bitmask
> > (which is all the current client has #defines for). The client then
> > later checks this bitmask and turns on the appropriate flags in the
> > sattr->ia_verify field for the following SETATTR call.
> >
> > This patch only currently checks to see if the server used the atime
> > and mtime slots for the verifier (which is what the Linux server uses
> > for this). I'm not sure of what other fields the server could
> > reasonably use, but adding checks for others should be trivial.
> >
> > Signed-off-by: Jeff Layton <[email protected]>
> > Signed-off-by: Trond Myklebust <[email protected]>
> > ---
> >
> > fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
> > fs/nfs/nfs4xdr.c | 9 +++++++--
> > include/linux/nfs4.h | 1 +
> > include/linux/nfs_xdr.h | 1 +
> > 4 files changed, 29 insertions(+), 2 deletions(-)
> >
> > diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> > index 3cc7544..fee2d14 100644
> > --- a/fs/nfs/nfs4proc.c
> > +++ b/fs/nfs/nfs4proc.c
> > @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
> > }
> >
> > /*
> > + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
> > + * fields corresponding to attributes that were used to store the verifier.
> > + * Make sure we clobber those fields in the later setattr call
> > + */
> > +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
> > +{
> > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
> > + !(sattr->ia_valid & ATTR_ATIME_SET))
> > + sattr->ia_valid |= ATTR_ATIME;
> > +
> > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
> > + !(sattr->ia_valid & ATTR_MTIME_SET))
> > + sattr->ia_valid |= ATTR_MTIME;
> > +}
> > +
> > +/*
> > * Returns a referenced nfs4_state
> > */
> > static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
> > @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
> > if (status != 0)
> > goto err_opendata_free;
> >
> > + if (opendata->o_arg.open_flags & O_EXCL)
> > + nfs4_exclusive_attrset(opendata, sattr);
> > +
> > status = -ENOMEM;
> > state = nfs4_opendata_to_nfs4_state(opendata);
> > if (state == NULL)
> > @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
> > status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
> > if (status == 0)
> > nfs_setattr_update_inode(state->inode, sattr);
> > + nfs_post_op_update_inode(state->inode, &fattr);
> > }
> > if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
> > status = nfs4_intent_set_file(nd, &path, state);
> > diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> > index 8003c91..5efd314 100644
> > --- a/fs/nfs/nfs4xdr.c
> > +++ b/fs/nfs/nfs4xdr.c
> > @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
> > static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > {
> > __be32 *p;
> > - uint32_t bmlen;
> > + uint32_t savewords, bmlen, i;
> > int status;
> >
> > status = decode_op_hdr(xdr, OP_OPEN);
> > @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > goto xdr_error;
> >
> > READ_BUF(bmlen << 2);
> > - p += bmlen;
> > + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
> > + for (i = 0; i < savewords; ++i)
> > + READ32(res->attrset[i]);
> > +
> > + p += (bmlen - savewords);
> > +
> > return decode_delegation(xdr, res);
> > xdr_error:
> > dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
> > diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> > index 7e7f33a..8726491 100644
> > --- a/include/linux/nfs4.h
> > +++ b/include/linux/nfs4.h
> > @@ -15,6 +15,7 @@
> >
> > #include <linux/types.h>
> >
> > +#define NFS4_BITMAP_SIZE 2
> > #define NFS4_VERIFIER_SIZE 8
> > #define NFS4_STATEID_SIZE 16
> > #define NFS4_FHSIZE 128
> > diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> > index 10c26ed..f7100df 100644
> > --- a/include/linux/nfs_xdr.h
> > +++ b/include/linux/nfs_xdr.h
> > @@ -144,6 +144,7 @@ struct nfs_openres {
> > nfs4_stateid delegation;
> > __u32 do_recall;
> > __u64 maxsize;
> > + __u32 attrset[NFS4_BITMAP_SIZE];
> > };
> >
> > /*
>
>
> --
> Stuart Anderson [email protected]
> http://www.ligo.caltech.edu/~anderson
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, 27 Jun 2007 17:59:57 -0700
Stuart Anderson <[email protected]> wrote:
> More precisely, applying this patch to the 2.6.20.14 kernel plus the
> revalidate-the-fsid patch did not result in any changes. Does the
> O_EXCL patch require some other supporting patches relative to 2.6.20.14?
>
> or perhaps it is necessary to rebase to a newer kernel before applying it?
> even tough it applies, builds and runs with 2.6.20.14?
>
> Thanks.
>
With Linux, both client and server side were broken in this respect.
The server didn't set the bitmask in the reply and the client didn't
look at it anyway. It's possible that Solaris server is broken in this
regard as well. It would be interesting to see a capture here,
particularly one containing the reply from the CREATE.
Exactly what version of Solaris is the server running?
>
> On Wed, Jun 27, 2007 at 05:49:04PM -0700, Stuart Anderson wrote:
> > Unfortunately this patch did not have any observable affects.
> >
> > On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
> > > On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
> > > > The following simple program creates files with un-initialized mtime values
> > > > on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
> > > > mtimes are wildly different each time the program is run. The problem is
> > > > reproducible but does not happen under any of the following circumstances:
> > > >
> > > > 1) Drop O_EXCL from open() call.
> > > > 2) NFS mount using v3.
> > > > 3) Switch NFS v4 client from Linux to Solaris.
> > > >
> > > > This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
> > > > revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
> > > > and nfs-utils 1.0.9-16 backported from CentOS 5.
> > > >
> > > > Thanks.
> > >
> > > Doesn't Jeff's patch fix it?
> > >
> > > Trond
> > >
> > >
> >
> > > From: Jeff Layton <[email protected]>
> > > Date: Tue, 5 Jun 2007 14:49:03 -0400
> > > NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
> > > verifier
> > > Subject: No Subject
> > >
> > > The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
> > > call and so it doesn't bother to reset any fields that may be holding
> > > the verifier. This patch has us save the first two words of the bitmask
> > > (which is all the current client has #defines for). The client then
> > > later checks this bitmask and turns on the appropriate flags in the
> > > sattr->ia_verify field for the following SETATTR call.
> > >
> > > This patch only currently checks to see if the server used the atime
> > > and mtime slots for the verifier (which is what the Linux server uses
> > > for this). I'm not sure of what other fields the server could
> > > reasonably use, but adding checks for others should be trivial.
> > >
> > > Signed-off-by: Jeff Layton <[email protected]>
> > > Signed-off-by: Trond Myklebust <[email protected]>
> > > ---
> > >
> > > fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
> > > fs/nfs/nfs4xdr.c | 9 +++++++--
> > > include/linux/nfs4.h | 1 +
> > > include/linux/nfs_xdr.h | 1 +
> > > 4 files changed, 29 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> > > index 3cc7544..fee2d14 100644
> > > --- a/fs/nfs/nfs4proc.c
> > > +++ b/fs/nfs/nfs4proc.c
> > > @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
> > > }
> > >
> > > /*
> > > + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
> > > + * fields corresponding to attributes that were used to store the verifier.
> > > + * Make sure we clobber those fields in the later setattr call
> > > + */
> > > +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
> > > +{
> > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
> > > + !(sattr->ia_valid & ATTR_ATIME_SET))
> > > + sattr->ia_valid |= ATTR_ATIME;
> > > +
> > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
> > > + !(sattr->ia_valid & ATTR_MTIME_SET))
> > > + sattr->ia_valid |= ATTR_MTIME;
> > > +}
> > > +
> > > +/*
> > > * Returns a referenced nfs4_state
> > > */
> > > static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
> > > @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
> > > if (status != 0)
> > > goto err_opendata_free;
> > >
> > > + if (opendata->o_arg.open_flags & O_EXCL)
> > > + nfs4_exclusive_attrset(opendata, sattr);
> > > +
> > > status = -ENOMEM;
> > > state = nfs4_opendata_to_nfs4_state(opendata);
> > > if (state == NULL)
> > > @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
> > > status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
> > > if (status == 0)
> > > nfs_setattr_update_inode(state->inode, sattr);
> > > + nfs_post_op_update_inode(state->inode, &fattr);
> > > }
> > > if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
> > > status = nfs4_intent_set_file(nd, &path, state);
> > > diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> > > index 8003c91..5efd314 100644
> > > --- a/fs/nfs/nfs4xdr.c
> > > +++ b/fs/nfs/nfs4xdr.c
> > > @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
> > > static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > {
> > > __be32 *p;
> > > - uint32_t bmlen;
> > > + uint32_t savewords, bmlen, i;
> > > int status;
> > >
> > > status = decode_op_hdr(xdr, OP_OPEN);
> > > @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > goto xdr_error;
> > >
> > > READ_BUF(bmlen << 2);
> > > - p += bmlen;
> > > + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
> > > + for (i = 0; i < savewords; ++i)
> > > + READ32(res->attrset[i]);
> > > +
> > > + p += (bmlen - savewords);
> > > +
> > > return decode_delegation(xdr, res);
> > > xdr_error:
> > > dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
> > > diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> > > index 7e7f33a..8726491 100644
> > > --- a/include/linux/nfs4.h
> > > +++ b/include/linux/nfs4.h
> > > @@ -15,6 +15,7 @@
> > >
> > > #include <linux/types.h>
> > >
> > > +#define NFS4_BITMAP_SIZE 2
> > > #define NFS4_VERIFIER_SIZE 8
> > > #define NFS4_STATEID_SIZE 16
> > > #define NFS4_FHSIZE 128
> > > diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> > > index 10c26ed..f7100df 100644
> > > --- a/include/linux/nfs_xdr.h
> > > +++ b/include/linux/nfs_xdr.h
> > > @@ -144,6 +144,7 @@ struct nfs_openres {
> > > nfs4_stateid delegation;
> > > __u32 do_recall;
> > > __u64 maxsize;
> > > + __u32 attrset[NFS4_BITMAP_SIZE];
> > > };
> > >
> > > /*
> >
> >
> > --
> > Stuart Anderson [email protected]
> > http://www.ligo.caltech.edu/~anderson
>
> --
> Stuart Anderson [email protected]
> http://www.ligo.caltech.edu/~anderson
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, Jun 27, 2007 at 09:15:59PM -0400, Jeff Layton wrote:
> On Wed, 27 Jun 2007 17:59:57 -0700
> Stuart Anderson <[email protected]> wrote:
>
> > More precisely, applying this patch to the 2.6.20.14 kernel plus the
> > revalidate-the-fsid patch did not result in any changes. Does the
> > O_EXCL patch require some other supporting patches relative to 2.6.20.14?
> >
> > or perhaps it is necessary to rebase to a newer kernel before applying it?
> > even tough it applies, builds and runs with 2.6.20.14?
> >
> > Thanks.
> >
>
> With Linux, both client and server side were broken in this respect.
> The server didn't set the bitmask in the reply and the client didn't
> look at it anyway. It's possible that Solaris server is broken in this
> regard as well. It would be interesting to see a capture here,
> particularly one containing the reply from the CREATE.
Possible, but when we switch the client from Linux to Solaris and share
the same filesytem from the same server there are no O_EXCL problems.
I will see if we can capture the bits. Any recomendations on which tool
with what options would be most useful to capture the useful traffic?
>
> Exactly what version of Solaris is the server running?
# cat /etc/release
Solaris 10 11/06 s10x_u3wos_10 X86
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 14 November 2006
# uname -v
Generic_118855-36
Somewhat related is the reason that we are tring NFSV4 on Linux clients is
that with NFSV3 ACL's do not work between Linux and Solaris if the filesystem
on the Solaris server is ZFS, e.g., cp -p will generate interesting error
messages. While I would rather stay with NFSV4 if possible, has anyone
gotten this configuration to work with V3?
>
> >
> > On Wed, Jun 27, 2007 at 05:49:04PM -0700, Stuart Anderson wrote:
> > > Unfortunately this patch did not have any observable affects.
> > >
> > > On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
> > > > On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
> > > > > The following simple program creates files with un-initialized mtime values
> > > > > on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
> > > > > mtimes are wildly different each time the program is run. The problem is
> > > > > reproducible but does not happen under any of the following circumstances:
> > > > >
> > > > > 1) Drop O_EXCL from open() call.
> > > > > 2) NFS mount using v3.
> > > > > 3) Switch NFS v4 client from Linux to Solaris.
> > > > >
> > > > > This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
> > > > > revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
> > > > > and nfs-utils 1.0.9-16 backported from CentOS 5.
> > > > >
> > > > > Thanks.
> > > >
> > > > Doesn't Jeff's patch fix it?
> > > >
> > > > Trond
> > > >
> > > >
> > >
> > > > From: Jeff Layton <[email protected]>
> > > > Date: Tue, 5 Jun 2007 14:49:03 -0400
> > > > NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
> > > > verifier
> > > > Subject: No Subject
> > > >
> > > > The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
> > > > call and so it doesn't bother to reset any fields that may be holding
> > > > the verifier. This patch has us save the first two words of the bitmask
> > > > (which is all the current client has #defines for). The client then
> > > > later checks this bitmask and turns on the appropriate flags in the
> > > > sattr->ia_verify field for the following SETATTR call.
> > > >
> > > > This patch only currently checks to see if the server used the atime
> > > > and mtime slots for the verifier (which is what the Linux server uses
> > > > for this). I'm not sure of what other fields the server could
> > > > reasonably use, but adding checks for others should be trivial.
> > > >
> > > > Signed-off-by: Jeff Layton <[email protected]>
> > > > Signed-off-by: Trond Myklebust <[email protected]>
> > > > ---
> > > >
> > > > fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
> > > > fs/nfs/nfs4xdr.c | 9 +++++++--
> > > > include/linux/nfs4.h | 1 +
> > > > include/linux/nfs_xdr.h | 1 +
> > > > 4 files changed, 29 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> > > > index 3cc7544..fee2d14 100644
> > > > --- a/fs/nfs/nfs4proc.c
> > > > +++ b/fs/nfs/nfs4proc.c
> > > > @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
> > > > }
> > > >
> > > > /*
> > > > + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
> > > > + * fields corresponding to attributes that were used to store the verifier.
> > > > + * Make sure we clobber those fields in the later setattr call
> > > > + */
> > > > +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
> > > > +{
> > > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
> > > > + !(sattr->ia_valid & ATTR_ATIME_SET))
> > > > + sattr->ia_valid |= ATTR_ATIME;
> > > > +
> > > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
> > > > + !(sattr->ia_valid & ATTR_MTIME_SET))
> > > > + sattr->ia_valid |= ATTR_MTIME;
> > > > +}
> > > > +
> > > > +/*
> > > > * Returns a referenced nfs4_state
> > > > */
> > > > static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
> > > > @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
> > > > if (status != 0)
> > > > goto err_opendata_free;
> > > >
> > > > + if (opendata->o_arg.open_flags & O_EXCL)
> > > > + nfs4_exclusive_attrset(opendata, sattr);
> > > > +
> > > > status = -ENOMEM;
> > > > state = nfs4_opendata_to_nfs4_state(opendata);
> > > > if (state == NULL)
> > > > @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
> > > > status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
> > > > if (status == 0)
> > > > nfs_setattr_update_inode(state->inode, sattr);
> > > > + nfs_post_op_update_inode(state->inode, &fattr);
> > > > }
> > > > if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
> > > > status = nfs4_intent_set_file(nd, &path, state);
> > > > diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> > > > index 8003c91..5efd314 100644
> > > > --- a/fs/nfs/nfs4xdr.c
> > > > +++ b/fs/nfs/nfs4xdr.c
> > > > @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > {
> > > > __be32 *p;
> > > > - uint32_t bmlen;
> > > > + uint32_t savewords, bmlen, i;
> > > > int status;
> > > >
> > > > status = decode_op_hdr(xdr, OP_OPEN);
> > > > @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > goto xdr_error;
> > > >
> > > > READ_BUF(bmlen << 2);
> > > > - p += bmlen;
> > > > + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
> > > > + for (i = 0; i < savewords; ++i)
> > > > + READ32(res->attrset[i]);
> > > > +
> > > > + p += (bmlen - savewords);
> > > > +
> > > > return decode_delegation(xdr, res);
> > > > xdr_error:
> > > > dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
> > > > diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> > > > index 7e7f33a..8726491 100644
> > > > --- a/include/linux/nfs4.h
> > > > +++ b/include/linux/nfs4.h
> > > > @@ -15,6 +15,7 @@
> > > >
> > > > #include <linux/types.h>
> > > >
> > > > +#define NFS4_BITMAP_SIZE 2
> > > > #define NFS4_VERIFIER_SIZE 8
> > > > #define NFS4_STATEID_SIZE 16
> > > > #define NFS4_FHSIZE 128
> > > > diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> > > > index 10c26ed..f7100df 100644
> > > > --- a/include/linux/nfs_xdr.h
> > > > +++ b/include/linux/nfs_xdr.h
> > > > @@ -144,6 +144,7 @@ struct nfs_openres {
> > > > nfs4_stateid delegation;
> > > > __u32 do_recall;
> > > > __u64 maxsize;
> > > > + __u32 attrset[NFS4_BITMAP_SIZE];
> > > > };
> > > >
> > > > /*
> > >
> > >
> > > --
> > > Stuart Anderson [email protected]
> > > http://www.ligo.caltech.edu/~anderson
> >
> > --
> > Stuart Anderson [email protected]
> > http://www.ligo.caltech.edu/~anderson
>
>
> --
> Jeff Layton <[email protected]>
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Jun 27, 2007, at 9:53 PM, Stuart Anderson wrote:
> On Wed, Jun 27, 2007 at 09:15:59PM -0400, Jeff Layton wrote:
>> On Wed, 27 Jun 2007 17:59:57 -0700
>> Stuart Anderson <[email protected]> wrote:
>>
>>> More precisely, applying this patch to the 2.6.20.14 kernel plus the
>>> revalidate-the-fsid patch did not result in any changes. Does the
>>> O_EXCL patch require some other supporting patches relative to
>>> 2.6.20.14?
>>>
>>> or perhaps it is necessary to rebase to a newer kernel before
>>> applying it?
>>> even tough it applies, builds and runs with 2.6.20.14?
>>>
>>> Thanks.
>>>
>>
>> With Linux, both client and server side were broken in this respect.
>> The server didn't set the bitmask in the reply and the client didn't
>> look at it anyway. It's possible that Solaris server is broken in
>> this
>> regard as well. It would be interesting to see a capture here,
>> particularly one containing the reply from the CREATE.
>
> Possible, but when we switch the client from Linux to Solaris and
> share
> the same filesytem from the same server there are no O_EXCL problems.
>
> I will see if we can capture the bits. Any recomendations on which
> tool
> with what options would be most useful to capture the useful traffic?
>
>>
>> Exactly what version of Solaris is the server running?
>
> # cat /etc/release
> Solaris 10 11/06 s10x_u3wos_10 X86
> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> Use is subject to license terms.
> Assembled 14 November 2006
> # uname -v
> Generic_118855-36
>
>
> Somewhat related is the reason that we are tring NFSV4 on Linux
> clients is
> that with NFSV3 ACL's do not work between Linux and Solaris if the
> filesystem
> on the Solaris server is ZFS, e.g., cp -p will generate interesting
> error
> messages. While I would rather stay with NFSV4 if possible, has anyone
> gotten this configuration to work with V3?
It will not. ZFS does not support the "old" ACL definition; only
NFSv4 support.
Spencer
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, Jun 27, 2007 at 10:09:23PM -0500, Spencer Shepler wrote:
> >Somewhat related is the reason that we are tring NFSV4 on Linux
> >clients is
> >that with NFSV3 ACL's do not work between Linux and Solaris if the
> >filesystem
> >on the Solaris server is ZFS, e.g., cp -p will generate interesting
> >error
> >messages. While I would rather stay with NFSV4 if possible, has anyone
> >gotten this configuration to work with V3?
>
> It will not. ZFS does not support the "old" ACL definition; only
> NFSv4 support.
>
Understood, so this is probably not a bug and just a feature, but why
does a Linux client try and then generate application level errors?
A Solaris client is willing to NFS mount vers=3 a ZFS filesystem and then
run GNU cp -rp, for example, without complaint. Perhaps there is a simple
method determine if ACL's are available/supported on a particular NFS
mount?
Thanks.
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Jun 27, 2007, at 10:23 PM, Stuart Anderson wrote:
> On Wed, Jun 27, 2007 at 10:09:23PM -0500, Spencer Shepler wrote:
>>> Somewhat related is the reason that we are tring NFSV4 on Linux
>>> clients is
>>> that with NFSV3 ACL's do not work between Linux and Solaris if the
>>> filesystem
>>> on the Solaris server is ZFS, e.g., cp -p will generate interesting
>>> error
>>> messages. While I would rather stay with NFSV4 if possible, has
>>> anyone
>>> gotten this configuration to work with V3?
>>
>> It will not. ZFS does not support the "old" ACL definition; only
>> NFSv4 support.
>>
>
> Understood, so this is probably not a bug and just a feature, but why
> does a Linux client try and then generate application level errors?
> A Solaris client is willing to NFS mount vers=3 a ZFS filesystem
> and then
> run GNU cp -rp, for example, without complaint. Perhaps there is a
> simple
> method determine if ACL's are available/supported on a particular NFS
> mount?
Does it fail on the "get" or "set" of the ACL?
Spencer
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, Jun 27, 2007 at 10:30:22PM -0500, Spencer Shepler wrote:
>
> On Jun 27, 2007, at 10:23 PM, Stuart Anderson wrote:
>
> >On Wed, Jun 27, 2007 at 10:09:23PM -0500, Spencer Shepler wrote:
> >>>Somewhat related is the reason that we are tring NFSV4 on Linux
> >>>clients is
> >>>that with NFSV3 ACL's do not work between Linux and Solaris if the
> >>>filesystem
> >>>on the Solaris server is ZFS, e.g., cp -p will generate interesting
> >>>error
> >>>messages. While I would rather stay with NFSV4 if possible, has
> >>>anyone
> >>>gotten this configuration to work with V3?
> >>
> >>It will not. ZFS does not support the "old" ACL definition; only
> >>NFSv4 support.
> >>
> >
> >Understood, so this is probably not a bug and just a feature, but why
> >does a Linux client try and then generate application level errors?
> >A Solaris client is willing to NFS mount vers=3 a ZFS filesystem
> >and then
> >run GNU cp -rp, for example, without complaint. Perhaps there is a
> >simple
> >method determine if ACL's are available/supported on a particular NFS
> >mount?
>
> Does it fail on the "get" or "set" of the ACL?
set
Here is an strace snippet from running cp -rp on a Linux NFSV3 client
of a Solaris 10 ZFS server:
getxattr("test1/oink", "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x04\x00\x04\x00\xff\xff\xff\xff\x10\x00\x07\x00\xff\xff\xff\xff \x00\x04\x00\xff\xff\xff\xff", 132) = 36
setxattr("test2/oink", "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x04\x00\x04\x00\xff\xff\xff\xff\x10\x00\x07\x00\xff\xff\xff\xff \x00\x04\x00\xff\xff\xff\xff", 36, 0) = -1 EINVAL (Invalid argument)
We used to have this same problem with Linux NFSV3 mounts from Solaris QFS servers
as well, but that was fixed in a recent update to QFS, so I am not really sure
who is "responsible" for bridging ACL's between the old/new schemes.
Thanks.
--
Stuart Anderson [email protected]
http://www.ligo.caltech.edu/~anderson
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Jun 27, 2007, at 10:44 PM, Stuart Anderson wrote:
> On Wed, Jun 27, 2007 at 10:30:22PM -0500, Spencer Shepler wrote:
>>
>> On Jun 27, 2007, at 10:23 PM, Stuart Anderson wrote:
>>
>>> On Wed, Jun 27, 2007 at 10:09:23PM -0500, Spencer Shepler wrote:
>>>>> Somewhat related is the reason that we are tring NFSV4 on Linux
>>>>> clients is
>>>>> that with NFSV3 ACL's do not work between Linux and Solaris if the
>>>>> filesystem
>>>>> on the Solaris server is ZFS, e.g., cp -p will generate
>>>>> interesting
>>>>> error
>>>>> messages. While I would rather stay with NFSV4 if possible, has
>>>>> anyone
>>>>> gotten this configuration to work with V3?
>>>>
>>>> It will not. ZFS does not support the "old" ACL definition; only
>>>> NFSv4 support.
>>>>
>>>
>>> Understood, so this is probably not a bug and just a feature, but
>>> why
>>> does a Linux client try and then generate application level errors?
>>> A Solaris client is willing to NFS mount vers=3 a ZFS filesystem
>>> and then
>>> run GNU cp -rp, for example, without complaint. Perhaps there is a
>>> simple
>>> method determine if ACL's are available/supported on a particular
>>> NFS
>>> mount?
>>
>> Does it fail on the "get" or "set" of the ACL?
>
> set
>
> Here is an strace snippet from running cp -rp on a Linux NFSV3 client
> of a Solaris 10 ZFS server:
>
> getxattr("test1/oink", "system.posix_acl_access", "\x02\x00\x00\x00
> \x01\x00\x06\x00\xff\xff\xff\xff\x04\x00\x04\x00\xff\xff\xff\xff\x10
> \x00\x07\x00\xff\xff\xff\xff \x00\x04\x00\xff\xff\xff\xff", 132) = 36
> setxattr("test2/oink", "system.posix_acl_access", "\x02\x00\x00\x00
> \x01\x00\x06\x00\xff\xff\xff\xff\x04\x00\x04\x00\xff\xff\xff\xff\x10
> \x00\x07\x00\xff\xff\xff\xff \x00\x04\x00\xff\xff\xff\xff", 36, 0)
> = -1 EINVAL (Invalid argument)
>
> We used to have this same problem with Linux NFSV3 mounts from
> Solaris QFS servers
> as well, but that was fixed in a recent update to QFS, so I am not
> really sure
> who is "responsible" for bridging ACL's between the old/new schemes.
The Solaris NFSv3 server will, upon "get acl", map the underlying
ZFS ACL format to the NFSv3/ACL format. However, on setting of
an acl, an error is returned (NOSUP). The decision was made that
because of the imperfect mapping on acl set in this path that
an error would be returned.
I believe the Solaris client would map the ENOTSUP back to the
application as is and that may help the application adapt
its behavior (avoid erroring out if that isn't desired).
Spencer
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, 27 Jun 2007 19:53:27 -0700
Stuart Anderson <[email protected]> wrote:
> On Wed, Jun 27, 2007 at 09:15:59PM -0400, Jeff Layton wrote:
> > On Wed, 27 Jun 2007 17:59:57 -0700
> > Stuart Anderson <[email protected]> wrote:
> >
> > > More precisely, applying this patch to the 2.6.20.14 kernel plus the
> > > revalidate-the-fsid patch did not result in any changes. Does the
> > > O_EXCL patch require some other supporting patches relative to 2.6.20.14?
> > >
> > > or perhaps it is necessary to rebase to a newer kernel before applying it?
> > > even tough it applies, builds and runs with 2.6.20.14?
> > >
> > > Thanks.
> > >
> >
> > With Linux, both client and server side were broken in this respect.
> > The server didn't set the bitmask in the reply and the client didn't
> > look at it anyway. It's possible that Solaris server is broken in this
> > regard as well. It would be interesting to see a capture here,
> > particularly one containing the reply from the CREATE.
>
> Possible, but when we switch the client from Linux to Solaris and share
> the same filesytem from the same server there are no O_EXCL problems.
>
NFSv3 didn't use this bitmask, so clients tended to just clobber the
mtime and atime on the subsequent setattr. It's possible that Solaris
is using the old semantics here with NFSv4. Of course, all of this is
speculation :-), a capture should give us a better clue.
> I will see if we can capture the bits. Any recomendations on which tool
> with what options would be most useful to capture the useful traffic?
>
tcpdump is what I'd recommend. It's pretty standard issue on Linux
boxes these days. From the client:
# tcpdump -i eth0 -s 0 -w /tmp/nfs4_create.pcap host nfs_server and port 2049
You might need to modify it some depending on your setup. that should
capture all NFS packets into /tmp/nfs4_create.pcap. Start up tcpdump,
run the reproducer, and then kill the tcpdump. That should give us a
pretty small capture file that just has what we need in it. If you want
to look at it yourself, then you can use wireshark (aka ethereal) on
Linux.
I have a solaris 11 qemu image. I'll try to reproduce this using it as
a server today sometime.
> >
> > Exactly what version of Solaris is the server running?
>
> # cat /etc/release
> Solaris 10 11/06 s10x_u3wos_10 X86
> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> Use is subject to license terms.
> Assembled 14 November 2006
> # uname -v
> Generic_118855-36
>
>
> Somewhat related is the reason that we are tring NFSV4 on Linux clients is
> that with NFSV3 ACL's do not work between Linux and Solaris if the filesystem
> on the Solaris server is ZFS, e.g., cp -p will generate interesting error
> messages. While I would rather stay with NFSV4 if possible, has anyone
> gotten this configuration to work with V3?
>
> >
> > >
> > > On Wed, Jun 27, 2007 at 05:49:04PM -0700, Stuart Anderson wrote:
> > > > Unfortunately this patch did not have any observable affects.
> > > >
> > > > On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
> > > > > On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
> > > > > > The following simple program creates files with un-initialized mtime values
> > > > > > on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
> > > > > > mtimes are wildly different each time the program is run. The problem is
> > > > > > reproducible but does not happen under any of the following circumstances:
> > > > > >
> > > > > > 1) Drop O_EXCL from open() call.
> > > > > > 2) NFS mount using v3.
> > > > > > 3) Switch NFS v4 client from Linux to Solaris.
> > > > > >
> > > > > > This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
> > > > > > revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
> > > > > > and nfs-utils 1.0.9-16 backported from CentOS 5.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > > Doesn't Jeff's patch fix it?
> > > > >
> > > > > Trond
> > > > >
> > > > >
> > > >
> > > > > From: Jeff Layton <[email protected]>
> > > > > Date: Tue, 5 Jun 2007 14:49:03 -0400
> > > > > NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
> > > > > verifier
> > > > > Subject: No Subject
> > > > >
> > > > > The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
> > > > > call and so it doesn't bother to reset any fields that may be holding
> > > > > the verifier. This patch has us save the first two words of the bitmask
> > > > > (which is all the current client has #defines for). The client then
> > > > > later checks this bitmask and turns on the appropriate flags in the
> > > > > sattr->ia_verify field for the following SETATTR call.
> > > > >
> > > > > This patch only currently checks to see if the server used the atime
> > > > > and mtime slots for the verifier (which is what the Linux server uses
> > > > > for this). I'm not sure of what other fields the server could
> > > > > reasonably use, but adding checks for others should be trivial.
> > > > >
> > > > > Signed-off-by: Jeff Layton <[email protected]>
> > > > > Signed-off-by: Trond Myklebust <[email protected]>
> > > > > ---
> > > > >
> > > > > fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
> > > > > fs/nfs/nfs4xdr.c | 9 +++++++--
> > > > > include/linux/nfs4.h | 1 +
> > > > > include/linux/nfs_xdr.h | 1 +
> > > > > 4 files changed, 29 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> > > > > index 3cc7544..fee2d14 100644
> > > > > --- a/fs/nfs/nfs4proc.c
> > > > > +++ b/fs/nfs/nfs4proc.c
> > > > > @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
> > > > > }
> > > > >
> > > > > /*
> > > > > + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
> > > > > + * fields corresponding to attributes that were used to store the verifier.
> > > > > + * Make sure we clobber those fields in the later setattr call
> > > > > + */
> > > > > +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
> > > > > +{
> > > > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
> > > > > + !(sattr->ia_valid & ATTR_ATIME_SET))
> > > > > + sattr->ia_valid |= ATTR_ATIME;
> > > > > +
> > > > > + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
> > > > > + !(sattr->ia_valid & ATTR_MTIME_SET))
> > > > > + sattr->ia_valid |= ATTR_MTIME;
> > > > > +}
> > > > > +
> > > > > +/*
> > > > > * Returns a referenced nfs4_state
> > > > > */
> > > > > static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
> > > > > @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
> > > > > if (status != 0)
> > > > > goto err_opendata_free;
> > > > >
> > > > > + if (opendata->o_arg.open_flags & O_EXCL)
> > > > > + nfs4_exclusive_attrset(opendata, sattr);
> > > > > +
> > > > > status = -ENOMEM;
> > > > > state = nfs4_opendata_to_nfs4_state(opendata);
> > > > > if (state == NULL)
> > > > > @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
> > > > > status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
> > > > > if (status == 0)
> > > > > nfs_setattr_update_inode(state->inode, sattr);
> > > > > + nfs_post_op_update_inode(state->inode, &fattr);
> > > > > }
> > > > > if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
> > > > > status = nfs4_intent_set_file(nd, &path, state);
> > > > > diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> > > > > index 8003c91..5efd314 100644
> > > > > --- a/fs/nfs/nfs4xdr.c
> > > > > +++ b/fs/nfs/nfs4xdr.c
> > > > > @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > > static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > > {
> > > > > __be32 *p;
> > > > > - uint32_t bmlen;
> > > > > + uint32_t savewords, bmlen, i;
> > > > > int status;
> > > > >
> > > > > status = decode_op_hdr(xdr, OP_OPEN);
> > > > > @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
> > > > > goto xdr_error;
> > > > >
> > > > > READ_BUF(bmlen << 2);
> > > > > - p += bmlen;
> > > > > + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
> > > > > + for (i = 0; i < savewords; ++i)
> > > > > + READ32(res->attrset[i]);
> > > > > +
> > > > > + p += (bmlen - savewords);
> > > > > +
> > > > > return decode_delegation(xdr, res);
> > > > > xdr_error:
> > > > > dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
> > > > > diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> > > > > index 7e7f33a..8726491 100644
> > > > > --- a/include/linux/nfs4.h
> > > > > +++ b/include/linux/nfs4.h
> > > > > @@ -15,6 +15,7 @@
> > > > >
> > > > > #include <linux/types.h>
> > > > >
> > > > > +#define NFS4_BITMAP_SIZE 2
> > > > > #define NFS4_VERIFIER_SIZE 8
> > > > > #define NFS4_STATEID_SIZE 16
> > > > > #define NFS4_FHSIZE 128
> > > > > diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> > > > > index 10c26ed..f7100df 100644
> > > > > --- a/include/linux/nfs_xdr.h
> > > > > +++ b/include/linux/nfs_xdr.h
> > > > > @@ -144,6 +144,7 @@ struct nfs_openres {
> > > > > nfs4_stateid delegation;
> > > > > __u32 do_recall;
> > > > > __u64 maxsize;
> > > > > + __u32 attrset[NFS4_BITMAP_SIZE];
> > > > > };
> > > > >
> > > > > /*
> > > >
> > > >
> > > > --
> > > > Stuart Anderson [email protected]
> > > > http://www.ligo.caltech.edu/~anderson
> > >
> > > --
> > > Stuart Anderson [email protected]
> > > http://www.ligo.caltech.edu/~anderson
> >
> >
> > --
> > Jeff Layton <[email protected]>
>
> --
> Stuart Anderson [email protected]
> http://www.ligo.caltech.edu/~anderson
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 28 Jun 2007 06:41:27 -0400
Jeff Layton <[email protected]> wrote:
> On Wed, 27 Jun 2007 19:53:27 -0700
> Stuart Anderson <[email protected]> wrote:
>
> > On Wed, Jun 27, 2007 at 09:15:59PM -0400, Jeff Layton wrote:
> > > On Wed, 27 Jun 2007 17:59:57 -0700
> > > Stuart Anderson <[email protected]> wrote:
> > >
> > > > More precisely, applying this patch to the 2.6.20.14 kernel plus the
> > > > revalidate-the-fsid patch did not result in any changes. Does the
> > > > O_EXCL patch require some other supporting patches relative to 2.6.20.14?
> > > >
> > > > or perhaps it is necessary to rebase to a newer kernel before applying it?
> > > > even tough it applies, builds and runs with 2.6.20.14?
> > > >
> > > > Thanks.
> > > >
> > >
> > > With Linux, both client and server side were broken in this respect.
> > > The server didn't set the bitmask in the reply and the client didn't
> > > look at it anyway. It's possible that Solaris server is broken in this
> > > regard as well. It would be interesting to see a capture here,
> > > particularly one containing the reply from the CREATE.
> >
> > Possible, but when we switch the client from Linux to Solaris and share
> > the same filesytem from the same server there are no O_EXCL problems.
> >
>
> NFSv3 didn't use this bitmask, so clients tended to just clobber the
> mtime and atime on the subsequent setattr. It's possible that Solaris
> is using the old semantics here with NFSv4. Of course, all of this is
> speculation :-), a capture should give us a better clue.
>
Yes, it looks like Solaris is sending back a zeroed out attrmask on
exclusive create. I'm testing against:
Solaris Nevada snv_54 X86
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 04 December 2006
SunOS solaris 5.11 snv_54 i86pc i386 i86pc
My guess is that they're using NFSv3 semantics in their NFSv4 client to
set the mtime and atime, and that's why it "works" there. This patchrev
is pretty old by now, so they may have already patched this issue. I'll
see if I can patch it and test again.
All this, of course, is contingent upon my having interpreted the spec
correctly. I think I have, but wouldn't mind if someone sanity checked
me on it.
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 2007-06-28 at 08:01 -0400, Jeff Layton wrote:
> Yes, it looks like Solaris is sending back a zeroed out attrmask on
> exclusive create. I'm testing against:
>
> Solaris Nevada snv_54 X86
> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> Use is subject to license terms.
> Assembled 04 December 2006
>
> SunOS solaris 5.11 snv_54 i86pc i386 i86pc
>
> My guess is that they're using NFSv3 semantics in their NFSv4 client to
> set the mtime and atime, and that's why it "works" there. This patchrev
> is pretty old by now, so they may have already patched this issue. I'll
> see if I can patch it and test again.
>
> All this, of course, is contingent upon my having interpreted the spec
> correctly. I think I have, but wouldn't mind if someone sanity checked
> me on it.
The only nit I can see is that the server is in theory allowed to store
the cookie in some writable attribute other than atime/mtime, but I
can't think of that many that would make sense. Perhaps time_backup and
time_create for those servers that support them? However since we don't
care about their values and since they don't support a
SET_TO_SERVER_TIME4 option, I'm not sure we want to bother...
Trond
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, 28 Jun 2007 09:19:45 -0400
Trond Myklebust <[email protected]> wrote:
> On Thu, 2007-06-28 at 08:01 -0400, Jeff Layton wrote:
> > Yes, it looks like Solaris is sending back a zeroed out attrmask on
> > exclusive create. I'm testing against:
> >
> > Solaris Nevada snv_54 X86
> > Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> > Use is subject to license terms.
> > Assembled 04 December 2006
> >
> > SunOS solaris 5.11 snv_54 i86pc i386 i86pc
> >
> > My guess is that they're using NFSv3 semantics in their NFSv4 client to
> > set the mtime and atime, and that's why it "works" there. This patchrev
> > is pretty old by now, so they may have already patched this issue. I'll
> > see if I can patch it and test again.
> >
> > All this, of course, is contingent upon my having interpreted the spec
> > correctly. I think I have, but wouldn't mind if someone sanity checked
> > me on it.
>
> The only nit I can see is that the server is in theory allowed to store
> the cookie in some writable attribute other than atime/mtime, but I
> can't think of that many that would make sense. Perhaps time_backup and
> time_create for those servers that support them? However since we don't
> care about their values and since they don't support a
> SET_TO_SERVER_TIME4 option, I'm not sure we want to bother...
>
> Trond
>
>
Yes, I didnt see other attrs that made sense either, but if we find any
it shouldn't be too hard to add them in.
In any case, Solaris pretty clearly seems to be using the mtime to
store the verifier but isn't setting the attrmask. I'd like to test a
later solaris rev, but I'm having some issues getting this image
patched.
Stuart, I'd recommend opening a case with Sun and referring them to
this discussion and the RFC. I'm pretty certain this is a server bug.
If Sun mentions this is already fixed, or issues a patch for it, then
please let us know in case others run into this problem.
Thanks,
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Wed, Jun 27, 2007 at 10:59:10PM -0500, Spencer Shepler wrote:
> The Solaris NFSv3 server will, upon "get acl", map the underlying
> ZFS ACL format to the NFSv3/ACL format. However, on setting of
> an acl, an error is returned (NOSUP). The decision was made that
> because of the imperfect mapping on acl set in this path that
> an error would be returned.
Mapping back from an NFSv3 to an NFSv4 ACL should be pretty easy,
shouldn't it? Especially if it's an ACL that was just fetched from
the server--so it shouldn't be particularly exotic.
> I believe the Solaris client would map the ENOTSUP back to the
> application as is and that may help the application adapt
> its behavior (avoid erroring out if that isn't desired).
But, yes, maybe cp could be smarter.
--b.
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Jeff Layton wrote:
> The attribute mask seems to be zeroed out here as well. What sort of server is this?
> -- Jeff
>
# cat /etc/release
Solaris 10 11/06 s10x_u3wos_10 X86
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 14 November 2006
# uname -v
Generic_118855-36
Thanks,
--
Erik A. Espinoza
Systems Administrator
LIGO/Caltech - MS 18-34
Pasadena, CA 91125
Ph: 626-395-8517
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Fri, 29 Jun 2007 15:31:13 -0700
"Erik A. Espinoza" <[email protected]> wrote:
> Jeff Layton wrote:
> > The attribute mask seems to be zeroed out here as well. What sort of server is this?
> > -- Jeff
> >
>
> # cat /etc/release
> Solaris 10 11/06 s10x_u3wos_10 X86
> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> Use is subject to license terms.
> Assembled 14 November 2006
> # uname -v
> Generic_118855-36
>
Ok, so looks like Solaris 10 has this bug too. You might want to open a
case with Sun if you have a support contract with them. Let them know
that they're not sending a correct attribute mask when they send an
exclusive create reply.
You might also try opening a bug about this at the opensolaris site.
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Interestingly enough, using Solaris as a client and server does not have
mtime issues.
Should I attempt to get a tcpdump of Sol NFSv4 server and client combo
doing the same thing?
Erik
Jeff Layton wrote:
> On Fri, 29 Jun 2007 15:31:13 -0700
> "Erik A. Espinoza" <[email protected]> wrote:
>
>> Jeff Layton wrote:
>>> The attribute mask seems to be zeroed out here as well. What sort of server is this?
>>> -- Jeff
>>>
>> # cat /etc/release
>> Solaris 10 11/06 s10x_u3wos_10 X86
>> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
>> Use is subject to license terms.
>> Assembled 14 November 2006
>> # uname -v
>> Generic_118855-36
>>
>
> Ok, so looks like Solaris 10 has this bug too. You might want to open a
> case with Sun if you have a support contract with them. Let them know
> that they're not sending a correct attribute mask when they send an
> exclusive create reply.
>
> You might also try opening a bug about this at the opensolaris site.
>
--
Erik A. Espinoza
Systems Administrator
LIGO/Caltech - MS 18-34
Pasadena, CA 91125
Ph: 626-395-8517
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Fri, 29 Jun 2007 20:09:16 -0700
"Erik A. Espinoza" <[email protected]> wrote:
> Interestingly enough, using Solaris as a client and server does not have
> mtime issues.
>
They're probably using NFSv3 semantics and are clobbering the mtime and
atime irregardless of the attrmask sent back.
>
> Should I attempt to get a tcpdump of Sol NFSv4 server and client combo
> doing the same thing?
I doubt it will tell us more than we already know. The captures we
already have indicate that they are not following the RFC.
>
> Erik
>
> Jeff Layton wrote:
> > On Fri, 29 Jun 2007 15:31:13 -0700
> > "Erik A. Espinoza" <[email protected]> wrote:
> >
> >> Jeff Layton wrote:
> >>> The attribute mask seems to be zeroed out here as well. What sort of server is this?
> >>> -- Jeff
> >>>
> >> # cat /etc/release
> >> Solaris 10 11/06 s10x_u3wos_10 X86
> >> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
> >> Use is subject to license terms.
> >> Assembled 14 November 2006
> >> # uname -v
> >> Generic_118855-36
> >>
> >
> > Ok, so looks like Solaris 10 has this bug too. You might want to open a
> > case with Sun if you have a support contract with them. Let them know
> > that they're not sending a correct attribute mask when they send an
> > exclusive create reply.
> >
> > You might also try opening a bug about this at the opensolaris site.
> >
>
> --
> Erik A. Espinoza
> Systems Administrator
> LIGO/Caltech - MS 18-34
> Pasadena, CA 91125
> Ph: 626-395-8517
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>
--
Jeff Layton <[email protected]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs