A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
server because nfs4_state_manager() fails. I suspect it doesn't like
something about the EXCHANGE_ID reply. This worked before Benny's last
update (late Dec?), and I can bisect if necessary, but wondered if someone
here can take a look and see if there is something obviously wrong.
I put a dump of the call and reply here:
http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr
I can of course provide more info if that would help.
On Wed, 2011-01-12 at 13:58 -0500, Jim Rees wrote:
> A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
> server because nfs4_state_manager() fails. I suspect it doesn't like
> something about the EXCHANGE_ID reply. This worked before Benny's last
> update (late Dec?), and I can bisect if necessary, but wondered if someone
> here can take a look and see if there is something obviously wrong.
>
> I put a dump of the call and reply here:
> http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr
>
> I can of course provide more info if that would help.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
That is correct. According to your wireshark dump, your server is
returning an illegal combination of the EXCHGID4_FLAG_USE_* flags.
The combination
(EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
EXCHGID4_FLAG_USE_NON_PNFS)
is not listed in the table in section 13.1 of RFC5661 as an acceptable
return value.
Cheers
Trond
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
Hi Jim
The EMC server is returning exchange_id flags = 0x00070000 which is illegal - it's claming the NON_PNFS, PNFS and DS roles.
#define EXCHGID4_FLAG_USE_NON_PNFS 0x00010000
#define EXCHGID4_FLAG_USE_PNFS_MDS 0x00020000
#define EXCHGID4_FLAG_USE_PNFS_DS 0x00040000
#define EXCHGID4_FLAG_MASK_PNFS 0x00070000
See Section 13.1 of RFC 5661
-->Andy
On Jan 12, 2011, at 1:58 PM, Jim Rees wrote:
> A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
> server because nfs4_state_manager() fails. I suspect it doesn't like
> something about the EXCHANGE_ID reply. This worked before Benny's last
> update (late Dec?), and I can bisect if necessary, but wondered if someone
> here can take a look and see if there is something obviously wrong.
>
> I put a dump of the call and reply here:
> http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr
>
> I can of course provide more info if that would help.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Trond Myklebust wrote:
The combination
(EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
EXCHGID4_FLAG_USE_NON_PNFS)
is not listed in the table in section 13.1 of RFC5661 as an acceptable
return value.
But section 13 doesn't apply to the block layout server, does it?
On Wed, 2011-01-12 at 19:07 -0500, Jim Rees wrote:
> Trond Myklebust wrote:
>
> The combination
>
> (EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
> EXCHGID4_FLAG_USE_NON_PNFS)
>
> is not listed in the table in section 13.1 of RFC5661 as an acceptable
> return value.
>
> But section 13 doesn't apply to the block layout server, does it?
If you are a block layout server, the only valid return value is
EXCHGID4_FLAG_USE_PNFS_MDS
'cos you are clearly not a non-metadata server, and you clearly can't be
a data server.
Cheers
Trond
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
On Wed, 2011-01-12 at 19:30 -0500, Jim Rees wrote:
> Trond Myklebust wrote:
>
> > But section 13 doesn't apply to the block layout server, does it?
>
> If you are a block layout server, the only valid return value is
>
> EXCHGID4_FLAG_USE_PNFS_MDS
>
> 'cos you are clearly not a non-metadata server, and you clearly can't be
> a data server.
>
> Sorry to be dense, this is the first time I've thought about this.
>
> At the time of the EXCHANGE_ID call, how is the server supposed to know what
> kind of layout is going to be negotiated? It doesn't yet know whether the
> client is even going to ask for a layout, does it?
It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
advertising to the client that it supports LAYOUTGET and other pNFS
related operations. The client is free to ignore that message if it so
desires, and just use read/write through MDS. The point here is to tell
the client whether or not it should try pNFS if it can.
Cheers
Trond
--
Trond Myklebust
Linux NFS client maintainer
NetApp
[email protected]
http://www.netapp.com
On 2011-01-13 02:54, Jim Rees wrote:
> Trond Myklebust wrote:
>
> > At the time of the EXCHANGE_ID call, how is the server supposed to know what
> > kind of layout is going to be negotiated? It doesn't yet know whether the
> > client is even going to ask for a layout, does it?
>
> It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
> advertising to the client that it supports LAYOUTGET and other pNFS
> related operations. The client is free to ignore that message if it so
> desires, and just use read/write through MDS. The point here is to tell
> the client whether or not it should try pNFS if it can.
>
> I guess that makes sense. But I think section 13 is a bit misleading since
> it contains requirements for the block layout too.
True, but section 18.35. Operation 42: EXCHANGE_ID - Instantiate Client ID
specifically says:
The EXCHGID4_FLAG_USE_NON_PNFS, EXCHGID4_FLAG_USE_PNFS_MDS, and
EXCHGID4_FLAG_USE_PNFS_DS bits are described in Section 13.1 and
convey roles the client ID is to be used for in a pNFS environment.
The server MUST set one of the acceptable combinations of these bits
(roles) in eir_flags, as specified in Section 13.1.
Benny
>
> 13. NFSv4.1 as a Storage Protocol in pNFS: the File Layout Type
>
> This section describes the semantics and format of NFSv4.1 file-based
> layouts for pNFS. NFSv4.1 file-based layouts use the
> LAYOUT4_NFSV4_1_FILES layout type. The LAYOUT4_NFSV4_1_FILES type
> defines striping data across multiple NFSv4.1 data servers.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Trond Myklebust wrote:
> But section 13 doesn't apply to the block layout server, does it?
If you are a block layout server, the only valid return value is
EXCHGID4_FLAG_USE_PNFS_MDS
'cos you are clearly not a non-metadata server, and you clearly can't be
a data server.
Sorry to be dense, this is the first time I've thought about this.
At the time of the EXCHANGE_ID call, how is the server supposed to know what
kind of layout is going to be negotiated? It doesn't yet know whether the
client is even going to ask for a layout, does it?
Trond Myklebust wrote:
> At the time of the EXCHANGE_ID call, how is the server supposed to know what
> kind of layout is going to be negotiated? It doesn't yet know whether the
> client is even going to ask for a layout, does it?
It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
advertising to the client that it supports LAYOUTGET and other pNFS
related operations. The client is free to ignore that message if it so
desires, and just use read/write through MDS. The point here is to tell
the client whether or not it should try pNFS if it can.
I guess that makes sense. But I think section 13 is a bit misleading since
it contains requirements for the block layout too.
13. NFSv4.1 as a Storage Protocol in pNFS: the File Layout Type
This section describes the semantics and format of NFSv4.1 file-based
layouts for pNFS. NFSv4.1 file-based layouts use the
LAYOUT4_NFSV4_1_FILES layout type. The LAYOUT4_NFSV4_1_FILES type
defines striping data across multiple NFSv4.1 data servers.