I've noticed a difference in the rsize used when mounting a file system
between using text and binary mount options.
The client is running a CentOS5 based distro with a 2.6.32-rc8 kernel
The server has a preferred rsize of 128kb and maximum rsize of 512kb
When I use mount.nfs from CentOS5/RHEL5 nfs-utils (based on v1.0.9) and
don't give any rsize option, it mounts the file system with a rsize of
128kb. This uses binary mount options
But, when using mount.nfs from nfs-utils 1.2.0, the file system is
mounted with an rsize of 512kb
Looking at the nfs-utils and kernel source, it appears that for binary
options, rsize is set to 0 if not given by mount.nfs, and the kernel
eventually, in this case, increases this to preferred size.
But for text mount options, if not set by mount.nfs, the default size is
set to NFS_MAX_FILE_IO_SIZE in the kernel, which, in this case, gets
reduced to the server maximum size.
Should the kernel be setting rsize (and wsize) to 0 by default?
Thanks
James Pearson
On Sep 8, 2009, at 12:47 PM, James Pearson wrote:
> I've noticed a difference in the rsize used when mounting a file
> system between using text and binary mount options.
>
> The client is running a CentOS5 based distro with a 2.6.32-rc8 kernel
>
> The server has a preferred rsize of 128kb and maximum rsize of 512kb
>
> When I use mount.nfs from CentOS5/RHEL5 nfs-utils (based on v1.0.9)
> and don't give any rsize option, it mounts the file system with a
> rsize of 128kb. This uses binary mount options
>
> But, when using mount.nfs from nfs-utils 1.2.0, the file system is
> mounted with an rsize of 512kb
>
> Looking at the nfs-utils and kernel source, it appears that for
> binary options, rsize is set to 0 if not given by mount.nfs, and the
> kernel eventually, in this case, increases this to preferred size.
>
> But for text mount options, if not set by mount.nfs, the default
> size is set to NFS_MAX_FILE_IO_SIZE in the kernel, which, in this
> case, gets reduced to the server maximum size.
>
> Should the kernel be setting rsize (and wsize) to 0 by default?
nfs(5) says:
"If an [rw]size value is not specified, or if the specified [rw]size
value is larger than the maximum that either client or server can
support, the client and server negotiate the largest [rw]size value
that they can both support."
So the text-based behavior is what is documented now.
Does anyone know of a reason to use the server's "preferred" transfer
size rather than the largest size supported by both client and
server? Usually those are the same.
> Thanks
>
> James Pearson
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Chuck Lever wrote:
>>
>> Should the kernel be setting rsize (and wsize) to 0 by default?
>
>
> nfs(5) says:
>
> "If an [rw]size value is not specified, or if the specified [rw]size
> value is larger than the maximum that either client or server can
> support, the client and server negotiate the largest [rw]size value
> that they can both support."
>
> So the text-based behavior is what is documented now.
>
> Does anyone know of a reason to use the server's "preferred" transfer
> size rather than the largest size supported by both client and server?
> Usually those are the same.
In this case, the manufacturer of the NFS server recommends using 128Kb
for rsize and 512Kb for wsize - although the maximum rsize it supports
is 512Kb. I assume in their testing, these values have given optimal
performance figures.
The default behaviour with binary mount options when no [rw]size is to
select these preferred values - which to me, makes sense - as by not
giving a [rw]size, you are leaving it up the server to pick the 'best'
values for you - which I guess in most (all other?) cases happen to be
the maximum size.
James Pearson
On Sep 9, 2009, at 5:47 PM, James Pearson wrote:
> Chuck Lever wrote:
>>>
>>> Should the kernel be setting rsize (and wsize) to 0 by default?
>> nfs(5) says:
>> "If an [rw]size value is not specified, or if the specified
>> [rw]size value is larger than the maximum that either client or
>> server can support, the client and server negotiate the largest
>> [rw]size value that they can both support."
>> So the text-based behavior is what is documented now.
>> Does anyone know of a reason to use the server's "preferred"
>> transfer size rather than the largest size supported by both
>> client and server? Usually those are the same.
>
> In this case, the manufacturer of the NFS server recommends using
> 128Kb for rsize and 512Kb for wsize - although the maximum rsize it
> supports is 512Kb. I assume in their testing, these values have
> given optimal performance figures.
In this case maybe you should set these explicitly.
> The default behaviour with binary mount options when no [rw]size is
> to select these preferred values - which to me, makes sense - as by
> not giving a [rw]size, you are leaving it up the server to pick the
> 'best' values for you - which I guess in most (all other?) cases
> happen to be the maximum size.
It might make sense to think of the server's preference as the value
you should use. But for the client, the larger the better in nearly
every case. So which one should you choose by default?
Someone could still make the case that the old behavior was better in
some way (and provide benchmark evidence for this, of course, for
multiple combinations of implementations across different networks).
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
On Wed, 2009-09-09 at 22:47 +0100, James Pearson wrote:
> The default behaviour with binary mount options when no [rw]size is to
> select these preferred values - which to me, makes sense - as by not
> giving a [rw]size, you are leaving it up the server to pick the 'best'
> values for you - which I guess in most (all other?) cases happen to be
> the maximum size.
Right. The above was indeed the guiding principle back when I did the
rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount code.
The NFS protocol specifies that the maximum values are there to tell you
that the server will do short read/writes if you exceed these. However,
the preferred values may correspond to a different 'sweet spot' for the
server read and write implementations.
Cheers
Trond
Trond Myklebust wrote:
> On Wed, 2009-09-09 at 22:47 +0100, James Pearson wrote:
>
>>The default behaviour with binary mount options when no [rw]size is to
>>select these preferred values - which to me, makes sense - as by not
>>giving a [rw]size, you are leaving it up the server to pick the 'best'
>>values for you - which I guess in most (all other?) cases happen to be
>>the maximum size.
>
>
> Right. The above was indeed the guiding principle back when I did the
> rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount code.
>
> The NFS protocol specifies that the maximum values are there to tell you
> that the server will do short read/writes if you exceed these. However,
> the preferred values may correspond to a different 'sweet spot' for the
> server read and write implementations.
So does that mean that the binary mount options are doing the right
thing, whereas the text mount options are not? Also, just to confirm,
I'm using NFSv3.
Thanks
James Pearson
On Sep 10, 2009, at 5:08 AM, James Pearson wrote:
> Trond Myklebust wrote:
>> On Wed, 2009-09-09 at 22:47 +0100, James Pearson wrote:
>>> The default behaviour with binary mount options when no [rw]size
>>> is to select these preferred values - which to me, makes sense -
>>> as by not giving a [rw]size, you are leaving it up the server to
>>> pick the 'best' values for you - which I guess in most (all
>>> other?) cases happen to be the maximum size.
>> Right. The above was indeed the guiding principle back when I did the
>> rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount
>> code.
>> The NFS protocol specifies that the maximum values are there to
>> tell you
>> that the server will do short read/writes if you exceed these.
>> However,
>> the preferred values may correspond to a different 'sweet spot' for
>> the
>> server read and write implementations.
>
> So does that mean that the binary mount options are doing the right
> thing, whereas the text mount options are not? Also, just to
> confirm, I'm using NFSv3.
I don't think there is a clear right or wrong here. The binary
behavior follows protocol guidelines, which is fine. But, there are
plenty of cases where using the server's preferred size by default is
not always optimal. For example, the server may advertise a
preference for 32768, but using that setting with UDP on a congested
network would result in poor performance.
In other words, the client may decide that using a larger or smaller
setting than the server's preference would be better. Or, it may use
the largest possible setting for r/wsize, but advertise the server's
preference number in stat(2) results.
Fwiw, I don't have any problem switching the text-based defaults to
zero.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com