I noticed that the Nagle algorithm was disabled just recently on the
client side, while it still seems to be enabled on the server side. The
previous patch came from Chuck Lever.
Below patch disables it on the server side as well. For latency reasons,
this should be the desired behaviour NFS at both client and server.
Diff is against 2.4.21-rc1.
---------------------------
--- linux-2.4.21-rc1/net/sunrpc/svcsock.c.orig
+++ linux-2.4.21-rc1/net/sunrpc/svcsock.c
@@ -932,6 +932,7 @@ static int
svc_tcp_init(struct svc_sock *svsk)
{
struct sock *sk = svsk->sk_sk;
+ struct tcp_opt *tp = &(sk->tp_pinfo.af_tcp);
svsk->sk_recvfrom = svc_tcp_recvfrom;
svsk->sk_sendto = svc_tcp_sendto;
@@ -948,6 +949,8 @@ svc_tcp_init(struct svc_sock *svsk)
svsk->sk_reclen = 0;
svsk->sk_tcplen = 0;
+ tp->nonagle = 1; /* disable Nagle's algorithm */
+
/* initialise setting must have enough space to
* receive and respond to one request.
* svc_tcp_recvfrom will re-adjust if necessary
---------------------------
Thanks,
Olof
--
Olof Johansson Office: 4E002/905
pSeries Linux Development IBM Systems Group
Email: [email protected] Phone: 512-838-9858
All opinions are my own and not those of IBM.
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Tue, 29 Apr 2003, Olof Johansson wrote:
> Below patch disables it on the server side as well. For latency reasons,
> this should be the desired behaviour NFS at both client and server.
I disagree with this general statement. Nagle is there in the TCP protocol
for a specific reason and enabling or disabling it gives the opportunity
to better tune the connection.
It seems like disabling Nagle is the bolierplate answer for anyone who
encounters GigEth or better networking simply because the TCP protocol was
not designed for such fast networks. But what happens to others that do
not have GigEth ? What is the impact of this change for a slower/congested
network ?
If anything, I would like to see this as a per-mount-point and
per-export-entry option...
--
Bogdan Costescu
IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: [email protected]
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Bogdan Costescu wrote:
> On Tue, 29 Apr 2003, Olof Johansson wrote:
>
>>Below patch disables it on the server side as well. For latency reasons,
>>this should be the desired behaviour NFS at both client and server.
>
>
> I disagree with this general statement. Nagle is there in the TCP protocol
> for a specific reason and enabling or disabling it gives the opportunity
> to better tune the connection.
Yes, Nagle is there to reduce overhead by aggregating data into segments
and piggybacking them with ACKs. Most significant differences will be
when dealing with a steady but slow pace of characters, say text being
typed over a 9600bps serial line.
> It seems like disabling Nagle is the bolierplate answer for anyone who
> encounters GigEth or better networking simply because the TCP protocol was
> not designed for such fast networks. But what happens to others that do
> not have GigEth ? What is the impact of this change for a slower/congested
> network ?
There is a chance that small requests and replies (~100 bytes) will not
be aggregated into the same segments but instead sent out separately. In
essence, this will result in some additional network overhead due to
headers, but the response times will be perceived as faster even for
slower networks.
Since NFS traffic is bursty, it is unlikely that any individual request
or reply will be split up between two segments when it didn't need to.
> If anything, I would like to see this as a per-mount-point and
> per-export-entry option...
I think it would just clutter the documentation and list of options,
since I don't forsee any practical scenarios in which anyone would have
a positive performance impact from having Nagle enabled.
On a side note: As far as I know, most (all?) other NFS implementations
out there already disable Nagle.
-Olof
--
Olof Johansson Office: 4E002/905
pSeries Linux Development IBM Systems Group
Email: [email protected] Phone: 512-838-9858
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Tue, 29 Apr 2003, Olof Johansson wrote:
> There is a chance that small requests and replies (~100 bytes) will not
> be aggregated into the same segments but instead sent out separately.
Although I haven't looked at the code recently, I think that what you said
is true when this is the only request from the client. But what happens
when there are several requests or when GETATTR calls happen at the same
time with some data transfer (which is normally the case for a multiuser
NFS client) ?
> In essence, this will result in some additional network overhead due to
> headers, but the response times will be perceived as faster even for
> slower networks.
True, until the network becomes too slow, then you see "server not
responding" messages...
> I think it would just clutter the documentation and list of options,
Yes, but Samba has this option and many others and still lots of people
are using it :-)
> since I don't forsee any practical scenarios in which anyone would have
> a positive performance impact from having Nagle enabled.
Well, my previous message included this question: "What is the impact of
this change for a slower/congested network ?" So I'll rephrase it: do you
have some data to show that disabling Nagle does not have a negative
impact ?
> On a side note: As far as I know, most (all?) other NFS implementations
> out there already disable Nagle.
I didn't say that the other NFS implementations are better than the Linux
implementation :-)
--
Bogdan Costescu
IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: [email protected]
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Bogdan Costescu <[email protected]> writes:
>> In essence, this will result in some additional network
>> overhead due to headers, but the response times will be
>> perceived as faster even for slower networks.
> True, until the network becomes too slow, then you see "server
> not responding" messages...
Anybody trying to run multi-user NFS over a 9.6Kbaud link needs their
head examined.
For *real use* situations, the total number of on-the-wire NFS
requests is currently limited to 16. In addition, TCP timeouts are
supposed to be > 1 minute. The above scenario simply will not happen.
For any reasonable setup over a slow network, the aggregation will
happen naturally anyway since the NFS layer both can and will fill the
buffers as soon as they start to empty (provided we have pending
datagrams).
For fast networks, we should at least ensure that single datagrams are
aggregated properly. The way to do that is to use MSG_MORE...
Cheers,
Trond
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Bogdan Costescu wrote:
> On Tue, 29 Apr 2003, Olof Johansson wrote:
>
>
>>There is a chance that small requests and replies (~100 bytes) will not
>>be aggregated into the same segments but instead sent out separately.
>
> Although I haven't looked at the code recently, I think that what you said
> is true when this is the only request from the client. But what happens
> when there are several requests or when GETATTR calls happen at the same
> time with some data transfer (which is normally the case for a multiuser
> NFS client) ?
This is exactly what I meant. There is a chance that the GETATTR (or
other small request and/or reply) will be sent out in an individual
segment instead of being aggregated. This will give some additional
network overhead, but give latency improvement. Actually, for a busy NFS
connection the small requests/replies will still be aggregated without
Nagle. At least that's what I've been seeing here.
>>In essence, this will result in some additional network overhead due to
>>headers, but the response times will be perceived as faster even for
>>slower networks.
>
> True, until the network becomes too slow, then you see "server not
> responding" messages...
I can't see how disabling Nagle would lead to RPC timeouts:
Lower network overhead during congestion is taken care of automatically:
As the network throughput decreases, TCP segments will be sent out at a
lower pace and/or smaller size (due to window sizes going down). There
will be more time to wait for more data to fill the segment with, even
without having Nagle enabled.
>>I think it would just clutter the documentation and list of options,
>
>
> Yes, but Samba has this option and many others and still lots of people
> are using it :-)
I didn't say that Samba is better than NFS :-)
Seriously: Samba has a configuration file, while NFS needs to stuff all
the options into a line or two of export and/or mount options. Option
bloat is not as big a problem when you have a large configuration file.
>>since I don't forsee any practical scenarios in which anyone would have
>>a positive performance impact from having Nagle enabled.
>
> Well, my previous message included this question: "What is the impact of
> this change for a slower/congested network ?" So I'll rephrase it: do you
> have some data to show that disabling Nagle does not have a negative
> impact ?
No, this would be to prove a negative which is always hard. But the same
reasoning as above regarding congested networks apply. Actually, it
applies to any scenario where the machine can process requests faster
than the network can send the replies.
>>On a side note: As far as I know, most (all?) other NFS implementations
>>out there already disable Nagle.
>
> I didn't say that the other NFS implementations are better than the Linux
> implementation :-)
I'm of the humble opinion that there is still a bit of room for
improvement in the Linux implementation, but it's continuosly getting
better. :-)
-Olof
--
Olof Johansson Office: 4E002/905
pSeries Linux Development IBM Systems Group
Email: [email protected] Phone: 512-838-9858
All opinions are my own and not those of IBM
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs