> The main point with putting all the transport to a given server on one
> socket is that the UDP/TCP congestion control algorithms can then
> function efficiently. Instead of competing against other packets
> originating from itself, the client only has to deal with competition
> from other clients.
i guess i would buy that if someone could convince me the RPC client
would actually do fair load balancing if given the opportunity. my
own experience with multi process workloads is that we aren't there
yet. in other words, TCP is *designed* to handle network contention,
so i don't have a problem letting multiple sockets from the same
client duke it out for network bandwidth.
i also have some SMP scalability concerns. using a single transport
means we have many processes on several CPUs that are contending
for the socket buffer space, the socket and xprt locks, and that are
holding the BKL while issuing RPCs.
i know that NFSv4 is designed to work with a single connection to
a server... so this is coming whether i like it or not :^) i hope
we can arrive at a design that addresses the NFSv4 requirements and
increases the per-client mount limit substantially, but does not
suffer from the scalability issues i have raised.
-------------------------------------------------------
This SF.net email sponsored by: Enterprise Linux Forum Conference & Expo
The Event For Linux Datacenter Solutions & Strategies in The Enterprise
Linux in the Boardroom; in the Front Office; & in the Server Room
http://www.enterpriselinuxforum.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Charles Lever <Lever> writes:
> i guess i would buy that if someone could convince me the RPC
> client would actually do fair load balancing if given the
> opportunity. my own experience with multi process workloads is
> that we aren't there yet.
Load balancing is a policy problem for the higher levels. The RPC
layer doesn't carry any information about task priorities beyond the
most basic FIFO-style queueing. Why should it?
> in other words, TCP is *designed* to handle network contention,
> so i don't have a problem letting multiple sockets from the
> same client duke it out for network bandwidth.
The slow-start algorithm is not an optimal method for dealing with
contention. Every time you collide, you a forced to growth the
bandwidth linearly from zero up to 1/2 that you initially had before
you collided. After that, you grow using a 1/n rule.
In practice, therefore, you will tend to find it faster if you shunt
everything through the same TCP socket and avoid the collision in the
first place.
Then, with TCP, there's also the issue of reconnections. If the server
starts dropping TCP connections in order to deal with a resource
starvation issue, reconnection is an expensive operation.
IOW: lack of scalability can be a performance problem too...
> i also have some SMP scalability concerns. using a single
> transport means we have many processes on several CPUs that are
> contending for the socket buffer space, the socket and xprt
> locks, and that are holding the BKL while issuing RPCs.
That problem has nothing to do with the issue of multiple sockets or
not. The BKL is held over all RPC calls. I have yet to see numbers
showing that the socket/xprt locks are a major contention problem.
Cheers,
Trond
-------------------------------------------------------
This SF.net email sponsored by: Enterprise Linux Forum Conference & Expo
The Event For Linux Datacenter Solutions & Strategies in The Enterprise
Linux in the Boardroom; in the Front Office; & in the Server Room
http://www.enterpriselinuxforum.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs