Hi,
Can someone give me a pointer to a FAQ on how to tune a 2.4 machine to
achieve high throughput (approx i/f speed 100Mbits/sec) on a link with the
following characteristics:
Latency Throughput
9-10ms 3.8 MByte/s
3-4ms 7-8MByte/s
I have implemented:
echo "4096 87380 4194304" > /proc/sys/net/ipv4/tcp_rmem
echo "4096 65536 4194304" > /proc/sys/net/ipv4/tcp_wmem
from http://www-didc.lbl.gov/tcp-wan.html
this lifted the performance from ~1MByte/s to the 3.8 above.
When the receiving machine is freebsd I get 10.05 MBytes/s which
is interesting - but when sending from BSD I get the same rate.
cheers
Mark
[root@tsaturn ncftp]# lsmod
Module Size Used by
autofs 11264 1 (autoclean)
3c59x 25344 1 (autoclean)
e100 44240 1 (autoclean)
ipchains 38976 0 (unused)
Mark Henson wrote:
>
> Hi,
>
> Can someone give me a pointer to a FAQ on how to tune a 2.4 machine to
> achieve high throughput (approx i/f speed 100Mbits/sec) on a link with the
> following characteristics:
>
> Latency Throughput
>
> 9-10ms 3.8 MByte/s
> 3-4ms 7-8MByte/s
>
> I have implemented:
>
> echo "4096 87380 4194304" > /proc/sys/net/ipv4/tcp_rmem
> echo "4096 65536 4194304" > /proc/sys/net/ipv4/tcp_wmem
>
> from http://www-didc.lbl.gov/tcp-wan.html
>
> this lifted the performance from ~1MByte/s to the 3.8 above.
>
> When the receiving machine is freebsd I get 10.05 MBytes/s which
> is interesting - but when sending from BSD I get the same rate.
>
Some things I change to increase performance (it seems to help...)
(I'm cutting & pasting from a perl script, so you'll need to interpret
it a bit..)
# NOTE: For faster systems I usually bump the *mem* settings up to 4MB.
my $netdev_max_backlog = 4096; # Maximum number of packets, queued on the INPUT side, when
# the interface receives pkts faster than it can process them.
my $wmem_max = 512000; # Write memory buffer. This is probably fine for any setup,
# and could be smaller (256000) for < 5Mbps connections.
my $wmem_default = 512000; # Write memory buffer. This is probably fine for any setup,
# and could be smaller (256000) for < 5Mbps connections.
my $rmem_max = 1024000; # Receive memory (packet) buffer. If you are running lots of very fast traffic,
# you may want to make this larger, up to 4096000 or so. If you are
# at around 20Mbps of traffic per connection or smaller, then 1024000
# is plenty. For < 5Mbps of traffic, 512000 should be fine.
my $rmem_default = 1024000; # Receive memory (packet) buffer. If you are running lots of very fast traffic,
# you may want to make this larger, up to 4096000 or so. If you are
# at around 20Mbps of traffic per connection or smaller, then 1024000
# is plenty. For < 5Mbps of traffic, 512000 should be fine.
printAndExec("echo $wmem_max > /proc/sys/net/core/wmem_max");
printAndExec("echo $wmem_default > /proc/sys/net/core/wmem_default");
printAndExec("echo $rmem_max > /proc/sys/net/core/rmem_max");
printAndExec("echo $rmem_default > /proc/sys/net/core/rmem_default");
printAndExec("echo $netdev_max_backlog > /proc/sys/net/core/netdev_max_backlog");
And of course, make sure you can get the performance with a known fast network
(and near zero latency) first!!
The e100 has some interesting options that seem to make it handle high packet
counts better, as well as giving it bigger descriptor lists, but I haven't
really benchmarked it..
Ben
> cheers
> Mark
>
> [root@tsaturn ncftp]# lsmod
> Module Size Used by
> autofs 11264 1 (autoclean)
> 3c59x 25344 1 (autoclean)
> e100 44240 1 (autoclean)
> ipchains 38976 0 (unused)
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Ben Greear <[email protected]> <Ben_Greear AT excite.com>
President of Candela Technologies Inc http://www.candelatech.com
ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
On Thu, 4 Oct 2001, Ben Greear wrote:
> printAndExec("echo $rmem_default > /proc/sys/net/core/rmem_default");
> printAndExec("echo $netdev_max_backlog > /proc/sys/net/core/netdev_max_backlog");
>
... thanks -
tried these parameters and saw no change in the throughput. I am
thinking that the problem is at the receiving end rather than the transmit
end because with the FreeBSD machine at 1 end I got a much higher
throughput when sending to that machine. (about 10 MBytes /sec)
I was suspicious of another problem with the FreeBSD machine which is why
I changed it out. (network occasionally locked up). The distance between
the local ethernet switches is about 1200km of fibre. Speed of light for
that distance is ~ 4ms. Light in fibre I believe to be about 0.6 giving
around 6.5ms for light in this loop - so 9.5 ms RTT seems pretty good
The machines (Compaq Deskpro ~800MHz) are able to reliably produce
~10MBytes/sec using ncftp which is hardly calibrated but is atleast
indicative of the rate I should be getting. Calculating transfer time of
26/27 secs of 106348240 gives me net of about 32Mbps
anyway thanks for your help
cheers
Mark
one machine is:
eth1 Link encap:Ethernet HWaddr 00:04:76:B8:8B:DC
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11757991 errors:0 dropped:0 overruns:1 frame:0
TX packets:11699364 errors:0 dropped:0 overruns:0 carrier:49
collisions:274 txqueuelen:100
Interrupt:5 Base address:0x1000
eth1: 3Com PCI 3cSOHO100-TX Hurricane at 0x1000, 00:04:76:b8:8b:dc, IRQ 5
product code 4d4d rev 00.12 date 06-29-01
8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
MII transceiver found at address 24, status 7849.
Enabling bus-master transmits and whole-frame receives.
eth1: scatter/gather disabled. h/w checksums enabled
eth1: using NWAY device table, not 8
the second is:
eth1 Link encap:Ethernet HWaddr 00:01:03:39:ED:5F
inet addr:192.168.1.3 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9567005 errors:0 dropped:0 overruns:0 frame:0
TX packets:9652682 errors:0 dropped:0 overruns:0 carrier:607
collisions:0 txqueuelen:100
Interrupt:5 Base address:0x1000
eth1: 3Com PCI 3c980 10/100 Base-TX NIC(Python-T) at 0x1000,
00:01:03:39:ed:5f, IRQ 5
product code 4b50 rev 00.14 date 03-12-01
8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
MII transceiver found at address 24, status 7809.
Enabling bus-master transmits and whole-frame receives.
>
> And of course, make sure you can get the performance with a known fast network
> (and near zero latency) first!!
>
> The e100 has some interesting options that seem to make it handle high packet
> counts better, as well as giving it bigger descriptor lists, but I haven't
> really benchmarked it..
>
> Ben
>
>