I have a situation where speed of processing of
network data through a LINUX cluster is extremely
crucial, and I need to shave 10 to 20 msec.
At the moment, I am running a stock REDHAT 7.1
distro with the more obviously unused modules
removed (like firewall services since the cluster
resides behind a firewalled router, etc).
First question: Will compiling a kernel with
the network module resident (as opposed to a loadable
module) make network performance any better? From
the reading, it appears that resident modules are only
faster in initialization, not runtime. I'm new to
this, so please correct me if I'm wrong.
Second: Threads. Each server runs one to several
custom services I've written, each of which performs a
part of data processing on the incoming data. Each
service consists of eight to thirty threads. The
question: Is there a way to tweak the kernel to
improve thread performance? I hear the 2.5 kernel
has thread performance improvements, but I don't want
to mess with it until I know more about it, or it
goes into release as v2.6.
Third: Shared libraries and task switching. My fear
is that overhead for kernel/OS and my custom services
may be grinding away at the disk drive unnecessarily.
Using sar(1), I see that no swap space is being used,
so I'm not hitting virtual memory. But I wonder
how likely it is that shared libs (used either by my
services or the kernel/OS) are being re-read from
disk? I am hoping that the libs get cached and thus
load from cache back into memory.
Fyi: Each server contains a dual CPU and either 500MB
of RAM or a full Gig. All machines are networked with
100baseT NICs. Pings between each server is down
around 10 to 30 usec, so I don't think our local
backbone is a significant issue.
The desired result is to shave off the full 20 msec.
If anyone knows about kernel futzing that will gain me
some performance, I'd love to hear it!
Many thanks!
-zap
__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com
On Sat, 2002-10-12 at 21:26, Zapp Foster wrote:
> First question: Will compiling a kernel with
> the network module resident (as opposed to a loadable
> module) make network performance any better? From
> the reading, it appears that resident modules are only
> faster in initialization, not runtime. I'm new to
> this, so please correct me if I'm wrong.
Modules are very very fractionally slower than compiled in code due to
TLB misses
> Second: Threads. Each server runs one to several
> custom services I've written, each of which performs a
> part of data processing on the incoming data. Each
> service consists of eight to thirty threads. The
> question: Is there a way to tweak the kernel to
> improve thread performance? I hear the 2.5 kernel
Update to the RH 7.3 kernel and you will get the O(1) scheduler too
> how likely it is that shared libs (used either by my
> services or the kernel/OS) are being re-read from
> disk? I am hoping that the libs get cached and thus
> load from cache back into memory.
They do
>> First question: Will compiling a kernel with
>> the network module resident (as opposed to a loadable
>> module) make network performance any better? From
>> the reading, it appears that resident modules are only
>> faster in initialization, not runtime. I'm new to
>> this, so please correct me if I'm wrong.
>
>Modules are very very fractionally slower than compiled in code due to
>TLB misses
Depends on which arch... on ppc32 (ahem... no bad joke pls ;),
function calls from modules to kernel or between modules has to
go through some "branch islands" as they don't fit within the
scope of a "short" branch PPC insn. So you also get a small
perf. loss there, but I bet it's barely measurable.
Ben.
REQ: More Info needed.
What's the kernel version you're using? Is it 2.4.19? If not, upgrade
it.
If fact, you may want to upgrade to Red Hat Linux 8.0 due to Ext3
support that wasn't available until Red Hat Linux 7.2. Ext3 mounted
with data=journal can offer significant performance increases for
servers with frequent I/O reads/writes over Ext2.
Is this a router? If so, make sure to check the "Optimize as router"
box in the kernel configuration.
ip_frag_low_thresh and ip_frag_high_thresh should be increased in some
situations. This will allow your server to take far more connections
and put them together in memory before having to ditch their fragments.
You can safely increase these to several megabytes without any problems.
I'm sorry, but I can't answer your questions.