Here at the OSDL we have been working on an automated testing system for the
Linux kernel. The hope is to provide a full framework for environment
documentation, test configuration, execution and report presentation.
Currently we have a working implementation available at:
http://www.osdlab.org/stp
We are calling this project the "Scalable Test Platform" or "STP".
The following tests are available (keep in mind, this is early-phase material):
bash-memory
UNIXbench
lmbench (both with or without memory test options)
bonnie++ (includes various runs for file sizes and number-of-files stress)
dbench (short run of samples from 1-100 clients and a long run for graphing)
We are working on:
LTP tests (not the full suite, but a large portion)
iozone
AIM
More bonnie++ tests
Methods to test varying file systems
The ability to run tests requiring more than one host (i.e. client/server)
Automatic patch generator (to include patches with response reports)
We maintain a kernel CVS repository available at khack.osdlab.org:/var/cvs
as anonymous (via pserver) with write access (via ssh) available upon request.
The way the system works is a developer checks their patch into their own
development tree in our kernel CVS repository and tags it. Then, using the
web interface the developer requests what test to run, points it at the kernel
tag and specifies what type of machine to execute the test on. After that
point the entire process is automated and when it's completed the user will
receive a EMail notifying them their test has completing and possibly
including a summary of the results (if that feature is supported by the
particular test).
Currently we have multiple 2, 4 and 8 way machines up and running. We have a
load of single processor machines to add to the group within (hopefully) the
next week.
Sample test runs against 2.4.12 are available at:
dbench long:
2-way: http://fire.osdlab.org/stp/results/38/
4-way: http://fire.osdlab.org/stp/results/39/
8-way: http://fire.osdlab.org/stp/results/40/
UNIXbench:
4-way: http://fire.osdlab.org/stp/results/41/
lmbench (with memory test):
2-way: http://fire.osdlab.org/stp/results/36/
Bonnie++:
4-way: http://fire.osdlab.org/stp/results/46/
We are highly interested in comments regarding the user interface, the
environment documentation data gathered and possible enhancements to the
system. If you have a moment, please look at some of the above tests and
let me know what additional information would be useful.
The SourceForge project for the STP framework is available at:
http://www.sourceforge.net/projects/stp
The system is available for OSDL associates (free quick sign up - no spam) who
want to do Linux kernel testing. Please let me know if you would like access
to the CVS repository.
Thanks,
Nathan Dabney
Open Source Development Lab