Contest is a system responsiveness benchmark for kernel development looking at
fair scheduling.
It can be found here:
http://contest.kolivas.net
Aggelos Economopoulos has completed a massive rewrite of the contest benchmark
in c to remove all BASH scripts and this will form the codebase for further
versions of contest.
Changes since the last release:
Complete change in the codebase, consolidation into one executable.
FreeBSD compatibility
Different format of the logs etc.
Addition of cacherun and dbench_loads
Rewrite of process_load by Rene Herman
More detailed instructions
Man Page
Lots of internal changes
This version will give different results to previous versions of contest so to
upgrade please delete all the .log files in your testbed directory.
Known Bugs:
Contest will still run each kernel compilation if the load fails to initialise
(eg dbench doesnt start) so please ensure all the necessary load applications
are in your PATH
Other minor annoyances
Aggelos is the main coder now but I am still the official maintainer so please
direct all bug reports, comments, patches etc to me. Further releases are
planned in the near future with minor changes, but results obtained from contest
should be compatible with this version.
Con
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Ok some mildly annoying bugs have already shown up in this release.
I've put up a contest-0.61pre on the contest website that addresses the one
which ruins the results and has some of the changes going into 0.61
http://contest.kolivas.net
Major problem is that you need to run make oldconfig and make dep manually
before running contest successfully the first time. This will be corrected by
the time 0.61 comes out (probably by weekend).
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)
iD8DBQE+JNKiF6dfvkL3i1gRAssdAJ0TRiDhbQUP3lcgTV86/iqS6SfdfgCeNI1d
t73yXAn48I+8j+w/l2ieDT0=
=TqgD
-----END PGP SIGNATURE-----
On Wed, Jan 15, 2003 at 02:16:48PM +1100, Con Kolivas wrote:
> Ok some mildly annoying bugs have already shown up in this release.
>
> I've put up a contest-0.61pre on the contest website that addresses the one
> which ruins the results and has some of the changes going into 0.61
Con/Aggelos,
What was the motivation for re-writing in C?
I've done some hacking on the old version here, and so I realise that
such a big shell script was getting a little out of hand, but surely
perl or python is a better option for this application?
If it's going to stay in C maybe we could integrate profiling support
from /proc/profile, bypassing readprofile? One of the guys here
recently wanted to get profiling information from his program, and it
would have been really good to have a library that could reset, start,
pause and return in some format the profiling data. If you think your
users might be interested in profile outputs I can write something
maybe we can both use.
-i
[email protected]
http://www.gelato.unsw.edu.au
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wednesday 15 Jan 2003 3:15 pm, Ian Wienand wrote:
> On Wed, Jan 15, 2003 at 02:16:48PM +1100, Con Kolivas wrote:
> > Ok some mildly annoying bugs have already shown up in this release.
> >
> > I've put up a contest-0.61pre on the contest website that addresses the
> > one which ruins the results and has some of the changes going into 0.61
>
> Con/Aggelos,
>
> What was the motivation for re-writing in C?
I believe Aggelos was looking at freebsd compatibility and c was his most
comfortable language. Since programming such a thing in c was way beyond my
capabilities and Aggelos had already taken on the task I was very happy to
use this as the new codebase. The previous BASH one was getting flakier the
more I tried to add to it. Also there were subtle things happening in the
BASH version that made for much more variation in results than this version.
So it's as much coincidence that Aggelos was writing a new version and I was
looking for someone to write one that this codebase was chosen for it.
> I've done some hacking on the old version here, and so I realise that
> such a big shell script was getting a little out of hand, but surely
> perl or python is a better option for this application?
Possibly but clearly c has no major limitations once the hard part (the
wrapper for the other applications) has been done.
> If it's going to stay in C maybe we could integrate profiling support
> from /proc/profile, bypassing readprofile? One of the guys here
> recently wanted to get profiling information from his program, and it
> would have been really good to have a library that could reset, start,
> pause and return in some format the profiling data. If you think your
> users might be interested in profile outputs I can write something
> maybe we can both use.
I don't know. Anything that may taint the results is bad but I have no idea
how many confounding variables this really introduces. I need feedback from
the developers that look at these results to know whether this is worthwhile
or not. While I'm the said maintainer, contest has grown into a beast of it's
own.
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)
iD8DBQE+JOYHF6dfvkL3i1gRAnY3AJ42aiDcMm1oMhT4fU1c+BhPKzgUGQCfQdu7
KzNQ4kT3OFvOY4dgHPlfAaM=
=so41
-----END PGP SIGNATURE-----
On Wed, Jan 15, 2003 at 03:39:34PM +1100, Con Kolivas wrote:
> more I tried to add to it. Also there were subtle things happening in the
> BASH version that made for much more variation in results than this version.
I agree, it had begun to outgrow it's bash roots (e.g. i think it was
the ratio section with variables named a b c d e which drove me nuts
when I couldn't figure out which one was giving me a zero value).
> Possibly but clearly c has no major limitations once the hard part (the
> wrapper for the other applications) has been done.
I would disagree. For example, playing tricks with strings,
pipes/redirects and files in C is a complete pain, compared with
perl/python. Most of the things I changed with the orignal bash
version were in this subset, and it allowed me to customise it for
what we were doing quickly and easily.
-i
[email protected]
http://www.gelato.unsw.edu.au
On Wednesday 15 January 2003 06:15, Ian Wienand wrote:
> On Wed, Jan 15, 2003 at 02:16:48PM +1100, Con Kolivas wrote:
> > Ok some mildly annoying bugs have already shown up in this release.
> >
> > I've put up a contest-0.61pre on the contest website that addresses the
> > one which ruins the results and has some of the changes going into 0.61
>
> Con/Aggelos,
>
> What was the motivation for re-writing in C?
I gave up trying to make the script work in my desktop, which runs freebsd
(too many little differences in the utilities used).
> I've done some hacking on the old version here, and so I realise that
> such a big shell script was getting a little out of hand, but surely
> perl or python is a better option for this application?
The script also had a few bugs that were actually artifacts of using bash
(e.g. you had to use killall instead of just killing your children).
> If it's going to stay in C maybe we could integrate profiling support
> from /proc/profile, bypassing readprofile? One of the guys here
> recently wanted to get profiling information from his program, and it
> would have been really good to have a library that could reset, start,
> pause and return in some format the profiling data. If you think your
> users might be interested in profile outputs I can write something
> maybe we can both use.
Feel free to do so. I won't be able to help because
a) I'm not interested the feature :)
b) I don't have the time, I'm just helping Con squash most of the bugs in my
code.
--
Follow each decision as closely as possible with its associated action.
- The Elements of Programming Style (Kernighan & Plaugher)
> On Wed, Jan 15, 2003 at 02:16:48PM +1100, Con Kolivas wrote:
> > Ok some mildly annoying bugs have already shown up in this release.
> >
> > I've put up a contest-0.61pre on the contest website that addresses the one
> > which ruins the results and has some of the changes going into 0.61
>
> Con/Aggelos,
>
> What was the motivation for re-writing in C?
>
> I've done some hacking on the old version here, and so I realise that
> such a big shell script was getting a little out of hand, but surely
> perl or python is a better option for this application?
>
> If it's going to stay in C maybe we could integrate profiling support
> from /proc/profile, bypassing readprofile? One of the guys here
> recently wanted to get profiling information from his program, and it
> would have been really good to have a library that could reset, start,
> pause and return in some format the profiling data. If you think your
> users might be interested in profile outputs I can write something
> maybe we can both use.
>
I'd be very interested in learning more about getting profile support in things
Might come in very handy for AIM7.
cliffw
OSDL
> -i
> [email protected]
> http://www.gelato.unsw.edu.au
>
> On Wed, Jan 15, 2003 at 03:39:34PM +1100, Con Kolivas wrote:
> > more I tried to add to it. Also there were subtle things happening in the
> > BASH version that made for much more variation in results than this version.
>
> I agree, it had begun to outgrow it's bash roots (e.g. i think it was
> the ratio section with variables named a b c d e which drove me nuts
> when I couldn't figure out which one was giving me a zero value).
>
> > Possibly but clearly c has no major limitations once the hard part (the
> > wrapper for the other applications) has been done.
>
> I would disagree. For example, playing tricks with strings,
> pipes/redirects and files in C is a complete pain, compared with
> perl/python. Most of the things I changed with the orignal bash
> version were in this subset, and it allowed me to customise it for
> what we were doing quickly and easily.
>
Good point.
One other note, the C version should have a much smaller memory footprint then
Perl or Python - less likely to perturb test results.
Speaking for myself, i think this is such a small pile of code, it might be fun
to have a parallel Perl/Python version. (it would be a good excuse for me to
learn some Python)
My personal choice would be C to generate the data and Perl to pretty-print
it.
I need to get into the code more for STP integration, so i'll have more
opinons
soon :)
cliffw
> -i
> [email protected]
> http://www.gelato.unsw.edu.au
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>