2003-06-27 18:21:29

by Mark Peloquin

Subject: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results

Nightly Regression Summary
for
2.5.73-bk2 vs 2.5.73-bk3


Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------

dbench.ext2 P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.ext3 P Y N 2.5.73-bk2 2.5.73-bk3 report
dbench.jfs P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.reiser P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.xfs P N N 2.5.73-bk2 2.5.73-bk3 report
kernbench P N N 2.5.73-bk2 2.5.73-bk3 report
lmbench P Y Y 2.5.73-bk2 2.5.73-bk3 report
rawiobench P Y Y 2.5.73-bk2 2.5.73-bk3 report
specjbb P Y N 2.5.73-bk2 2.5.73-bk3 report
specsdet P N N 2.5.73-bk2 2.5.73-bk3 report
tbench P Y Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.ext2 P N N 2.5.73-bk2 2.5.73-bk3 report
tiobench.ext3 P N Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.jfs P N N 2.5.73-bk2 2.5.73-bk3 report
tiobench.reiser P N Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.xfs P N N 2.5.73-bk2 2.5.73-bk3 report
volanomark P N N 2.5.73-bk2 2.5.73-bk3 report

http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk3/2.5.73-bk2-vs-2.5.73-bk3/

Nightly Regression Summary
for
2.5.73 vs 2.5.73-bk3


Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------

dbench.ext2 P N N 2.5.73 2.5.73-bk3 report
dbench.ext3 P N N 2.5.73 2.5.73-bk3 report
dbench.jfs P N N 2.5.73 2.5.73-bk3 report
dbench.reiser P N N 2.5.73 2.5.73-bk3 report
dbench.xfs P N N 2.5.73 2.5.73-bk3 report
kernbench P N N 2.5.73 2.5.73-bk3 report
lmbench P Y Y 2.5.73 2.5.73-bk3 report
rawiobench P Y Y 2.5.73 2.5.73-bk3 report
specjbb P Y Y 2.5.73 2.5.73-bk3 report
specsdet P Y N 2.5.73 2.5.73-bk3 report
tbench P N Y 2.5.73 2.5.73-bk3 report
tiobench.ext2 P N N 2.5.73 2.5.73-bk3 report
tiobench.ext3 P Y N 2.5.73 2.5.73-bk3 report
tiobench.jfs P N N 2.5.73 2.5.73-bk3 report
tiobench.reiser P N N 2.5.73 2.5.73-bk3 report
tiobench.xfs P N N 2.5.73 2.5.73-bk3 report
volanomark P N N 2.5.73 2.5.73-bk3 report

http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk3/2.5.73-vs-2.5.73-bk3/

Nightly Regression Summary
for
2.5.73-bk3 vs 2.5.73-bk4


Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------

dbench.ext2 P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.ext3 P N Y 2.5.73-bk3 2.5.73-bk4 report
dbench.jfs P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.reiser P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.xfs P N N 2.5.73-bk3 2.5.73-bk4 report
kernbench P N N 2.5.73-bk3 2.5.73-bk4 report
lmbench P Y Y 2.5.73-bk3 2.5.73-bk4 report
rawiobench P Y Y 2.5.73-bk3 2.5.73-bk4 report
specjbb P N Y 2.5.73-bk3 2.5.73-bk4 report
specsdet P N N 2.5.73-bk3 2.5.73-bk4 report
tbench P Y Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.ext2 P N N 2.5.73-bk3 2.5.73-bk4 report
tiobench.ext3 P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.jfs P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.reiser P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.xfs P N Y 2.5.73-bk3 2.5.73-bk4 report
volanomark P N N 2.5.73-bk3 2.5.73-bk4 report

http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk4/2.5.73-bk3-vs-2.5.73-bk4/

Nightly Regression Summary
for
2.5.73 vs 2.5.73-bk4


Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------

dbench.ext2 P N N 2.5.73 2.5.73-bk4 report
dbench.ext3 P N N 2.5.73 2.5.73-bk4 report
dbench.jfs P N N 2.5.73 2.5.73-bk4 report
dbench.reiser P N N 2.5.73 2.5.73-bk4 report
dbench.xfs P N N 2.5.73 2.5.73-bk4 report
kernbench P N N 2.5.73 2.5.73-bk4 report
lmbench P Y Y 2.5.73 2.5.73-bk4 report
rawiobench P Y Y 2.5.73 2.5.73-bk4 report
specjbb P Y Y 2.5.73 2.5.73-bk4 report
specsdet P Y N 2.5.73 2.5.73-bk4 report
tbench P Y Y 2.5.73 2.5.73-bk4 report
tiobench.ext2 P N Y 2.5.73 2.5.73-bk4 report
tiobench.ext3 P N Y 2.5.73 2.5.73-bk4 report
tiobench.jfs P N Y 2.5.73 2.5.73-bk4 report
tiobench.reiser P N Y 2.5.73 2.5.73-bk4 report
tiobench.xfs P N Y 2.5.73 2.5.73-bk4 report
volanomark P N N 2.5.73 2.5.73-bk4 report

http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk4/2.5.73-vs-2.5.73-bk4/

Nightly Regression Summary
for
2.5.73 vs 2.5.73-mjb1


Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------

dbench.ext2 P N N 2.5.73 2.5.73-mjb1 report
dbench.ext3 P N Y 2.5.73 2.5.73-mjb1 report
dbench.jfs P N N 2.5.73 2.5.73-mjb1 report
dbench.reiser P Y N 2.5.73 2.5.73-mjb1 report
dbench.xfs P N N 2.5.73 2.5.73-mjb1 report
kernbench P N N 2.5.73 2.5.73-mjb1 report
lmbench P Y Y 2.5.73 2.5.73-mjb1 report
rawiobench P Y Y 2.5.73 2.5.73-mjb1 report
specjbb P N Y 2.5.73 2.5.73-mjb1 report
specsdet P Y N 2.5.73 2.5.73-mjb1 report
tbench P N Y 2.5.73 2.5.73-mjb1 report
tiobench.ext2 P Y N 2.5.73 2.5.73-mjb1 report
tiobench.ext3 P Y Y 2.5.73 2.5.73-mjb1 report
tiobench.jfs P N Y 2.5.73 2.5.73-mjb1 report
tiobench.reiser P N N 2.5.73 2.5.73-mjb1 report
tiobench.xfs P Y Y 2.5.73 2.5.73-mjb1 report
volanomark P N N 2.5.73 2.5.73-mjb1 report

http://ltcperf.ncsa.uiuc.edu/data/2.5.73-mjb1/2.5.73-vs-2.5.73-mjb1/

Mark

2003-06-27 18:29:55

by Dave Hansen

Subject: Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results

On Fri, 2003-06-27 at 11:34, Mark Peloquin wrote:
> Nightly Regression Summary
> for
...

Have you ever had these regression tests run against the same kernel?
say, 2.5.73 vs 2.5.73

How often do you see improvements or regressions reported then?

--
Dave Hansen
[email protected]

2003-06-27 18:28:03

by Jeff Garzik

Subject: Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results

On Fri, Jun 27, 2003 at 01:34:37PM -0500, Mark Peloquin wrote:
> Nightly Regression Summary
> for
> 2.5.73-bk2 vs 2.5.73-bk3


This is very cool stuff.

Thanks!

Jeff

2003-06-27 18:37:45

by Mark Peloquin

Subject: Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results

Jeff Garzik wrote:

>On Fri, Jun 27, 2003 at 01:34:37PM -0500, Mark Peloquin wrote:
>
>
>>Nightly Regression Summary
>>for
>>2.5.73-bk2 vs 2.5.73-bk3
>>
>>
>
>
>This is very cool stuff.
>
>Thanks!
>
> Jeff
>

Glad you like it!

Mark

2003-06-27 18:41:24

by Mark Peloquin

Subject: Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results

Dave Hansen wrote:

>On Fri, 2003-06-27 at 11:34, Mark Peloquin wrote:
>
>
>>Nightly Regression Summary
>>for
>>
>>
>...
>
>Have you ever had these regression tests run against the same kernel?
>say, 2.5.73 vs 2.5.73
>
>How often do you see improvements or regressions reported then?
>
>

Its benchmark specific. Currently dbench, specjbb, and lmbench have a
few problematic data points that aren't terribly repeatable. So on the
same kernel, one can see improvements and regressions. That said, the
majority of data points are very repeatable and give good indications of
real change. We will continue to tweak things here and there to reduce
the number of false positives.

Mark