Message-ID: <3F521B4E.10909@kegel.com>
Date: Sun, 31 Aug 2003 08:59:10 -0700
From: Dan Kegel <dank@kegel.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624
MIME-Version: 1.0
To: Daniel Jacobowitz <dan@debian.org>
CC: GCC Mailing List <gcc@gcc.gnu.org>, linux-kernel@vger.kernel.org
Subject: Re: LMbench as gcc performance regression test?
References: <3F51A201.8090108@kegel.com> <20030831152449.GA6893@nevyn.them.org>
In-Reply-To: <20030831152449.GA6893@nevyn.them.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1824
Lines: 42

Daniel Jacobowitz wrote:
> On Sun, Aug 31, 2003 at 12:21:37AM -0700, Dan Kegel wrote:
> 
>>http://cs.nmu.edu/~benchmark/ has an interesting little graph
>>of LMBench results vs. Linux kernel version, all done with the
>>same compiler.
>>
>>Has anyone seen a similar graph showing LMBench results vs. gcc version,
>>all done with the same Linux kernel?
>>And does everyone agree that's a meaningful way to compare the
>>performance of code generated by different compilers?
> 
> It's been a while since I looked at lmbench but: why do you think this
> would be useful?  It's a system and kernel benchmark; I doubt
> optimization makes much difference at all.

I need to make sure that moving to a newer compiler for our kernel
will cause no performance regressions.  Before bothering to bring up a
real-world networking application and measuring its performance
under the new compiler, it seems sensible to use a couple microbenchmarks
to verify that identifiable parts of the system have
not degraded in performance.

I myself am quite convinced I need to move to a newer compiler,
since I keep running into problems building various things with
old compilers, but my users are very conservative and skeptical;
I have to build a solid case for updating.  Hence the insane amount
of time I spent figuring out and documenting how to build and test
the various versions of gcc and glibc (http://kegel.com/crosstool),
and then understanding the regression test failures.
- Dan

-- 
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/