I performed contest benchmarks on kernel 2.5.38 when the kernel is compiled with
gcc3.2 to gcc2.95.3
warning: The following benchmarks may be disturbing to some viewers:
No Load:
Kernel Time CPU
2.5.38 68.25 99%
2.5.38-gcc32 103.03 99%
Process Load:
Kernel Time CPU
2.5.38 71.60 95%
2.5.38-gcc32 112.98 91%
IO Half Load:
Kernel Time CPU
2.5.38 81.26 90%
2.5.38-gcc32 168.25 70%
IO Full Load:
Kernel Time CPU
2.5.38 170.21 42%
2.5.38-gcc32 1405.25 8%
Mem Load:
Kernel Time CPU
2.5.38 104.22 70%
2.5.38-gcc32 153.00 74%
Both kernels had identical configs and contest was run using gcc2.95.3 (I can't
see how this would affect it but tell me if you think it will). The only
difference was the original 2.5.38 was compiled with gcc2.95.3 and -gcc32 was
compiled with gcc3.2
Note the lower performance even with noload, and the totally abysmal performance
under full load. See the logs below for the page fault differences too.
Full logs follow:
2.5.38:
noload Time: 68.25 CPU: 99% Major Faults: 204613 Minor Faults: 255906
process_load Time: 71.60 CPU: 95% Major Faults: 204019 Minor Faults: 255238
io_halfmem Time: 81.26 CPU: 90% Major Faults: 204019 Minor Faults: 255325
Was writing number 4 of a 112Mb sized io_load file after 90 seconds
io_fullmem Time: 170.21 CPU: 42% Major Faults: 204019 Minor Faults: 255272
Was writing number 6 of a 224Mb sized io_load file after 194 seconds
mem_load Time: 104.22 CPU: 70% Major Faults: 204120 Minor Faults: 256271
2.5.38-gcc32:
noload Time: 103.03 CPU: 99% Major Faults: 237750 Minor Faults: 439982
process_load Time: 112.98 CPU: 91% Major Faults: 236958 Minor Faults: 439030
io_halfmem Time: 168.25 CPU: 70% Major Faults: 236958 Minor Faults: 439261
Was writing number 16 of a 112Mb sized io_load file after 202 seconds
io_fullmem Time: 1405.25 CPU: 8% Major Faults: 237199 Minor Faults: 440233
Was writing number 59 of a 224Mb sized io_load file after 1496 seconds
mem_load Time: 153.00 CPU: 74% Major Faults: 237863 Minor Faults: 440524
These are disturbing to me.
Comments?
Con.
On Sun, 2002-09-22 at 23:04, Con Kolivas wrote:
> IO Full Load:
> Kernel Time CPU
> 2.5.38 170.21 42%
> 2.5.38-gcc32 1405.25 8%
Ugh?? Something is _seriously_ messed up here.
The CPU utilization is only 8% but the time is nearly 10x worse. You
sure the only difference was the compiler? I could think gcc-3.2 makes
some poorer choices wrt code optimization, but nothing feasible can come
to mind that would produce such terrible results.
Also, I believe RedHat is compiling their kernel in 8.0 with gcc-3.2,
unless they reintroduced kgcc. Surely that are not seeing these abysmal
numbers.
Robert Love
Quoting Robert Love <[email protected]>:
> On Sun, 2002-09-22 at 23:04, Con Kolivas wrote:
>
> > IO Full Load:
> > Kernel Time CPU
> > 2.5.38 170.21 42%
> > 2.5.38-gcc32 1405.25 8%
>
> Ugh?? Something is _seriously_ messed up here.
Agreed!
> The CPU utilization is only 8% but the time is nearly 10x worse. You
> sure the only difference was the compiler? I could think gcc-3.2 makes
> some poorer choices wrt code optimization, but nothing feasible can come
> to mind that would produce such terrible results.
Absolutely certain. I'm shaking from the results still... hard to type...
> Also, I believe RedHat is compiling their kernel in 8.0 with gcc-3.2,
> unless they reintroduced kgcc. Surely that are not seeing these abysmal
> numbers.
contest is a new benchmark. Noone has ever done anything like this before so it
wouldn't have shown up in ordinary benchmarks. Mandrake has done the same with
LM9.0 I believe
Con
On Sun, 2002-09-22 at 23:04, Con Kolivas wrote:
> I performed contest benchmarks on kernel 2.5.38 when the kernel is compiled with
> gcc3.2 to gcc2.95.3
>
> warning: The following benchmarks may be disturbing to some viewers:
Daniel Jacobowitz, a stunning gcc hacker, suggests you retry with
gcc-3.2 from CVS. Apparently some fixes have gone in since 3.2 release.
If the problem still occurs, there is either a serious problem in gcc or
we have some code that is miscompiled by gcc-3.2 (perhaps not its fault
but the kernel's).
Robert Love
Con Kolivas wrote:
>
> I performed contest benchmarks on kernel 2.5.38 when the kernel is compiled with
> gcc3.2 to gcc2.95.3
>
> warning: The following benchmarks may be disturbing to some viewers:
>
> No Load:
> Kernel Time CPU
> 2.5.38 68.25 99%
> 2.5.38-gcc32 103.03 99%
>
Try gcc-2.91.66. It might break the 45 second mark.
> IO Full Load:
> Kernel Time CPU
> 2.5.38 170.21 42%
> 2.5.38-gcc32 1405.25 8%
The streaming write is stalling gcc's read for long enough for gcc's
working set to be evicted. And the working set cannot be reestablished
because the streaming write prevents it. Meltdown.
I have fixed this. Hang around.
On Sun, Sep 22, 2002 at 08:41:51PM -0700, Andrew Morton wrote:
> Con Kolivas wrote:
> >
> > I performed contest benchmarks on kernel 2.5.38 when the kernel is compiled with
> > gcc3.2 to gcc2.95.3
> >
> > warning: The following benchmarks may be disturbing to some viewers:
> >
> > No Load:
> > Kernel Time CPU
> > 2.5.38 68.25 99%
> > 2.5.38-gcc32 103.03 99%
> >
>
> Try gcc-2.91.66. It might break the 45 second mark.
>
> > IO Full Load:
> > Kernel Time CPU
> > 2.5.38 170.21 42%
> > 2.5.38-gcc32 1405.25 8%
>
> The streaming write is stalling gcc's read for long enough for gcc's
> working set to be evicted. And the working set cannot be reestablished
> because the streaming write prevents it. Meltdown.
>
> I have fixed this. Hang around.
That sounds like a slowdown in _running_ GCC... The worrisome part of
Con's post is that he was talking about which compiler he'd built the
kernel with. Right?
--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer
On Sun, 2002-09-22 at 23:41, Andrew Morton wrote:
> Try gcc-2.91.66. It might break the 45 second mark.
>
> > IO Full Load:
> > Kernel Time CPU
> > 2.5.38 170.21 42%
> > 2.5.38-gcc32 1405.25 8%
>
> The streaming write is stalling gcc's read for long enough for gcc's
> working set to be evicted. And the working set cannot be reestablished
> because the streaming write prevents it. Meltdown.
>
> I have fixed this. Hang around.
Ehh, I was under the impression he was benchmarking kernels compiled
WITH these compilers, using contest?
Your post seems to imply he was using the compilers as the benchmark.
If so, I retract my previous post - I know gcc 3.x is slow as puke. I
think, however, he is comparing the resulting kernels, in which case
there is a serious issue at hand...
Robert Love
Quoting Daniel Jacobowitz <[email protected]>:
> On Sun, Sep 22, 2002 at 08:41:51PM -0700, Andrew Morton wrote:
> > Con Kolivas wrote:
> > >
> > > I performed contest benchmarks on kernel 2.5.38 when the kernel is
> compiled with
> > > gcc3.2 to gcc2.95.3
> > >
> > > warning: The following benchmarks may be disturbing to some viewers:
> > >
> > > No Load:
> > > Kernel Time CPU
> > > 2.5.38 68.25 99%
> > > 2.5.38-gcc32 103.03 99%
> > >
> >
> > Try gcc-2.91.66. It might break the 45 second mark.
> >
> > > IO Full Load:
> > > Kernel Time CPU
> > > 2.5.38 170.21 42%
> > > 2.5.38-gcc32 1405.25 8%
> >
> > The streaming write is stalling gcc's read for long enough for gcc's
> > working set to be evicted. And the working set cannot be reestablished
> > because the streaming write prevents it. Meltdown.
> >
> > I have fixed this. Hang around.
>
> That sounds like a slowdown in _running_ GCC... The worrisome part of
> Con's post is that he was talking about which compiler he'd built the
> kernel with. Right?
Correct. contest was run with gcc2.95.3 only. The kernels were compiled with
2.95.3 and 3.2 respectively.
Con.
Quoting Andrew Morton <[email protected]>:
> Con Kolivas wrote:
> >
> > Quoting Andrew Morton <[email protected]>:
> >
> > > Con Kolivas wrote:
> > > >
> > > > Correct. contest was run with gcc2.95.3 only. The kernels were
> compiled
> > > with
> > > > 2.95.3 and 3.2 respectively.
> > >
> > > I think you made a mistake. Please rerun. Just one data point will do.
> > >
> >
> > Ok here are two points to confirm the results and their reproducibility:
> >
> > No Load:
> > 2.5.38 68.25 99%
> > 2.5.38-gcc32 103.03 99%
> > 2.5.38-gcc32a 103.47 99%
> >
> > Process Load:
> > 2.5.38 71.60 95%
> > 2.5.38-gcc32 112.98 91%
> > 2.5.38-gcc32a 113.60 91%
> >
>
> beats the hell out of me. Frankly, I *still* think
> you made a mistake (at least on the io load thing)
> because the CPU time went down by a mile - it was
> waiting on disk all the time.
If you think I've made a mistake then you're probably correct. I'm investigating
this further. Please do NOT pass judgement on these benchmarks until I
completely retest everything, ensuring gcc is fixed for everything except the
kernel being tested. Disregard until I have a fresh set of confirmed results.
Con.
Con Kolivas wrote:
> If you think I've made a mistake then you're probably correct. I'm investigating
> this further. Please do NOT pass judgement on these benchmarks until I
> completely retest everything, ensuring gcc is fixed for everything except the
> kernel being tested. Disregard until I have a fresh set of confirmed results.
And do not forget to use gcc from CVS, they're at 3.2.1 (prerelease)
right now. Or even checkout 3.3 while you're at it.
Regards,
Axel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Monday 23 September 2002 06:16, Con Kolivas wrote:
> >
> > Ugh?? Something is _seriously_ messed up here.
>
The most important question to ask here is: What flags did you compile both
?!? I wouldn't count on the flags that were designed for gcc 2.95 to be any
good for 3.2... Could the original poster comment on this ?
Any GCC maintainers on this list to comment ? Is there any set of flags to be
passed to gcc 3.2 to replicate 2.95 behaviour ? I wouldn't rule out gcc 3.2
having a totaly different set of optimizations geared towards user space C++.
Again, any gcc maintainers comments ?!?
Since most of the code in gcc is for C++ most of the changes in gcc should
have been geared towards C++ (yes - quite a monstrous language). It seems to
me that the changes in C compilation between 2.95 and 3.2 should be minor
EXCEPT in terms of C optimization. Can anyone with assembly knowledge take
apart two identical drivers and see the better machine code produced by 2.95
as compared to 3.2 ? If so - can this be reported to the gcc folk ?
It seems to me that the difference is so huge that even user space
applications could show the difference. I suggest compiling a large C program
(emphasis on the C) in user space and doing the comparison... I would guess
that this should have been done by the gcc folk but because of the
hideousness of the C++ language I would guess that they mostly concentrated
on C++ and didn't bother to benchmark regular C optimization. This is quite
awful as the bulk of lower level open source code is in C and not C++ so this
kind of test has a lot of meaning for any distribution that is going to be
based on gcc 3.2...
If this benchmark turns out to be right then it seems to me that the only
conclusion is that the gcc folk let their interest in aesoteric features of
C++ (which has about 1/2 a billion of those) override the basic need for
strong C optimization. Yes - it now seems that the C++ language (which is
quite an abomination in terms of engineering and the KISS principle) is
actually hurting open source (which has been my conclusion for quite some
time).
Mark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9jvWZxlxDIcceXTgRAnxpAKDYz61RWvceqD13Z889rwtZLOaomwCggmmj
ixt6x1e+zXewlrYCCHbiN9Y=
=snOl
-----END PGP SIGNATURE-----
On Mon, Sep 23, 2002 at 02:06:01PM +0300, Mark Veltzer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Monday 23 September 2002 06:16, Con Kolivas wrote:
>
> > >
> > > Ugh?? Something is _seriously_ messed up here.
> >
>
> The most important question to ask here is: What flags did you compile both
> ?!? I wouldn't count on the flags that were designed for gcc 2.95 to be any
> good for 3.2... Could the original poster comment on this ?
>
> Any GCC maintainers on this list to comment ? Is there any set of flags to be
> passed to gcc 3.2 to replicate 2.95 behaviour ? I wouldn't rule out gcc 3.2
> having a totaly different set of optimizations geared towards user space C++.
> Again, any gcc maintainers comments ?!?
>
> Since most of the code in gcc is for C++ most of the changes in gcc should
> have been geared towards C++ (yes - quite a monstrous language). It seems to
> me that the changes in C compilation between 2.95 and 3.2 should be minor
> EXCEPT in terms of C optimization. Can anyone with assembly knowledge take
> apart two identical drivers and see the better machine code produced by 2.95
> as compared to 3.2 ? If so - can this be reported to the gcc folk ?
>
> It seems to me that the difference is so huge that even user space
> applications could show the difference. I suggest compiling a large C program
> (emphasis on the C) in user space and doing the comparison... I would guess
> that this should have been done by the gcc folk but because of the
> hideousness of the C++ language I would guess that they mostly concentrated
> on C++ and didn't bother to benchmark regular C optimization. This is quite
> awful as the bulk of lower level open source code is in C and not C++ so this
> kind of test has a lot of meaning for any distribution that is going to be
> based on gcc 3.2...
>
> If this benchmark turns out to be right then it seems to me that the only
> conclusion is that the gcc folk let their interest in aesoteric features of
> C++ (which has about 1/2 a billion of those) override the basic need for
> strong C optimization. Yes - it now seems that the C++ language (which is
> quite an abomination in terms of engineering and the KISS principle) is
> actually hurting open source (which has been my conclusion for quite some
> time).
Mark, if you followed the GCC development process you'd realize that
all of your above ranting about C++ is completely unfounded. Most
people doing performance work - and there are a good number of them -
focus on language-independent optimizations and check them primarily in
C.
And I've no idea what you mean by "EXCEPT in terms of C optimization".
First of all you're completely wrong - 3.2 supports most of C99, which
is substantial - and secondly, of course the bulk of changes in support
for a language are optimization. And those are substantial too.
--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer