2004-09-28 07:56:52

by Andrew Walrond

[permalink] [raw]
Subject: [OT] Microsoft claim 267% better peak performance than linux?

I was pointed to this (rotating) banner advert at the top of http://www.eweek.com

It claims that when comparing Red Hat AS2.1 with Windows Server 2003 on a dual processor machine, Windows Server 2003 gives 276% better peak performance, quoting Veritest as the source.

I think the report in question must be this one

http://download.microsoft.com/download/0/7/1/0715a190-70f5-4b0d-8ced-f9d1e046aa6a/netbench.pdf

Its dated April 2003, so this was probably discussed a long time ago, but can anyone point me in the direction of an analysis/rebuttal of this report from the linux community? "Windows webserver gives 3x performance of linux" is something I need to be able to debunk, assuming it's inaccurate.

Andrew


2004-09-28 08:05:58

by Andrew Walrond

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

On Tuesday 28 Sep 2004 08:55, Andrew Walrond wrote:
>
> I think the report in question must be this one
>
> http://download.microsoft.com/download/0/7/1/0715a190-70f5-4b0d-8ced-f9d1e0
>46aa6a/netbench.pdf
>

Wrong link. This is the one

http://download.microsoft.com/download/0/7/1/0715a190-70f5-4b0d-8ced-f9d1e046aa6a/webbench.pdf

2004-09-28 08:09:51

by Norbert van Nobelen

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

Just the fact that the benchmark is this old, is a good reason to doubt
it.

On Tue, 2004-09-28 at 09:55, Andrew Walrond wrote:
> I was pointed to this (rotating) banner advert at the top of http://www.eweek.com
>
> It claims that when comparing Red Hat AS2.1 with Windows Server 2003 on a dual processor machine, Windows Server 2003 gives 276% better peak performance, quoting Veritest as the source.
>
> I think the report in question must be this one
>
> http://download.microsoft.com/download/0/7/1/0715a190-70f5-4b0d-8ced-f9d1e046aa6a/netbench.pdf
>
> Its dated April 2003, so this was probably discussed a long time ago, but can anyone point me in the direction of an analysis/rebuttal of this report from the linux community? "Windows webserver gives 3x performance of linux" is something I need to be able to debunk, assuming it's inaccurate.
>
> Andrew
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Met vriendelijke groet,

Norbert van Nobelen
EduSupport

Postbus 95963
2509CZ Den Haag
T: 070-3280200
M: 06-43036586
F: 070-3280029
E: [email protected]
I: http://www.edusupport.nl
B: ABN AMRO
R: 47.38.00.411

2004-09-28 12:22:43

by Denis Vlasenko

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

On Tuesday 28 September 2004 07:55, Andrew Walrond wrote:
> I was pointed to this (rotating) banner advert at the top of http://www.eweek.com
>
> It claims that when comparing Red Hat AS2.1 with Windows Server 2003 on a
> dual processor machine, Windows Server 2003 gives 276% better peak
> performance, quoting Veritest as the source.

It is very easy to 'slightly' misconfigure Linux machine so that it
slows to a crawl. For webservers, classic way to do it is to force
Apache to log a fqdn of incoming connections instead of numeric IP.

>From pdf:
> Microsoft commissioned VeriTest, a division of Lionbridge
> Technologies, Inc., to conduct a series of tests comparing
> the Web serving performance of the following server operating
> system configurations running on a variety of server hardware
> and processor configurations...

Do you seriously expect that MS-funded tests can ever find Linux
to be faster?
--
vda

2004-09-28 12:35:58

by Norbert van Nobelen

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

The document shows some interesting points though:
- They describe what they did to make redhat/apache perform better
- ISAPI is pretty fast compared to CGI (Didn't apache recently release a
programming interface which is cross platform and does something like
this too?)
- Multithreading works in their advantage (Compare 2003 normal against
multithreaded, maybe too little amount of apache threads running, or
some optimization needed in the linux kernel here?)
And last but not least:
WIndows 2003 suddenly realizes that is has a few CPU's left and starts
using them (-: See the graphs where there is a weird bent in the 8 CPU
curve.

And it still doesn't matter actually: 286% faster is still about 3 times
more expensive, which is the point on which is must be compared (lower
TCO because we are faster, so MS TCO is lower, but still too high)



On Tue, 2004-09-28 at 17:24, Denis Vlasenko wrote:
> On Tuesday 28 September 2004 07:55, Andrew Walrond wrote:
> > I was pointed to this (rotating) banner advert at the top of http://www.eweek.com
> >
> > It claims that when comparing Red Hat AS2.1 with Windows Server 2003 on a
> > dual processor machine, Windows Server 2003 gives 276% better peak
> > performance, quoting Veritest as the source.
>
> It is very easy to 'slightly' misconfigure Linux machine so that it
> slows to a crawl. For webservers, classic way to do it is to force
> Apache to log a fqdn of incoming connections instead of numeric IP.
>
> From pdf:
> > Microsoft commissioned VeriTest, a division of Lionbridge
> > Technologies, Inc., to conduct a series of tests comparing
> > the Web serving performance of the following server operating
> > system configurations running on a variety of server hardware
> > and processor configurations...
>
> Do you seriously expect that MS-funded tests can ever find Linux
> to be faster?
--
Met vriendelijke groet,

Norbert van Nobelen
EduSupport

Postbus 95963
2509CZ Den Haag
T: 070-3280200
M: 06-43036586
F: 070-3280029
E: [email protected]
I: http://www.edusupport.nl
B: ABN AMRO
R: 47.38.00.411

2004-09-28 13:02:19

by Denis Vlasenko

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

On Tuesday 28 September 2004 12:35, Norbert van Nobelen wrote:
> The document shows some interesting points though:
> - They describe what they did to make redhat/apache perform better

Yeah, like running 2.4.9 kernel... why not 2.0.0 kernel? ;)

Also, see page 33:

Operating System Key Exchange Algorithm Message Digest
Windows Server 2003 RSA RC4 (128-bit) MD5
Red Hat Linux Advanced Server 2.1 Diffie-Helman 3DES (168-bit) SHA1
Red Hat Linux 8.0 Professional Diffie-Helman 3DES (168-bit) SHA1

and:

...
Set HKLM\System\CurrentControlSet\Control\FileSystem\NtfsDisableLastAccess to 1.
(but conveniently forgot to mount Linux partitions noatime)
--
vda

Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Denis Vlasenko schrieb:
<snip>
| Do you seriously expect that MS-funded tests can ever find Linux
| to be faster?

helo denis,
helo andrew,

denis is right, in my opinion, ms would never publish a docuemnt which
shows that linux is much more faster then there own priducts.

i would like to recommand a book to you, andrew. the book rebel code
written by glynn moody ia a great funny book which deals with the open
source philosophy and the bad benchmark tests.
it also explains how many the linux community influenced the set up of
the test machine in relation to microsoft.

you should read it, it's a great book.

patrick kiwitter

- --
.O.
..O http://www.catb.org/hacker-emblem/
OOO

Administrator of http://www.devilcode.de - the developers community
Developer for http://www.lbsdde.de - the localized BSD project
chaos computer club http://www.ccc.de - cable salad is healthy

- -----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GIT d+ dx d s--:-- a-- C+++ UL+++(U-) P+++ L+++
E--- W+++ N++ w--- !O++ >V >Y+ b++(+++) h!(*)
- ------END GEEK CODE BLOCK------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFBWWA/DHDJZHMNHXYRArXsAJwKvkxebFMPoAg3B1VUENtMRZGQrQCgsgBr
2e/XcLTRIF16ltHnvecOr70=
=fe+M
-----END PGP SIGNATURE-----

2004-09-28 13:14:38

by Felipe Alfaro Solana

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

On Sep 28, 2004, at 14:35, Norbert van Nobelen wrote:

> The document shows some interesting points though:
> - They describe what they did to make redhat/apache perform better
> - ISAPI is pretty fast compared to CGI (Didn't apache recently release
> a
> programming interface which is cross platform and does something like
> this too?)

Of course ISAPI is faster than CGI. The same happens with NSAPI.
However, both are proprietary, have a steep learning curve, and
usually, xSAPI extensions run on the same address space as the Web
server, improving performance but decreasing stability. However, CGI
runs as a separate process which adds a lot of overhead. There are
solutions like FastCGI with less overhead and that allow persistence
(the executable CGI is not destroyed, but stays in memory waiting for
future requests).

MS commissioned studies are totally useless: they only probe what MS
wants. If MS wants us to believe earth is flat, a MS commissioned study
will reveal so.

2004-09-28 14:46:50

by Jeff V. Merkey

[permalink] [raw]
Subject: Re: [OT] Linux Claims 100,000% larger installed base than Microsoft


If they ran it on a 2.6 release, these numbers are probably accurate
with some of the memory sickness. AS looks like a middleware component.

:-)

Jeff



2004-09-29 19:32:09

by Tonnerre

[permalink] [raw]
Subject: Re: [OT] Microsoft claim 267% better peak performance than linux?

Salut,

On Tue, Sep 28, 2004 at 03:11:52PM +0200, Felipe Alfaro Solana wrote:
> MS commissioned studies are totally useless: they only probe what MS
> wants. If MS wants us to believe earth is flat, a MS commissioned study
> will reveal so.

Reminds me in the Apple studies in the mid-90s when they were still
technologically behind. They always issued that Apple computers were
faster than Linux/BSD in some strange benchmark under unnatural
conditions.

Didn't Microsoft once compare Windows 2000 against Linux 1.2?

Tonnerre


Attachments:
(No filename) (549.00 B)
signature.asc (189.00 B)
Digital signature
Download all attachments