2002-01-25 00:52:16

by jjs

[permalink] [raw]
Subject: Testing the effects of the low latency patch

I had earlier posted reports about the low latency patch in terms
that are too subjective - e.g. saying that "quake 3 arena feels much
smoother and I frag a lot more" isn't the kind of hard statistical
evidence demanded by some. I have attempted to quanitify the latency
differences in one of the workloads where I see and feel a difference.

I built and benchmarked 2 kernels for comparison - my normal kernel,
which is at present 2.4.18-pre6 + Trond M's nfs fixes + Ingo M's tux
webserver + Andrew M's Low Latency patch; and a test kernel which was
built identically to the first kernel but without the low latency patch.


The bottom line is that the latency improvements are clearly visible
in the histograms and and the graphs (attached), and the obligatory
dbench runs at 2, 4, 8, 16, 32, 64 and 80 procs show that at worst,
the low latency patches do no harm to throughput, and may even improve
throughput. To be certain of the latter, better statistical samples
should of course be taken.


Note - I would be happy to add comparisons of the preempt patches if
and when their mortal conflict with tux is resolved.


The results:


Latency tests
--------------

Procedure:

1. Start realfeel
2. Play a round of "Return to castle Wolfenstein" online
3. Terminate realfeel and generate graphs with the included script
4. Compare the resulting histograms and graphs

2.4.18-pre6+tux+nfs-fixes
-------------
0.0 1964970
0.1 2809
0.2 2273
0.3 1330
0.4 943
0.5 826
0.6 832
0.7 657
0.8 117
0.9 3
1.0 5
1.1 9
1.2 15
1.3 5
1.4 14
1.5 6
1.6 3
1.7 3
1.8 1
1.9 2
2.0 10
2.1 2
2.2 1
2.7 1
3.3 1
3.4 2
3.6 1
3.7 2
7.6 1
7.8 1
21.1 1


2.4.18-pre6+tux+nfs-fixes + low latency patch
-------------
0.0 2596680
0.1 3303
0.2 4047
0.3 2341
0.4 1821
0.5 1803
0.6 1788
0.7 1505
0.8 272
0.9 1
1.0 12
1.1 18
1.2 42
1.3 26
1.4 23
1.5 9



The dbench results:

2.4.18-pre6+tux+nfs-fixes
---------------------------------
Throughput 25.873 MB/sec (NB=32.3413 MB/sec 258.73 MBit/sec) 80 procs
Throughput 34.8035 MB/sec (NB=43.5043 MB/sec 348.035 MBit/sec) 64 procs
Throughput 42.8055 MB/sec (NB=53.5069 MB/sec 428.055 MBit/sec) 32 procs
Throughput 48.9432 MB/sec (NB=61.179 MB/sec 489.432 MBit/sec) 16 procs
Throughput 111.575 MB/sec (NB=139.469 MB/sec 1115.75 MBit/sec) 8 procs
Throughput 111.66 MB/sec (NB=139.575 MB/sec 1116.6 MBit/sec) 4 procs
Throughput 112.307 MB/sec (NB=140.384 MB/sec 1123.07 MBit/sec) 2 procs

2.4.18-pre6+tux+nfs-fixes + low latency patch
---------------------------------
Throughput 32.1261 MB/sec (NB=40.1576 MB/sec 321.261 MBit/sec) 80 procs
Throughput 32.8968 MB/sec (NB=41.1211 MB/sec 328.968 MBit/sec) 64 procs
Throughput 48.2933 MB/sec (NB=60.3666 MB/sec 482.933 MBit/sec) 32 procs
Throughput 106.361 MB/sec (NB=132.951 MB/sec 1063.61 MBit/sec) 16 procs
Throughput 106.489 MB/sec (NB=133.111 MB/sec 1064.89 MBit/sec) 8 procs
Throughput 112.754 MB/sec (NB=140.942 MB/sec 1127.54 MBit/sec) 4 procs
Throughput 113.354 MB/sec (NB=141.693 MB/sec 1133.54 MBit/sec) 2 procs

Best,

Joe






2002-01-25 01:10:28

by Andrew Morton

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

J Sloan wrote:
>
> I had earlier posted reports about the low latency patch in terms
> that are too subjective - e.g. saying that "quake 3 arena feels much
> smoother and I frag a lot more" isn't the kind of hard statistical
> evidence demanded by some. I have attempted to quanitify the latency
> differences in one of the workloads where I see and feel a difference.

mm. Numbers. Nice.

>
> 2.4.18-pre6+tux+nfs-fixes
> -------------
> ...
> 7.6 1
> 7.8 1
> 21.1 1

This is the stock kernel. In twenty minutes you suffered
precisely *one* scheduling overrun which is perceptible
by a human. The rest are much shorter than your monitor's
refresh interval. Interesting, yes?

These results are better than the ones I normally measure.

> ...
> The dbench results:
>
> 2.4.18-pre6+tux+nfs-fixes
> ---------------------------------
> ...
> Throughput 48.9432 MB/sec (NB=61.179 MB/sec 489.432 MBit/sec) 16 procs
> ...
>
> 2.4.18-pre6+tux+nfs-fixes + low latency patch
> ---------------------------------
> ...
> Throughput 106.361 MB/sec (NB=132.951 MB/sec 1063.61 MBit/sec) 16 procs
> ...

Now that's odd.

-

2002-01-25 01:15:08

by jjs

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

Andrew Morton wrote:

>J Sloan wrote:
>
>>I had earlier posted reports about the low latency patch in terms
>>that are too subjective - e.g. saying that "quake 3 arena feels much
>>smoother and I frag a lot more" isn't the kind of hard statistical
>>evidence demanded by some. I have attempted to quanitify the latency
>>differences in one of the workloads where I see and feel a difference.
>>
>
>mm. Numbers. Nice.
>
>>2.4.18-pre6+tux+nfs-fixes
>>-------------
>>...
>>7.6 1
>>7.8 1
>>21.1 1
>>
>
>This is the stock kernel. In twenty minutes you suffered
>precisely *one* scheduling overrun which is perceptible
>by a human. The rest are much shorter than your monitor's
>refresh interval. Interesting, yes?
>
Yes, the stock kernel is much improved from
say 6 months ago. I will take a look at the
kernel that shipped with my distro just for
giggles as well...

>>
>>The dbench results:
>>
>>2.4.18-pre6+tux+nfs-fixes
>>---------------------------------
>>...
>>Throughput 48.9432 MB/sec (NB=61.179 MB/sec 489.432 MBit/sec) 16 procs
>>...
>>
>>2.4.18-pre6+tux+nfs-fixes + low latency patch
>>---------------------------------
>>...
>>Throughput 106.361 MB/sec (NB=132.951 MB/sec 1063.61 MBit/sec) 16 procs
>>...
>>
>
>Now that's odd.
>
Yes this smells like a statistical anomaly -

Will run mulitple tests tonight and see if
there's an actual trend there or just a
blip on the screen.

Joe

>

2002-01-25 01:19:38

by jjs

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

Oops, braino - if anyone is curious about the
graphs even after seeing the raw histograms,
they are at

ftp://wintermute.toyota.com/pub/graphs


Joe

2002-01-25 01:29:49

by Andrew Morton

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

J Sloan wrote:
>
> >>2.4.18-pre6+tux+nfs-fixes
> >>-------------
> >>...
> >>7.6 1
> >>7.8 1
> >>21.1 1
> >>
> >
> >This is the stock kernel. In twenty minutes you suffered
> >precisely *one* scheduling overrun which is perceptible
> >by a human. The rest are much shorter than your monitor's
> >refresh interval. Interesting, yes?
> >
> Yes, the stock kernel is much improved from
> say 6 months ago. I will take a look at the
> kernel that shipped with my distro just for
> giggles as well...
>

Was you histogram generated during a game session, or during
dbench? write()-intensive workloads are the main common
offender.

-

2002-01-25 01:37:39

by jjs

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

It was during gameplay, no dbench running, which
as it turns out is a workload of interest for me -

Joe

Andrew Morton wrote:

>J Sloan wrote:
>
>>>>2.4.18-pre6+tux+nfs-fixes
>>>>-------------
>>>>...
>>>>7.6 1
>>>>7.8 1
>>>>21.1 1
>>>>
>>>This is the stock kernel. In twenty minutes you suffered
>>>precisely *one* scheduling overrun which is perceptible
>>>by a human. The rest are much shorter than your monitor's
>>>refresh interval. Interesting, yes?
>>>
>>Yes, the stock kernel is much improved from
>>say 6 months ago. I will take a look at the
>>kernel that shipped with my distro just for
>>giggles as well...
>>
>
>Was you histogram generated during a game session, or during
>dbench? write()-intensive workloads are the main common
>offender.
>
>-
>


2002-01-25 01:39:09

by Mark Hahn

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

> ftp://wintermute.toyota.com/pub/graphs

I found it hard to grasp the separate graphs, so:
http://hahn.mcmaster.ca/~hahn/feel.png

also, I noticed that there were 30% more counts in ll;
I hope that means you simply enjoyed the game more, and played it longer ;)

2002-01-25 02:26:38

by jjs

[permalink] [raw]
Subject: Re: Testing the effects of the low latency patch

Mark Hahn wrote:

>>ftp://wintermute.toyota.com/pub/graphs
>>
>
>I found it hard to grasp the separate graphs, so:
>http://hahn.mcmaster.ca/~hahn/feel.png
>
Thanks that is a nicer view -

>also, I noticed that there were 30% more counts in ll;
>I hope that means you simply enjoyed the game more, and played it longer ;)
>
Well the length of games are somewhat arbitrary,
but I think it's safe to say it's more enjoyable with
low latency!

Joe