2017-07-03 02:57:55

by kernel test robot

[permalink] [raw]
Subject: [lkp-robot] [scsi] 5c279bd9e4: blogbench.read_score -10.0% regression


Greeting,

FYI, we noticed a -10.0% regression of blogbench.read_score due to commit:


commit: 5c279bd9e40624f4ab6e688671026d6005b066fa ("scsi: default to scsi-mq")
https://git.kernel.org/cgit/linux/kernel/git/jejb/scsi.git misc

in testcase: blogbench
on test machine: 16 threads Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz with 8G memory
with following parameters:

disk: 1SSD
fs: ext4
cpufreq_governor: performance

test-description: Blogbench is a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server.
test-url: https://www.pureftpd.org/project/blogbench


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

git clone https://github.com/01org/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml

testcase/path_params/tbox_group/run: blogbench/1SSD-ext4-performance/lkp-bdw-de1

6d311fa7d2c18659 5c279bd9e40624f4ab6e688671
---------------- --------------------------
%stddev change %stddev
\ | \
292048 -10% 262860 blogbench.read_score
6628 ? 4% 50% 9913 ? 4% blogbench.write_score
104681 319% 439016 blogbench.time.involuntary_context_switches
1.178e+08 -27% 86266452 blogbench.time.file_system_inputs
3112606 -8% 2877831 blogbench.time.voluntary_context_switches
209 214 blogbench.time.percent_of_cpu_this_job_got
603 620 blogbench.time.system_time
80955370 37% 1.11e+08 blogbench.time.file_system_outputs
193975 -27% 141449 vmstat.io.bi
152861 35% 205958 vmstat.io.bo
24997 124% 56094 vmstat.system.cs
33317 7% 35633 vmstat.system.in
428.54 ? 10% -30% 300.32 ? 8% iostat.sda.rrqm/s
9670 -28% 6988 iostat.sda.r/s
193974 -27% 141449 iostat.sda.rkB/s
172120 31% 225064 iostat.sda.wkB/s
8583.63 4% 8896.15 iostat.sda.wrqm/s
6074 77% 10748 iostat.sda.w/s
7593732 125% 17107441 perf-stat.context-switches
21914349 ? 3% 24% 27154223 ? 4% perf-stat.dTLB-store-misses
3.952e+11 8% 4.27e+11 perf-stat.dTLB-loads
2.524e+11 7% 2.701e+11 perf-stat.branch-instructions
4.634e+10 6% 4.892e+10 perf-stat.cache-references
1.468e+09 11% 1.627e+09 perf-stat.branch-misses
4.634e+10 6% 4.892e+10 perf-stat.cache-misses
1.314e+12 8% 1.425e+12 perf-stat.instructions
2.289e+08 25% 2.86e+08 ? 4% perf-stat.iTLB-loads
2.279e+08 15% 2.632e+08 ? 3% perf-stat.dTLB-load-misses
0.06 7% 0.06 perf-stat.dTLB-load-miss-rate%
0.59 10% 0.65 perf-stat.ipc
0.01 ? 3% 13% 0.01 ? 4% perf-stat.dTLB-store-miss-rate%
2.291e+11 9% 2.503e+11 perf-stat.dTLB-stores
0.58 4% 0.60 perf-stat.branch-miss-rate%
285094 22% 346640 perf-stat.cpu-migrations



blogbench.write_score

12000 ++------------------------------------------------------------------+
| |
11000 ++ O |
| O O O O O O O O |
O OO O O |
10000 ++ O O O O O O O O O O |
|O O O O O O OO |
9000 ++ |
| O |
8000 ++ |
| |
| |
7000 ** ***.* **** * ***.*** *** * .*** * * **. ** **
| * * ****.* ***** * ** * ***** *.**** *** * ::|
6000 ++----------------------------------------------------------------*-+


[*] bisect-good sample
[O] bisect-bad sample


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Xiaolong


Attachments:
(No filename) (4.99 kB)
config-4.12.0-rc5-00248-g5c279bd (155.56 kB)
job-script (6.89 kB)
job.yaml (4.50 kB)
reproduce (404.00 B)
Download all attachments

2017-07-05 23:45:30

by Ming Lei

[permalink] [raw]
Subject: Re: [lkp-robot] [scsi] 5c279bd9e4: blogbench.read_score -10.0% regression

Hi Xiaolong,

On Mon, Jul 3, 2017 at 10:57 AM, kernel test robot
<[email protected]> wrote:
>
> Greeting,
>
> FYI, we noticed a -10.0% regression of blogbench.read_score due to commit:

Looks like related with mq scheduler, could you test the following patch to see
if your issue can be fixed?

https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-linus&id=32825c45ff8f4cce937ab85b030dc693ceb1aa0a


Thanks,
Ming Lei

2017-07-06 02:54:59

by kernel test robot

[permalink] [raw]
Subject: Re: [lkp-robot] [scsi] 5c279bd9e4: blogbench.read_score -10.0% regression

Hi, Ming Lei

On 07/06, Ming Lei wrote:
>Hi Xiaolong,
>
>On Mon, Jul 3, 2017 at 10:57 AM, kernel test robot
><[email protected]> wrote:
>>
>> Greeting,
>>
>> FYI, we noticed a -10.0% regression of blogbench.read_score due to commit:
>
>Looks like related with mq scheduler, could you test the following patch to see
>if your issue can be fixed?
>
>https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-linus&id=32825c45ff8f4cce937ab85b030dc693ceb1aa0a
>

Here is the comparison of commit 5c279bd9e4 and 32825c45, it does help recover
performance back.

$ compare -at 5c279bd9e40624f4ab6e688671026d6005b066fa 32825c45ff8f4cce937ab85b030dc693ceb1aa0a
tests: 1
testcase/path_params/tbox_group/run: blogbench/1SSD-ext4-performance/lkp-bdw-de1

5c279bd9e40624f4 32825c45ff8f4cce937ab85b03
---------------- --------------------------
%stddev change %stddev
\ | \
262860 36% 358558 blogbench.read_score
9913 ? 4% -33% 6654 blogbench.write_score
86266452 31% 1.129e+08 blogbench.time.file_system_inputs
29.89 22% 36.46 blogbench.time.user_time
214 13% 242 blogbench.time.percent_of_cpu_this_job_got
620 12% 696 blogbench.time.system_time
1.11e+08 -29% 79019314 blogbench.time.file_system_outputs
439016 -69% 135286 blogbench.time.involuntary_context_switches
0.64 22% 0.78 mpstat.cpu.usr%
7.02 ? 4% -47% 3.72 mpstat.cpu.idle%
237216 1344% 3425183 softirqs.RCU
430504 376% 2048486 softirqs.TIMER
188376 211% 586308 softirqs.SCHED
306478 ? 32% -99% 4150 ? 36% softirqs.NET_RX
86266452 31% 1.129e+08 time.file_system_inputs
29.89 22% 36.46 time.user_time

Thanks,
Xiaolong

>
>Thanks,
>Ming Lei

2017-07-06 07:12:34

by Ming Lei

[permalink] [raw]
Subject: Re: [lkp-robot] [scsi] 5c279bd9e4: blogbench.read_score -10.0% regression

On Thu, Jul 6, 2017 at 10:54 AM, Ye Xiaolong <[email protected]> wrote:
> Hi, Ming Lei
>
> On 07/06, Ming Lei wrote:
>>Hi Xiaolong,
>>
>>On Mon, Jul 3, 2017 at 10:57 AM, kernel test robot
>><[email protected]> wrote:
>>>
>>> Greeting,
>>>
>>> FYI, we noticed a -10.0% regression of blogbench.read_score due to commit:
>>
>>Looks like related with mq scheduler, could you test the following patch to see
>>if your issue can be fixed?
>>
>>https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-linus&id=32825c45ff8f4cce937ab85b030dc693ceb1aa0a
>>
>
> Here is the comparison of commit 5c279bd9e4 and 32825c45, it does help recover
> performance back.

That is great, thanks for your test!

This patch has been in block tree, and will be pushed to v4.13.

Thanks,
Ming