LinuxLists.cc - [percpu_ref] 2b0d3d3e4f: reaim.jobs_per

2021-01-10 14:19:40

Subject: [percpu_ref] 2b0d3d3e4f: reaim.jobs_per_min -18.4% regression

Greeting,

FYI, we noticed a

commit: 2b0d3d3e4fcfb19d10f In addition to that,
+------------------+------- | testcase: change | test machine | test parameters | | runtime=300s | | | ucode=0x5003003 +------------------+------- | testcase: change | test machine | test parameters | | mode=process | | nr_task=50% | | test=page_fault2 | | ucode=0x16 +------------------+------- | testcase: change | test machine | test parameters | | mode=process | | nr_task=50% | | test=malloc1 | | ucode=0x2006906 +------------------+------- | testcase: change | test machine | test parameters | | runtime=300s | | | ucode=0x5002f01 +------------------+------- | testcase: change | test machine | test parameters | bs=4k | | | disk=2pmem | | fs=xfs | | ioengine=libaio | | nr_task=50% | | runtime=200s | | rw=randread | | test_size=200G | | time_based=tb | | ucode=0x5002f01 +------------------+------- | testcase: change | test machine | test parameters | class=memory | | | disk=1HDD | | nr_threads=100% | | testtime=10s | | ucode=0x5002f01 +------------------+-------

If you fix the issue, Reported-by: kernel

Details are as below:
---------------------------

To reproduce:

git clone bin/lkp install job.yaml bin/lkp run job.yaml

=========================== compiler/cpufreq_governor/k gcc-9/performance/x86_64-
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- fail:runs %reproduction | | 0:4 -3% 0:4 -3% %stddev %change \ | 11528218 ? 4% 60042 ? 4% -18.4% 0.10 ? 5% +22.4% 174625 1.513e+10 ?147% 1016862 ? 55% -70.9% 145649 ? 16% -30.6% 1570 ? 4% +50.8% 240.50 ? 46% +150.9% 240.50 ? 46% +150.9% 6155 +53.8% 963.75 ? 46% +150.6% 963.75 ? 46% +150.5% 712842 ? 18% +24.0% 7.84 ? 10% +165.6% 54.83 ? 20% +266.1% 3396 ? 10% +23.3% 1690 ? 9% +22.6% 2.95 ? 72% -1.3 22.55 ? 36% +9.7 218409 ?103% -65.0% 9803231 ? 8% 585.32 ? 9% 578.76 90.03 +1.7 2123315 ? 13% 216902 ? 4% +20.4% 82.94 ? 2% +3.9 1453851 ? 11% 23.54 ? 36% +10.3 0.03 ?107% -0.0 595.44 ? 5% 90.62 +1.4 92.13 +2.4 217636 ?103% -65.1% 9768608 ? 8% 576.69 2116240 ? 13% 216186 ? 4% +20.3% 1449221 ? 11% 353.75 ? 40% +56.2% 353.75 ? 40% +56.2% 4.75 ? 22% +794.7% 297.25 ? 33% +67.1% 297.25 ? 33% +67.1% 364.50 ? 27% +37.2% 364.50 ? 27% +37.2% 252.75 ? 33% +68.2% 252.75 ? 33% +68.2% 256.75 ? 29% +85.3% 256.75 ? 29% +85.3% 1905 ? 6% +12.0% 361.00 ? 30% +35.0% 361.00 ? 30% +35.0% 288.75 ? 40% +59.3% 288.75 ? 40% +59.3% 345.00 ? 30% +31.4% 345.00 ? 30% +31.4% 233.25 ? 29% +67.7% 233.25 ? 29% +67.7% 1898 ? 6% +10.5% 181.75 ? 50% +110.5% 181.75 ? 50% +110.5% 220.75 ? 37% +70.3% 220.75 ? 37% +70.3% 208.25 ? 29% +119.1% 208.25 ? 29% +119.1% 245.50 ? 18% +56.5% 245.50 ? 18% +56.5% 210.25 ? 30% +82.0% 210.25 ? 30% +82.0% 353.00 ? 31% +43.2% 353.00 ? 31% +43.2% 212.50 ? 33% +79.5% 212.50 ? 33% +79.5% 212.75 ? 27% +66.3% 212.75 ? 27% +66.3% 42.50 ?105% +771.8% 1904 ? 7% +11.0% 34.75 ? 88% +128.8% 278.50 ? 32% +70.0% 278.50 ? 32% +70.0% 1897 ? 7% +11.1% 243.50 ? 34% +92.3% 243.50 ? 34% +92.3% 248.50 ? 34% +86.9% 248.50 ? 34% +86.9% 248.25 ? 36% +123.3% 248.25 ? 36% +123.3% 245.25 ? 34% +91.3% 245.25 ? 34% +91.3% 256.75 ? 36% +85.5% 256.75 ? 36% +85.5% 1880 ? 7% +10.8% 127.00 ? 81% -70.5% 220.25 ?123% -84.7% 94.00 ? 31% -66.8% 345.50 ? 26% +56.9% 345.50 ? 26% +56.9% 362.25 ? 31% +42.3% 362.25 ? 31% +42.3% 1897 ? 6% +16.4% 207.50 ? 37% +82.4% 207.50 ? 37% +82.4% 344.75 ? 25% +51.2% 344.75 ? 25% +51.2% 194.25 ? 52% +93.8% 194.25 ? 52% +93.8% 209.75 ? 34% +111.1% 209.75 ? 34% +111.1% 240.75 ? 27% +51.0% 240.75 ? 27% +51.0% 207.00 ? 36% +99.0% 207.00 ? 36% +99.0% 224.25 ? 37% +86.1% 224.25 ? 37% +86.1% 275.75 ? 31% +74.0% 275.75 ? 31% +74.0% 301.75 ? 41% +66.2% 301.75 ? 41% +66.2% 41.75 ? 76% +225.7% 1.50 ? 57% +3383.3% 35.75 ? 79% +138.5% 270.00 ? 32% +74.1% 270.00 ? 32% +74.1% 270.50 ? 31% +70.0% 270.50 ? 31% +70.0% 271.00 ? 32% +73.8% 271.00 ? 32% +73.8% 1881 ? 7% +14.6% 266.25 ? 27% +55.4% 266.25 ? 27% +55.4% 312.00 ? 42% +59.6% 312.00 ? 42% +59.6% 95.75 ? 36% -63.4% 380.25 ? 31% +40.1% 380.25 ? 31% +40.1% 251.75 ? 34% +103.7% 251.75 ? 34% +103.7% 298.50 ? 32% +88.9% 298.50 ? 32% +88.9% 350.50 ? 29% +45.1% 350.50 ? 29% +45.1% 28021 ? 5% -14.3% 23196 ? 13% -19.4% 23411 ? 18% -21.0% 23559 ? 17% -20.7% 23284 ? 18% -22.5% 22593 ? 14% -17.5% 23011 ? 12% -23.1% 22987 ? 14% -17.3% 22712 ? 14% -18.4% 22141 ? 13% -15.3% 22745 ? 17% -20.0% 21804 ? 10% -15.1% 20638 ? 10% -12.4% 21668 ? 16% -16.8% 20410 ? 10% -12.5% 21318 ? 17% -16.1% 21459 ? 15% -16.4% 21209 ? 14% -15.9% 21358 ? 14% -16.0% 95166 ? 16% +26.0% 21081 ? 15% -14.5% 21176 ? 14% -14.9% 21994 ? 17% -17.7% 22188 ? 19% -17.8% 21899 ? 21% -17.8% 21750 ? 19% -18.2% 21867 ? 17% -17.0% 22381 ? 16% -17.7% 21269 ? 13% -15.6% 22361 ? 18% -19.0% 21626 ? 19% -18.6% 21584 ? 18% -16.7% 21734 ? 17% -17.6% 21808 ? 18% -19.7% 21955 ? 14% -21.6% 22032 ? 12% -17.3% 21421 ? 8% -16.0% 20917 ? 11% -15.7% 22688 ? 16% -20.9% 20652 ? 15% -20.6% 21169 ? 16% -22.2% 20426 ? 14% -20.0% 21625 ? 17% -21.5% 21946 ? 19% -22.2% 21167 ? 9% -16.0% 21075 ? 13% -18.9% 21493 ? 15% -21.5% 21895 ? 21% -22.9% 22117 ? 15% -19.8% 22486 ? 15% -20.8% 22380 ? 14% -20.2% 22120 ? 13% -18.8% 22114 ? 13% -16.6% 100728 ? 13% +20.6% 22219 ? 19% -18.7% 20735 ? 11% -13.2% 94892 ? 15% +26.0% 20572 ? 8% -11.6% 21251 ? 11% -15.1% 95475 ? 15% +26.9% 21109 ? 11% -14.6% 20821 ? 11% -13.9% 94143 ? 15% +25.8% 20844 ? 9% -12.6% 93752 ? 14% +27.3% 20568 ? 10% -11.5% 22463 ? 16% -17.7% 22775 ? 17% -15.4% 23173 ? 17% -17.1% 22320 ? 16% -17.4% 22639 ? 15% -16.8% 22286 ? 13% -15.3% 22906 ? 17% -16.9% 22821 ? 17% -17.7% 23166 ? 11% -19.2% 22471 ? 10% -16.8% 22829 ? 9% -18.8% 22761 ? 11% -18.4% 22837 ? 13% -18.6% 22845 ? 13% -23.3% 22758 ? 12% -19.5% 23001 ? 13% -20.2% 22963 ? 14% -20.0% 22936 ? 13% -21.1% 22766 ? 13% -17.9% 22959 ? 14% -19.0% 22932 ? 15% -19.1% 22831 ? 12% -19.4% 22669 ? 13% -18.2% 22686 ? 13% -17.5% 21877 ? 10% -16.2% 21869 ? 11% -15.5% 21881 ? 9% -15.4% 21951 ? 10% -17.2% 21683 ? 10% -15.5% 22006 ? 8% -15.9% 22111 ? 13% -19.9% 21272 ? 10% -18.4% 21917 ? 13% -20.5% 21445 ? 12% -18.9% 21787 ? 14% -20.5% 22125 ? 14% -21.8% 22946 ? 15% -20.9% 22885 ? 14% -21.6% 23161 ? 17% -21.2% 23270 ? 16% -21.3% 22469 ? 10% -19.3% 22471 ? 12% -18.1% 22579 ? 14% -21.2% 22837 ? 13% -20.4% 23093 ? 16% -21.3% 22140 ? 11% -18.2% 23049 ? 16% -21.0% 22935 ? 14% -21.7% 22840 ? 15% -21.9% 22834 ? 14% -20.5% 22849 ? 14% -18.2% 22539 ? 12% -17.6% 23145 ? 17% -20.0% 23363 ? 15% -19.7% 22821 ? 16% -19.1% 4231487 ? 14% 2.41 ? 46% -1.2 2.10 ? 54% -1.2 2.08 ? 53% -1.1 0.74 ? 24% -0.5 1.47 ? 16% -0.4 0.70 ? 12% -0.4 1.21 ? 17% -0.4 1.19 ? 16% -0.4 0.96 ? 18% -0.3 0.96 ? 18% -0.3 0.92 ? 18% -0.3 1.81 ? 6% +0.2 2.02 ? 8% +0.3 1.79 ? 6% +0.3 1.79 ? 6% +0.3 0.27 ?100% +0.3 1.27 ? 19% +0.5 1.26 ? 19% +0.5 1.37 ? 16% +0.5 1.35 ? 17% +0.5 1.34 ? 17% +0.5 1.33 ? 18% +0.5 1.34 ? 17% +0.5 0.44 ? 61% +0.6 0.62 ? 59% +0.6 0.54 ? 59% +0.6 0.18 ?173% +0.7 0.18 ?173% +0.7 0.18 ?173% +0.7 0.18 ?173% +0.7 0.18 ?173% +0.7 0.52 ? 61% +0.7 0.52 ? 61% +0.7 0.31 ?102% +0.8 2.60 ? 28% +1.1 3.76 ? 29% +1.5 2.60 ? 36% +2.0 2.50 ? 28% +2.1 0.00 +2.5 7.28 ? 30% +3.5 7.70 ? 28% +3.6 6.87 ? 31% +3.6 6.61 ? 31% +3.6 6.81 ? 31% +3.6 7.17 ? 29% +3.8 7.12 ? 29% +3.8 7.12 ? 29% +3.8 20.83 ? 22% +6.5 19.68 ? 24% +6.9 19.46 ? 24% +7.1 3.93 ? 39% -1.7 2.71 ? 42% -1.2 2.38 ? 49% -1.1 2.36 ? 49% -1.1 0.56 ?126% -0.5 1.47 ? 16% -0.4 1.14 ? 21% -0.4 1.21 ? 17% -0.4 1.19 ? 16% -0.4 0.97 ? 17% -0.3 0.96 ? 18% -0.3 1.11 ? 15% -0.3 0.93 ? 18% -0.3 0.83 ? 18% -0.3 0.86 ? 17% -0.3 1.11 ? 13% -0.3 0.99 ? 15% -0.3 0.71 ? 26% -0.2 0.69 ? 17% -0.2 0.74 ? 24% -0.2 0.66 ? 16% -0.2 0.69 ? 26% -0.2 0.71 ? 25% -0.2 0.63 ? 15% -0.2 0.76 ? 13% -0.2 0.59 ? 14% -0.2 0.56 ? 6% -0.2 0.39 ? 21% -0.2 0.51 ? 15% -0.1 0.56 ? 6% -0.1 0.37 ? 11% -0.1 0.28 ? 31% -0.1 0.29 ? 32% -0.1 0.34 ? 31% -0.1 0.29 ? 19% -0.1 0.26 ? 25% -0.1 0.59 -0.1 0.28 ? 18% -0.1 0.33 ? 22% -0.1 0.56 ? 3% -0.1 0.31 ? 5% -0.1 0.56 ? 4% -0.1 0.48 ? 3% -0.1 0.51 ? 2% -0.1 0.17 ? 32% -0.1 0.20 ? 22% -0.1 0.17 ? 37% -0.1 0.31 ? 4% -0.1 0.12 ? 29% -0.1 0.19 ? 26% -0.1 0.24 ? 21% -0.1 0.23 ? 8% -0.1 0.22 ? 3% -0.1 0.30 ? 6% -0.1 0.18 ? 12% -0.1 0.22 ? 21% -0.1 0.12 ? 47% -0.1 0.20 ? 26% -0.1 0.15 ? 12% -0.1 0.14 ? 41% -0.1 0.21 ? 20% -0.0 0.15 ? 14% -0.0 0.15 ? 14% -0.0 0.13 ? 28% -0.0 0.08 ? 19% -0.0 0.24 ? 2% -0.0 0.15 ? 7% -0.0 0.12 ? 29% -0.0 0.12 ? 29% -0.0 0.18 ? 4% -0.0 0.22 ? 7% -0.0 0.10 ? 24% -0.0 0.10 ? 24% -0.0 0.11 ? 7% -0.0 0.13 ? 11% -0.0 0.11 ? 17% -0.0 0.13 ? 14% -0.0 0.11 ? 12% -0.0 0.21 ? 7% -0.0 0.14 ? 6% -0.0 0.13 ? 10% -0.0 0.11 ? 21% -0.0 0.12 ? 8% -0.0 0.07 ? 14% -0.0 0.09 ? 11% -0.0 0.12 ? 3% -0.0 0.26 ? 3% -0.0 0.09 ? 14% -0.0 0.09 ? 12% -0.0 0.08 ? 5% -0.0 0.08 ? 6% +0.0 0.10 ? 12% +0.0 0.04 ? 59% +0.0 0.01 ?173% +0.0 0.02 ?173% +0.1 0.03 ?100% +0.1 0.04 ?103% +0.1 0.06 ? 66% +0.1 0.09 ? 31% +0.1 0.10 ? 15% +0.1 0.00 +0.1 0.12 ? 41% +0.1 0.06 ? 63% +0.1 0.06 ? 63% +0.1 0.06 ? 63% +0.1 0.06 ? 64% +0.1 0.15 ? 29% +0.1 0.15 ? 29% +0.1 0.12 ? 36% +0.1 0.15 ? 32% +0.1 0.17 ? 29% +0.1 0.00 +0.1 0.12 ? 28% +0.1 0.14 ? 29% +0.1 0.14 ? 29% +0.1 0.14 ? 32% +0.1 0.15 ? 28% +0.1 0.31 ? 4% +0.1 0.13 ? 30% +0.1 0.14 ? 32% +0.1 0.15 ? 35% +0.1 0.55 ? 5% +0.1 0.04 ?101% +0.1 0.12 ? 54% +0.2 0.10 ? 74% +0.2 0.29 ? 29% +0.2 0.27 ? 31% +0.2 0.29 ? 32% +0.2 0.26 ? 36% +0.2 0.19 ? 35% +0.2 0.33 ? 34% +0.2 0.21 ? 33% +0.2 0.34 ? 32% +0.2 0.33 ? 32% +0.3 0.49 ? 28% +0.3 0.49 ? 28% +0.3 0.67 ? 27% +0.3 0.50 ? 26% +0.3 1.13 ? 20% +0.3 0.40 ? 33% +0.3 0.56 ? 30% +0.4 0.55 ? 30% +0.4 1.02 ? 24% +0.4 0.48 ? 31% +0.4 1.39 ? 16% +0.5 1.44 ? 23% +0.5 1.44 ? 23% +0.5 1.52 ? 21% +0.5 1.52 ? 21% +0.5 1.01 ? 34% +0.9 1.02 ? 33% +0.9 1.60 ? 38% +1.7 7.28 ? 31% +2.7 7.28 ? 30% +3.5 7.71 ? 28% +3.6 5.37 ? 35% +4.0 0.97 ? 59% +4.2 21.64 ? 21% +6.3 21.54 ? 21% +6.3 20.84 ? 22% +6.5 19.69 ? 24% +6.9 19.46 ? 24% +7.1 14.05 ? 30% +7.4 14.06 ? 30% +7.5 14.21 ? 30% +7.5 45.93 ? 22% +12.2 47.83 ? 21% +12.2 1.45 ? 16% -0.4 1.19 ? 17% -0.4 1.18 ? 16% -0.3 1.08 ? 15% -0.3 0.81 ? 17% -0.3 0.94 ? 16% -0.3 0.70 ? 26% -0.2 0.55 ? 6% -0.2 0.51 ? 15% -0.1 0.23 ? 15% -0.1 0.19 ? 39% -0.1 0.37 ? 12% -0.1 0.35 ? 19% -0.1 0.27 ? 33% -0.1 0.18 ? 16% -0.1 0.31 ? 23% -0.1 0.17 ? 32% -0.1 0.21 ? 24% -0.1 0.20 ? 22% -0.1 0.13 ? 46% -0.1 0.12 ? 29% -0.1 0.23 ? 21% -0.1 0.12 ? 17% -0.1 0.14 ? 26% -0.1 0.13 ? 10% -0.1 0.14 ? 15% -0.0 0.07 ? 14% -0.0 0.13 ? 14% -0.0 0.10 ? 24% -0.0 0.15 ? 14% -0.0 0.09 ? 13% -0.0 0.07 ? 10% -0.0 0.10 ? 10% -0.0 0.10 ? 11% +0.0 0.00 +0.1 0.00 +0.1 0.01 ?173% +0.1 0.04 ?103% +0.1 0.05 ? 58% +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.04 ?103% +0.1 0.27 ? 33% +0.2 0.12 ? 24% +0.2 0.86 ? 61% +0.9 2.05 ? 66% +2.0 2.34 ? 62% +2.2 2.06 ? 57% +2.4 0.93 ? 63% +4.2

1.35e+07 +--------------- 1.3e+07 |-+ | 1.25e+07 |-+ 1.2e+07 |-+ + | :+ 1.15e+07 |:+ : + 1.1e+07 |:+ 1.05e+07 |-+. .+.: | +.+ + :+ + 1e+07 |-+ 9.5e+06 |-+ O | O O 9e+06 |-+ O O 8.5e+06 +--------------- [*] bisect-good sample
[O] bisect-bad sample

*************************** lkp-csl-2ap4: 192 =========================== compiler/cpufreq_governor/k gcc-9/performance/x86_64-
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- %stddev %change \ | 0.10 ? 4% -7.9% 8954 -3.5% 1719804 373.87 373.87 50141867 3281287 903440 57148 +2.8% 37.04 ? 2% -17.0% 512689 +11.0% 0.21 ? 5% +0.0 75909996 ? 6% 9637 ? 10% -15.6% 240528 ? 26% +76.2% 8929 ? 5% -18.8% 8929 ? 5% -18.8% 25465 ? 6% +1123.1% 73758 ? 7% +546.8% 34434 ? 6% +55.6% 9624 ? 10% -15.6% 60560 ? 26% +76.1% 3263942 ? 5% 18.00 -5.6% 81.00 +1.2% 760.75 +11.4% 5806 -2.1% 13992 ? 3% +9.7% 15299 ? 4% -5.7% 13526 ? 6% +9.6% 14687 ? 4% -9.6% 10180 ? 6% +9.4% 222.50 ? 19% -38.9% 341.25 ? 3% +11.6% 126.25 ? 28% -42.6% 223.25 ? 56% -63.3% 72.50 ? 9% +25.5% 65.25 ? 14% +36.4% 75.50 ? 17% +29.8% 106.75 ? 18% +134.4% 347.50 ? 3% +26.8% 221.75 ? 7% -14.1% 225.00 ? 10% -15.4% 352654 197.25 ? 5% -11.9% 206.75 ? 2% -17.8% 349942 8333 ? 3% -36.9% 8333 ? 3% -36.9% 134.50 ? 17% -25.5% 0.43 ? 44% +284.6% 0.22 ? 39% +4571.0% 0.24 ? 64% -57.5% 0.05 ?173% +1165.8% 0.11 ? 52% +65236.0% 0.50 ?173% +547.7% 0.00 ?173% +21233.3% 3.22 ? 8% +17.3% 193.42 ? 26% -76.3% 510.90 ?100% +1957.9% 147.57 ? 18% +29.4% 202.28 ? 11% +702.2% 3.22 ? 8% +17.3% 3.54 ?169% +402.5% 4.75 ? 9% -34.7% 83.21 ? 33% -73.2% 193.17 ? 26% -54.5% 2.37 ? 49% +191.0% 147.56 ? 18% +29.4% 3783 ? 7% -79.9% 16.11 ? 48% +459.4% 195.61 ? 13% +474.7% 112.38 ? 23% +50.7% 990.53 ? 3% +48.7% 193180 1142 ? 5% -18.1% 128456 +11.0% 128457 +11.0% 2.916e+08 1.295e+09 1.295e+09 2.916e+08 2.918e+08 3708 ? 2% -7.3% 1.577e+09 734770 1.029e+08 -2.7% 1.593e+09 50090667 734770 2.127e+09 2.279e+09 1.518e+08 ? 2% 1.439e+09 1.533e+09 94594773 ? 3% 81169135 ? 2% 17896413 4815275 1.515e+08 4.58 +17.1% 1.328e+10 1.085e+08 2.313e+08 5710 -2.2% 7.07 +3.0% 4.732e+11 4016 -2.4% 1.516e+10 4.05e+09 7160344 5.956e+10 8057 +4.3% 0.18 -5.1% 130812 170.49 77.69 +1.5 23966652 11624715 137592 3.86 +4.5% 7.97 +2.3% 4408 -2.7% 8457 +4.3% 0.13 -2.3% 78.50 +1.6 59.97 +0.8 1.359e+10 1.101e+08 2.353e+08 5768 -2.1% 1.551e+10 4.126e+09 7206955 901608 ? 2% 6.094e+10 133601 24391184 11852943 140341 4.53 ? 11% -0.9 4.40 ? 11% -0.8 4.25 ? 12% -0.8 3.85 ? 13% -0.8 3.85 ? 13% -0.8 4.44 ? 3% -0.4 4.29 ? 3% -0.4 4.47 ? 3% -0.3 4.46 ? 3% -0.3 4.47 ? 3% -0.3 2.32 ? 3% -0.2 1.83 ? 3% -0.1 5.13 ? 9% -0.9 4.74 ? 11% -0.9 4.59 ? 12% -0.8 4.42 ? 12% -0.8 4.47 ? 3% -0.4 4.45 ? 3% -0.4 4.47 ? 3% -0.4 4.47 ? 3% -0.4 4.30 ? 3% -0.4 2.35 ? 3% -0.2 0.98 ? 6% -0.2 1.83 ? 3% -0.1 1.04 ? 2% -0.1 0.72 -0.0 0.59 ? 3% -0.0 0.16 ? 2% -0.0 0.16 ? 2% -0.0 0.10 ? 5% +0.0 0.18 ? 4% +0.0 0.00 +0.1 0.03 ?100% +0.1 0.14 ? 3% +0.1 0.23 +0.1 0.23 ? 12% +0.1 0.18 ? 2% +0.1 0.24 ? 14% +0.1 0.21 ? 3% +0.1 0.61 ? 4% +0.2 0.15 ? 7% +0.2 0.28 ? 4% -0.2 2.33 ? 3% -0.2 1.81 ? 2% -0.1 1.03 ? 2% -0.1 0.42 ? 3% -0.0 0.33 -0.0 0.14 ? 3% -0.0 0.08 ? 5% +0.0 0.11 ? 24% +0.0 0.00 +0.1 0.03 ?100% +0.1 0.20 ? 12% +0.1 0.12 ? 3% +0.1 0.21 ? 3% +0.1 0.15 ? 7% +0.2

*************************** lkp-hsw-4ex1: 144 =========================== compiler/cpufreq_governor/k gcc-9/performance/x86_64-
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- fail:runs %reproduction | | 4:4 6% 4:4 6% 3:4 5% %stddev %change \ | 11917882 165525 +14.5% 11917882 5.751e+09 ? 41% 5.51 -0.9 3075 ? 6% +23.6% 52736 ? 3% +11.3% 52960 ? 3% +11.2% 10628 ? 6% +10.5% 10628 ? 6% +10.5% 37829 ? 4% -28.9% 6200 ? 7% -13.0% 37829 ? 4% -28.9% 63426 ? 8% -25.5% 101256 ? 3% -26.7% 42587 ? 8% +20.2% 68736 ? 6% +25.3% 9.114e+08 ? 2% 9.114e+08 ? 2% 9.009e+08 9.009e+08 8.939e+08 8.939e+08 8.868e+08 ? 3% 8.868e+08 ? 3% 35598 ? 2% -59.8% 1208645 32598 +1.2% 53316 +1.5% 1827 ?125% -97.2% 3.591e+09 3.591e+09 11689 ? 17% -45.3% 42327 ? 21% -46.3% 3.593e+09 3.582e+09 3.59e+09 11689 ? 17% -45.3% 4.538e+08 ? 3% 4.538e+08 ? 3% 6198 ? 7% -13.0% 9456 ? 4% -28.9% 15856 ? 8% -25.5% 4.489e+08 4.488e+08 4.452e+08 4.451e+08 10646 ? 8% +20.2% 4.416e+08 ? 3% 4.415e+08 ? 3% 124943 -17.1% 157.96 ? 21% +3658.8% 47229 ? 3% -27.2% 29466 ? 17% +1441.3% 3501079 ? 3% 12.72 ? 2% +35.2% 35.46 ? 8% +29.6% 3501081 ? 3% 1325375 ? 6% 38786 ? 14% -25.1% 4150 ? 8% -19.5% 35676 ? 16% -25.6% 604.12 ? 4% +14.9% 17716 ? 16% -26.0% 12.21 ? 14% +133.1% 1885 ? 11% -19.3% 18816 ? 10% -25.8% 219.25 ? 4% +23.8% 1952 ? 3% -19.1% 202.42 ? 3% +18.3% 40818 ? 2% -31.4% 9367 ? 9% -18.4% 12582 ? 46% +101.2% 23835 ? 13% -26.3% 30004 ? 22% -35.2% 9330 ? 4% -16.0% 10955 ? 10% -22.2% 9739 ? 10% -19.4% 10422 ? 10% -16.5% 9955 ? 6% -18.4% 10056 ? 6% -21.7% 12538 ? 51% +102.1% 11539 ? 50% +115.6% 10011 ? 9% -21.5% 9823 ? 7% -14.3% 9736 ? 4% -16.7% 9971 ? 7% -16.5% 9735 ? 4% -12.8% 10286 ? 8% -18.6% 9144 ? 8% -10.2% 10268 ? 4% -17.1% 9510 ? 6% -14.8% 9648 ? 10% -16.8% 9861 ? 4% -13.5% 9851 ? 10% -18.8% 12163 ? 58% +90.4% 10364 ? 11% -20.8% 3348 ? 14% +349.4% 29070 ? 20% -44.8% 31270 ? 17% -42.5% 1283184 ? 2% 33.50 ? 34% +235.1% 2909 ? 32% +172.4% 2909 ? 32% +172.4% 3855 ? 63% +61.9% 3855 ? 63% +61.9% 662.75 8.00 ? 42% +393.8% 7.25 ? 17% +441.4% 7378 ? 13% -49.4% 7378 ? 13% -49.4% 4443 ? 16% +53.8% 4443 ? 16% +53.8% 2429 ? 44% +90.4% 2429 ? 44% +90.4% 2309 ? 45% +69.5% 2309 ? 45% +69.5% 2741 ? 57% +98.9% 2741 ? 57% +98.9% 2797 ? 32% +37.2% 2797 ? 32% +37.2% 5760 ? 31% -45.4% 5760 ? 31% -45.4% 654.50 +43.4% 942.75 ? 37% -30.6% 793.50 ? 10% -17.6% 6309 ? 24% -54.7% 6309 ? 24% -54.7% 5327 ? 30% -41.7% 5327 ? 30% -41.7% 3778 ? 66% +66.3% 3778 ? 66% +66.3% 662.25 ? 4% 17.28 -7.5% 1.115e+10 0.52 -0.0 56107202 35.61 +1.8 3.572e+08 9.999e+08 3.60 -16.1% 605.08 -14.0% 1.761e+10 84716478 1.086e+10 91.12 +1.0 35528786 5.787e+10 1633 +3.8% 0.28 +19.1% 285.23 +17.5% 11858192 11851969 2.897e+08 16.96 +1.4 9075374 44586298 11858196 17.28 -7.4% 0.50 -0.0 35.72 +1.8 3.60 -16.0% 582.56 -13.6% 91.23 +1.0 1629 +3.5% 0.28 +19.0% 16.91 +1.4 1462076 1.111e+10 55895802 3.56e+08 9.965e+08 1.755e+10 84430787 1.082e+10 35410048 5.768e+10 11818655 11809592 2.887e+08 9045541 44438814 11818659 1.743e+13 11.82 ? 7% -4.7 5.08 ? 7% -2.3 1.10 ? 9% -0.8 1.07 ? 9% -0.8 1.42 ? 9% -0.7 1.38 ? 9% -0.7 1.69 ? 9% -0.6 1.51 ? 9% -0.6 0.68 ? 10% +0.2 0.70 ? 10% +0.2 0.44 ? 57% +0.2 0.43 ? 57% +0.2 0.00 +0.6 0.72 ? 8% +1.0 0.00 +1.2 2.25 ? 13% +1.2 2.27 ? 12% +1.2 5.23 ? 11% +1.4 5.65 ? 11% +1.4 3.75 ? 12% +1.9 3.79 ? 12% +1.9 5.58 ? 11% +1.9 5.71 ? 11% +1.9 11.84 ? 7% -4.7 5.09 ? 7% -2.3 2.50 ? 9% -1.3 2.70 ? 9% -1.1 2.70 ? 9% -0.9 1.69 ? 9% -0.6 1.10 ? 9% -0.6 0.52 ? 10% -0.4 0.18 ? 10% -0.1 0.10 ? 9% +0.0 0.05 ? 8% +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.71 ? 11% +0.2 0.42 ? 8% +0.8 0.73 ? 8% +1.0 6.07 ? 11% +1.6 6.35 ? 11% +1.6 5.59 ? 11% +1.9 5.71 ? 11% +1.9 6.36 ? 12% +3.3 7.27 ? 12% +3.5 4.82 ? 8% -2.9 5.06 ? 7% -2.3 2.50 ? 9% -1.3 2.69 ? 9% -1.1 2.68 ? 8% -0.9 0.52 ? 9% -0.4 0.18 ? 10% -0.1 0.13 ? 9% +0.1 0.01 ?173% +0.1 0.00 +0.1 0.32 ? 10% +0.2 0.44 ? 10% +0.6 0.38 ? 9% +0.8 7.27 ? 12% +3.5

*************************** lkp-skl-fpga01: 104 =========================== compiler/cpufreq_governor/k gcc-9/performance/x86_64-
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- fail:runs %reproduction | | 0:2 11% 0:2 5% %stddev %change \ | 1217676 23416 -13.0% 1217676 2169754 ? 3% 6.607e+09 ? 66% 23549 ? 3% -7.5% 3974 ? 5% -11.6% 3.646e+08 ? 11% 3.646e+08 ? 11% 26799 ? 25% -74.8% 6950 ? 95% +287.9% 1423 ? 2% -12.8% 1423 ? 2% -12.8% 4704 ? 4% -7.9% 4704 ? 4% -7.9% 1195 +1.1% 15135 ? 52% -82.6% 7.333e+08 7.332e+08 13562 ? 18% -81.4% 60036 ? 15% -80.5% 7.341e+08 3.676e+08 7.342e+08 13562 ? 18% -81.4% 389.50 ? 2% +82.5% 3663 +26.2% 380.00 ? 3% +91.0% 575.00 ? 18% +290.4% 10662 +19.0% 389.50 ? 2% +82.5% 1.821e+08 ? 11% 1.821e+08 ? 11% 28890 ? 23% -58.0% 5040 -17.8% 816.50 -41.1% 13208 -15.4% 9272 ? 99% -99.9% 962811 ? 99% -99.9% 94016 ? 99% -99.9% 9272 ? 99% -99.9% 962811 ? 99% -99.9% 94016 ? 99% -99.9% 1.22 ? 10% -14.4% 2.64 ? 26% +90.9% 21.39 ? 8% +37.2% 49.00 ? 10% -39.2% -16.00 +54.2% 11716 ? 10% -19.2% 1687 ? 2% +74.6% 1559 ? 2% +82.4% 42649 +19.0% 14400 +24.7% 1516 ? 3% +92.0% 42649 +19.0% 2303 ? 18% +289.9% 134586 +11.9% 52831 -15.4% 19878 -17.2% 1336465 3275 -41.3% 52831 -15.4% 19450 ? 83% -83.6% 32978 ? 12% -90.5% 4148 ? 28% +662.9% 20248 ? 84% -84.5% 20356 ? 83% -84.9% 20649 ? 82% -86.3% 34769 ? 8% -84.1% 18485 ? 82% -82.7% 3179 ? 4% +1097.6% 20469 ? 84% -84.7% 20475 ? 84% -46.6% 20919 ? 83% -51.0% 31296 ? 12% -89.6% 20495 ? 84% -85.1% 20518 ? 84% -84.7% 39351 ? 3% -67.4% 20716 ? 83% -29.5% 9714 ? 39% +290.2% 22690 ? 66% -52.6% 20285 ? 84% -28.7% 20274 ? 84% -84.6% 7772 ? 48% +386.1% 36570 ? 2% -74.0% 6234 ? 50% +463.3% 20516 ? 84% -75.0% 38065 -91.7% 20484 ? 84% -84.8% 20571 ? 84% -84.7% 20537 ? 84% -75.9% 3134 +833.3% 35999 ? 5% -35.1% 18.62 +15.8% 6.176e+09 0.48 +0.0 29155676 25.60 +0.9 1.349e+08 5.17 +14.4% 1086 -4.5% 4863474 7.44e+09 4871891 3.183e+09 4.51 +0.1 4946681 1.051e+08 2.827e+10 5802 -3.2% 0.20 -12.5% 168.00 -12.0% 1216995 17897885 2058143 17131463 1216996 18.62 +15.8% 0.47 +0.0 25.62 +0.9 5.17 +14.4% 1084 -4.4% 4.49 +0.1 5714 -3.4% 0.19 -12.5% 6.154e+09 28990877 1.344e+08 4846986 7.413e+09 4855347 3.172e+09 4929905 1.048e+08 2.817e+10 1212867 17835927 2051127 17073809 1212867 8.503e+12 8982 ? 66% -92.0% 5376 ? 46% -40.1% 5376 ? 46% -40.1% 5385 ? 46% -43.1% 5385 ? 46% -43.1% 79.00 ? 50% -74.3% 3398 ? 15% +93.1% 3398 ? 15% +93.1% 68.50 ? 66% -74.2% 2144 ? 33% +267.2% 2144 ? 33% +267.2% 5908 ? 33% -51.0% 5908 ? 33% -51.0% 16.50 ? 3% +540.4% 8.50 ? 88% +586.3% 10.00 ? 40% +930.0% 122.00 ? 5% -86.9% 510.50 ? 8% -49.5% 476.50 ? 4% 2146 ? 33% +267.0% 2146 ? 33% +267.0% 67.00 ? 76% -81.1% 8982 ? 66% -92.0% 5907 ? 33% -51.5% 5907 ? 33% -51.5% 1103 ? 59% -59.3% 68.50 ? 63% -61.6% 3409 ? 15% -16.0% 3409 ? 15% -16.0% 63.00 ? 76% -72.5% 10.00 ? 30% +956.7% 3400 ? 15% -29.8% 3400 ? 15% -29.8% 67.00 ? 68% -81.1% 2160 ? 32% +203.7% 2160 ? 32% +203.7% 13.00 +679.5% 11.00 ? 18% +830.3% 7.50 ? 60% +1268.9% 5376 ? 46% -24.5% 5376 ? 46% -24.5% 13.00 ? 7% +425.6% 7879 -69.7% 7879 -69.7% 1.50 ? 33% +14011.1% 10.00 ? 80% +3200.0% 5374 ? 46% -24.5% 5374 ? 46% -24.5% 8.50 ?100% +1256.9% 9.50 ?100% +1380.7% 353.50 ? 99% -99.2% 24.00 ? 29% +366.7% 3405 ? 15% +125.2% 3405 ? 15% +125.2% 92.50 ? 23% -71.2% 3405 ? 15% +131.2% 3405 ? 15% +131.2% 7878 -69.6% 7878 -69.6% 68.00 ? 75% -73.0% 2857 +175.5% 2857 +175.5% 452.00 +129.6% 5372 ? 46% -24.5% 5372 ? 46% -24.5% 1.00 +9866.7% 5371 ? 46% -64.2% 5371 ? 46% -64.2% 416.50 ? 71% -81.8% 451.00 +149.4% 66.50 ? 66% -81.5% 8.50 ? 29% +1221.6% 7876 -63.7% 7876 -63.7% 61.00 ? 80% -87.4% 13.00 ? 15% +674.4% 63.50 ? 77% -68.5% 926.00 ? 51% -51.3% 2857 +175.7% 2857 +175.7% 547.50 ? 16% -17.3% 5396 ? 45% -19.6% 5396 ? 45% -19.6% 539.00 ? 15% -16.3% 11.50 ? 4% +705.8% 8.50 ? 17% +990.2% 5373 ? 46% -40.0% 5373 ? 46% -40.0% 5383 ? 46% -55.6% 5383 ? 46% -55.6% 55.00 ? 94% -93.3% 11.00 +733.3% 5366 ? 46% -55.2% 5366 ? 46% -55.2% 60.00 ? 76% -84.4% 11.50 ? 4% +700.0% 7875 -43.9% 7875 -43.9% 1325 ? 5% -44.9% 12.26 -1.7 12.14 -1.7 12.03 -1.7 2.77 ? 6% -0.7 1.77 ? 12% -0.5 0.88 ? 8% -0.4 0.86 ? 8% -0.4 1.92 -0.2 1.52 -0.1 1.94 -0.1 1.83 -0.1 1.21 -0.1 1.17 -0.1 1.17 -0.1 0.99 -0.1 0.83 -0.0 0.54 -0.0 0.66 +0.0 0.84 +0.1 11.16 ? 3% +1.5 31.93 +2.0 33.98 +2.2 29.06 +2.3 29.89 +2.4 30.15 +2.5 1.73 ? 6% +2.6 12.27 -1.7 12.16 -1.7 12.04 -1.7 2.77 ? 6% -0.7 1.73 ? 8% -0.5 1.77 ? 12% -0.5 1.94 -0.2 1.94 -0.1 1.89 -0.1 0.56 -0.1 1.17 -0.1 0.35 -0.1 1.17 -0.1 0.41 -0.1 0.84 -0.1 0.40 ? 2% -0.1 1.00 -0.1 0.34 -0.1 0.17 ? 2% -0.0 0.41 ? 3% -0.0 0.24 ? 2% -0.0 0.34 ? 4% -0.0 0.83 -0.0 0.20 -0.0 0.13 -0.0 0.20 ? 4% -0.0 0.55 ? 2% -0.0 0.11 -0.0 0.09 -0.0 0.16 ? 6% -0.0 0.17 -0.0 0.10 -0.0 0.19 -0.0 0.10 ? 5% -0.0 0.52 -0.0 0.10 -0.0 0.07 -0.0 0.13 -0.0 0.07 -0.0 0.07 -0.0 0.09 -0.0 0.08 -0.0 0.08 -0.0 0.41 +0.0 0.07 +0.0 0.14 +0.0 0.07 ? 7% +0.0 0.34 +0.0 0.33 ? 4% +0.0 0.09 +0.0 0.85 +0.0 1.05 +0.1 0.30 ? 3% +0.1 0.61 +0.2 0.77 ? 5% +0.2 0.58 ? 8% +0.2 31.94 +2.0 34.31 +2.2 29.09 +2.3 29.91 +2.5 30.16 +2.5 13.37 ? 3% +3.9 12.02 -1.7 9.63 ? 7% -1.4 1.73 ? 8% -0.5 1.77 ? 12% -0.5 1.42 ? 9% -0.5 0.56 -0.1 0.33 -0.1 0.41 ? 2% -0.1 0.55 -0.1 0.17 ? 2% -0.1 0.36 ? 2% -0.1 0.35 ? 2% -0.0 0.30 -0.0 0.17 ? 2% -0.0 0.15 ? 3% -0.0 0.14 ? 3% -0.0 0.09 -0.0 0.17 -0.0 0.10 -0.0 0.15 ? 3% -0.0 0.15 ? 3% -0.0 0.15 ? 3% -0.0 0.15 -0.0 0.08 ? 6% -0.0 0.10 ? 5% -0.0 0.07 -0.0 0.07 -0.0 0.07 +0.0 0.62 +0.0 0.26 +0.0 0.06 +0.0 0.33 ? 4% +0.0 0.30 ? 3% +0.1 0.81 ? 2% +0.2 0.60 +0.2 0.58 ? 8% +0.2 17.82 ? 2% +0.8 13.30 ? 3% +3.9

*************************** lkp-csl-2sp4: 96 threads =========================== compiler/cpufreq_governor/i gcc-9/performance/socket/
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- fail:runs %reproduction | | :2 50% :2 100%

*************************** lkp-csl-2ap4: 192 =========================== compiler/cpufreq_governor/k gcc-9/performance/x86_64-
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- %stddev %change \ | 0.11 ? 2% -8.2% 8938 -2.9% 1716509 376.47 376.47 791740 50418905 3245026 ? 2% 915679 57413 +2.2% 37.40 ? 4% -17.1% 5794 -4.6% 0.19 ? 10% +0.0 0.09 ? 3% -0.0 79269198 ? 5% 79300987 ? 5% 13153 ? 3% -9.9% 12965 ? 3% -5.7% 14643 ? 2% +13.1% 107963 ? 45% -53.7% 136276 ? 24% -59.7% 48958 ? 65% -60.8% 80284 ? 21% +128.8% 83830 ? 23% +153.6% 5279876 27015 ? 45% -53.7% 34088 ? 24% -59.7% 12250 ? 65% -60.7% 34096 ? 24% -59.7% 20114 ? 21% +128.2% 20998 ? 23% +153.0% 21006 ? 23% +152.9% 14.25 ? 15% -33.3% 1342050 ? 2% 346.25 ? 2% 58.00 ? 67% +101.7% 904477 ? 9% -22.9% 94786 ? 13% -21.0% 7107 ? 5% -5.8% -17.81 +54.9% 5854 ? 6% -7.1% 4066 ? 8% -7.7% 2559 ? 6% -8.1% 1645 ? 9% -8.3% 966.85 ? 8% 2188 ? 6% -8.1% 6675 ? 10% -14.2% 1586 ? 8% -8.2% 716.80 ? 9% -14.4% 66724832 133.50 ? 20% -30.5% 111.50 ? 11% -35.0% 98.75 ? 24% -28.4% 73.75 ? 11% +52.2% 5275 ? 34% +60.9% 5275 ? 34% +60.9% 140.75 ? 34% -46.0% 352.50 ? 3% +13.8% 345.75 +11.1% 304.75 ? 8% +11.4% 273.25 ? 11% -18.1% 258.50 ? 9% -14.7% 204.75 ? 7% -16.7% 245.25 ? 16% -25.7% 203.50 ? 9% -20.4% 127.00 ? 12% +17.1% 131.75 ? 5% +14.2% 193430 2914 ? 2% -9.9% 12178694 ? 13% 449.75 ? 7% -15.5% 1284 ? 6% -28.6% 4939422 1394 +3.0% 33973 -2.1% 268.25 ? 69% +70.5% 16132 ? 24% -29.8% 10077 ? 22% -67.5% 1.308e+09 1.308e+09 3836 ? 3% -9.6% 789083 10560572 1.585e+09 738463 1.031e+08 1.602e+09 50366944 738463 2.139e+09 2.291e+09 1.515e+08 ? 4% 1.447e+09 1.542e+09 94366150 ? 4% 82472355 97789 ? 3% -6.1% 17986006 4831818 1.532e+08 1.325e+10 1.084e+08 5701 -4.6% 7.08 +3.1% 4036 -2.0% 1.513e+10 4.042e+09 6991250 1009631 -15.3% 5.943e+10 0.18 -4.6% 130861 170.05 6009 -4.2% 78.56 +1.0 23838801 6329763 11697762 136870 3.88 +4.0% 7.98 +2.6% 4410 -1.9% 0.13 -2.5% 79.26 +1.3 59.71 +1.0 1.355e+10 1.099e+08 5751 -4.5% 1.547e+10 4.113e+09 7030143 ? 2% 905221 -14.8% 6.075e+10 133605 5974 -4.2% 24223707 6339047 11918840 139580 5.98 ? 20% -1.7 4.49 ? 26% -1.7 4.42 ? 26% -1.7 15.79 ? 6% -1.4 1.79 ? 15% +0.6 0.64 ? 71% +0.9 36.99 ? 3% +2.8 36.38 ? 3% +3.0 6.17 ? 20% -1.8 16.58 ? 6% -1.6 78.11 -0.9 4.75 ? 14% -0.8 1.20 ? 7% -0.3 0.33 ? 28% -0.2 0.33 ? 28% -0.2 0.96 -0.1 1.04 ? 2% -0.1 0.54 -0.0 0.61 ? 4% -0.0 0.36 ? 4% -0.0 0.09 ? 4% +0.0 0.18 ? 5% +0.0 0.10 ? 17% +0.0 0.09 ? 20% +0.0 0.04 ? 60% +0.0 0.16 ? 23% +0.1 0.00 +0.1 0.16 ? 23% +0.1 0.22 ? 20% +0.1 0.23 ? 21% +0.1 0.01 ?173% +0.1 0.23 ? 4% +0.1 0.19 ? 5% +0.1 0.23 ? 3% +0.1 0.23 ? 4% +0.2 0.14 ? 16% +0.2 0.30 ? 31% +0.2 0.22 ? 14% +0.2 0.16 ? 5% +0.2 0.63 ? 12% +0.2 0.99 ? 26% +0.6 1.80 ? 15% +0.6 78.11 -0.9 0.28 ? 10% -0.2 0.08 ? 10% +0.0 0.00 +0.1 0.01 ?173% +0.1 0.42 ? 14% +0.1 0.20 ? 4% +0.1 0.13 ? 15% +0.1 0.22 ? 7% +0.1 0.30 ? 31% +0.2 0.16 ? 5% +0.2

*************************** lkp-csl-2sp6: 96 threads =========================== bs/compiler/cpufreq_governo 4k/gcc-9/performance/2pme
commit:
cf785af193 ("block: 2b0d3d3e4f ("percpu_ref:
cf785af19319f9fc 2b0d3d3e4f ---------------- ---------- %stddev %change \ | 0.10 ? 16% -0.1 0.82 ? 2% +0.1 17300 -4.8% 336083 6007071 4429022 8889 +5.0% 196778 7.094e+09 328976 804.97 ? 3% 8.868e+08 4.13 ? 3% -9.6% 4.17 ? 3% -0.4 2971 ? 10% -17.1% 17366856 5408 -2.5% 9848 ? 9% -19.4% 9560 ? 19% +44.4% 10360 ? 15% +32.6% 8973 ? 12% +36.0% 14698 ? 10% -23.5% 2008 +10.1% 2008 +10.1% 4030 ? 15% -100.0% 4030 ? 15% -100.0% 4273 ? 8% +11.5% 4273 ? 8% +11.5% 1987 ? 13% +30.1% 1987 ? 13% +30.1% 926.00 ? 4% +8.9% 262223 4446 ? 8% -12.8% 12208 -3.6% 320735 ? 7% 8.214e+08 ? 2% 8.214e+08 ? 2% 12234 -3.4% 32372075 8.554e+08 8.785e+08 3.547e+09 6.603e+08 7.124e+08 6.603e+08 7.124e+08 0.50 ?173% +1.3e+05% 65651 ? 3% -3.6% 378.75 ? 6% -22.4% 313.75 ? 14% -25.0% 690.25 ? 3% 4085 ? 13% +68.1% 4085 ? 13% +68.1% 671.00 ? 3% +14.8% 53.00 ? 52% +67.0% 399.00 ? 5% -19.1% 4277 ? 36% +62.0% 4277 ? 36% +62.0% 3663 ? 34% +60.5% 3663 ? 34% +60.5% 333.75 ? 20% -29.1% 3167 ? 34% +76.1% 3167 ? 34% +76.1% 1776 ? 20% +206.2% 1776 ? 20% +206.2% 5996 ? 15% +20.0% 1.58e+10 0.59 ? 3% -0.0 91868813 ? 4% 7.197e+08 ? 2% 5406 -2.5% 1.68 +4.4% 127.32 197.31 2.079e+10 1.215e+10 79953507 ? 8% 5545653 ? 11% 8.065e+10 1021 ? 6% +12.9% 0.60 -4.3% 520.33 1.111e+08 ? 4% 0.58 ? 3% -0.0 1.67 +4.6% 187.16 ? 2% 1014 ? 6% +13.2% 0.60 -4.4% 1.572e+10 91415445 ? 4% 7.162e+08 ? 2% 5377 -2.5% 126.96 2.069e+10 1.209e+10 79559944 ? 8% 5518625 ? 11% 8.025e+10 1.106e+08 ? 4% 1.622e+13 13.83 ? 3% -2.0 13.74 ? 3% -2.0 11.37 ? 3% -1.7 11.32 ? 3% -1.7 6.72 ? 2% -1.0 3.68 ? 3% -0.6 3.62 ? 3% -0.6 3.50 ? 3% -0.6 3.96 ? 3% -0.5 0.64 ? 4% -0.4 2.11 ? 4% -0.3 2.04 ? 4% -0.3 1.35 ? 6% -0.3 1.00 ? 9% -0.2 0.97 ? 6% -0.2 1.22 ? 2% -0.2 1.28 ? 2% -0.2 1.05 ? 4% -0.2 1.17 ? 3% -0.1 0.90 ? 4% -0.1 0.89 ? 6% -0.1 0.88 ? 2% -0.1 0.85 ? 4% -0.1 0.95 ? 2% -0.1 0.80 ? 2% -0.1 0.71 -0.1 0.00 +0.6 13.84 ? 3% -2.0 13.74 ? 3% -2.0 11.37 ? 3% -1.7 11.33 ? 3% -1.7 6.08 -1.0 7.30 -1.0 4.07 ? 2% -0.6 3.68 ? 3% -0.6 3.63 ? 3% -0.6 3.97 ? 3% -0.5 3.11 ? 2% -0.4 3.19 ? 3% -0.4 2.30 ? 3% -0.3 1.98 -0.3 0.65 ? 3% -0.2 1.04 ? 6% -0.2 0.72 ? 2% -0.2 1.23 ? 2% -0.2 1.09 ? 4% -0.2 1.28 ? 2% -0.2 1.06 ? 4% -0.2 0.84 ? 3% -0.2 1.11 ? 3% -0.2 1.17 ? 3% -0.1 0.64 ? 4% -0.1 0.90 ? 4% -0.1 0.89 ? 2% -0.1 0.90 ? 6% -0.1 0.97 ? 2% -0.1 0.96 ? 2% -0.1 0.81 ? 2% -0.1 0.59 ? 5% -0.1 0.42 ? 6% -0.1 0.54 ? 2% -0.1 0.42 ? 3% -0.1 0.49 ? 2% -0.1 0.34 ? 3% -0.1 0.41 ? 3% -0.1 0.44 ? 2% -0.1 0.30 ? 2% -0.1 0.25 ? 4% -0.1 0.25 ? 7% -0.1 0.32 ? 2% -0.0 0.23 ? 6% -0.0 0.23 ? 8% -0.0 0.33 ? 4% -0.0 0.25 ? 5% -0.0 0.35 ? 4% -0.0 0.21 ? 4% -0.0 0.13 ? 9% -0.0 0.40 ? 4% -0.0 0.18 ? 7% -0.0 0.28 -0.0 0.21 ? 2% -0.0 0.26 -0.0 0.14 ? 5% -0.0 0.18 ? 7% -0.0 0.23 ? 3% -0.0 0.13 ? 5% -0.0 0.12 ? 4% -0.0 0.06 -0.0 0.16 ? 6% -0.0 0.11 ? 6% -0.0 0.01 ?173% +0.1 0.08 ? 8% +0.1 0.46 ? 3% +0.1 0.27 ? 4% +0.2 0.44 ? 5% +0.2 0.56 ? 2% +0.2 0.73 ? 2% +0.2 0.57 ? 2% +0.2 0.61 ? 2% +0.2 3.85 ? 2% -0.6 0.59 ? 9% -0.4 2.24 ? 3% -0.3 1.97 -0.3 0.72 ? 2% -0.2 1.09 ? 3% -0.2 0.84 ? 3% -0.2 0.41 ? 3% -0.1 0.51 ? 4% -0.1 0.41 ? 6% -0.1 0.36 ? 2% -0.1 0.34 ? 6% -0.1 0.29 ? 2% -0.1 0.28 ? 5% -0.1 0.21 ? 3% -0.0 0.20 ? 3% -0.0 0.31 ? 3% -0.0 0.18 ? 2% -0.0 0.19 ? 5% -0.0 0.10 ? 8% -0.0 0.26 -0.0 0.21 ? 3% -0.0 0.23 ? 4% -0.0 0.18 ? 7% -0.0 0.15 ? 8% -0.0 0.17 ? 4% -0.0 0.19 ? 2% -0.0 0.14 ? 8% -0.0 0.19 ? 10% -0.0 0.12 ? 3% -0.0 0.14 ? 3% -0.0 0.12 ? 3% -0.0 0.10 ? 4% -0.0 0.11 ? 4% -0.0 0.09 ? 9% -0.0 0.09 ? 8% -0.0 0.10 ? 8% -0.0 0.10 ? 8% -0.0 0.09 ? 4% -0.0 0.04 ? 57% +0.0 0.01 ?173% +0.1 0.34 ? 5% +0.1 0.08 ? 5% +0.1 0.24 ? 4% +0.1 0.49 ? 2% +0.1 0.44 ? 5% +0.2

Disclaimer:
Results have been for informational design or configuration

Thanks,
Oliver Sang

-18.4% regression of reaim.jobs_per_min due to commit:
9a82910b8f0f05c56ee3e ("percpu_ref: reduce memory footprint of percpu_ref in fast path")
l.org/cgit/linux/kernel/git/torvalds/linux.git">https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
REAIM is an updated and improved version of AIM 7 benchmark.
/sourceforge.net/projects/re-aim-7/">https://sourceforge.net/projects/re-aim-7/
the commit also has significant impact on the following tests:
--------------------------------------------------------------------+
| vm-scalability: vm-scalability.throughput -2.8% regression |
| 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| cpufreq_governor=performance |
|
| test=lru-file-mmap-read-rand |
|
--------------------------------------------------------------------+
| will-it-scale: will-it-scale.per_process_ops 14.5% improvement |
| 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory |
| cpufreq_governor=performance |
|
|
|
|
--------------------------------------------------------------------+
| will-it-scale: will-it-scale.per_process_ops -13.0% regression |
| 104 threads Skylake with 192G memory |
| cpufreq_governor=performance |
|
|
|
|
--------------------------------------------------------------------+
| vm-scalability: vm-scalability.throughput -2.3% regression |
| 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
| cpufreq_governor=performance |
|
| test=lru-file-mmap-read-rand |
|
--------------------------------------------------------------------+
| fio-basic: fio.read_iops -4.8% regression |
| 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
|
| cpufreq_governor=performance |
|
|
|
|
|
|
|
|
|
--------------------------------------------------------------------+
| stress-ng: stress-ng.stackmmap.ops_per_sec -45.4% regression |
| 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
|
| cpufreq_governor=performance |
|
|
|
|
--------------------------------------------------------------------+
kindly add following tag
test robot <[email protected]>
----------------------------------------------------------------------->
href="https://github.com/intel/lkp-tests.git">https://github.com/intel/lkp-tests.git
# job file is attached in this email
==============================================================
config/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/short/reaim/0x5002f01
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
fail:runs
|
0:4 perf-profile.children.cycles-pp.error_entry
0:4 perf-profile.self.cycles-pp.error_entry
%stddev
\
-18.4% 9411679 ? 4% reaim.jobs_per_min
49019 ? 4% reaim.jobs_per_min_child
0.13 ? 3% reaim.parent_time
-2.0% 171063 reaim.time.major_page_faults
-94.5% 8.371e+08 ? 62% cpuidle.C6.time
296190 ? 10% cpuidle.POLL.time
101128 ? 6% cpuidle.POLL.usage
2367 ? 32% numa-vmstat.node1.nr_mapped
603.50 ? 34% numa-vmstat.node2.nr_active_anon
603.50 ? 34% numa-vmstat.node2.nr_zone_active_anon
9468 ? 32% numa-meminfo.node1.Mapped
2415 ? 34% numa-meminfo.node2.Active
2414 ? 34% numa-meminfo.node2.Active(anon)
884148 ? 22% numa-meminfo.node2.MemUsed
20.82 ? 47% sched_debug.cfs_rq:/.load_avg.avg
200.73 ? 70% sched_debug.cfs_rq:/.load_avg.stddev
4186 ? 10% sched_debug.cpu.sched_count.stddev
2073 ? 10% sched_debug.cpu.sched_goidle.stddev
1.65 perf-stat.i.branch-miss-rate%
32.23 ? 9% perf-stat.i.cache-miss-rate%
76415 ? 3% perf-stat.i.dTLB-store-misses
+7.7% 10559584 perf-stat.i.iTLB-load-misses
-8.5% 535.65 ? 2% perf-stat.i.instructions-per-iTLB-miss
-1.9% 567.72 perf-stat.i.major-faults
91.72 perf-stat.i.node-load-miss-rate%
+42.7% 3029950 ? 12% perf-stat.i.node-load-misses
261168 ? 5% perf-stat.i.node-loads
86.81 perf-stat.i.node-store-miss-rate%
+21.5% 1766524 ? 8% perf-stat.i.node-store-misses
33.82 ? 10% perf-stat.overall.cache-miss-rate%
0.01 ? 2% perf-stat.overall.dTLB-store-miss-rate%
-6.3% 557.82 perf-stat.overall.instructions-per-iTLB-miss
91.99 perf-stat.overall.node-load-miss-rate%
94.54 perf-stat.overall.node-store-miss-rate%
75985 ? 3% perf-stat.ps.dTLB-store-misses
+7.5% 10503288 perf-stat.ps.iTLB-load-misses
-2.2% 564.02 perf-stat.ps.major-faults
+42.4% 3012930 ? 12% perf-stat.ps.node-load-misses
259980 ? 5% perf-stat.ps.node-loads
+21.2% 1755769 ? 8% perf-stat.ps.node-store-misses
552.50 ? 12% interrupts.CPU0.NMI:Non-maskable_interrupts
552.50 ? 12% interrupts.CPU0.PMI:Performance_monitoring_interrupts
42.50 ? 97% interrupts.CPU0.TLB:TLB_shootdowns
496.75 ? 10% interrupts.CPU101.NMI:Non-maskable_interrupts
496.75 ? 10% interrupts.CPU101.PMI:Performance_monitoring_interrupts
500.25 ? 5% interrupts.CPU106.NMI:Non-maskable_interrupts
500.25 ? 5% interrupts.CPU106.PMI:Performance_monitoring_interrupts
425.00 ? 22% interrupts.CPU107.NMI:Non-maskable_interrupts
425.00 ? 22% interrupts.CPU107.PMI:Performance_monitoring_interrupts
475.75 ? 9% interrupts.CPU108.NMI:Non-maskable_interrupts
475.75 ? 9% interrupts.CPU108.PMI:Performance_monitoring_interrupts
2133 ? 6% interrupts.CPU11.CAL:Function_call_interrupts
487.50 ? 7% interrupts.CPU11.NMI:Non-maskable_interrupts
487.50 ? 7% interrupts.CPU11.PMI:Performance_monitoring_interrupts
460.00 ? 12% interrupts.CPU115.NMI:Non-maskable_interrupts
460.00 ? 12% interrupts.CPU115.PMI:Performance_monitoring_interrupts
453.50 ? 11% interrupts.CPU116.NMI:Non-maskable_interrupts
453.50 ? 11% interrupts.CPU116.PMI:Performance_monitoring_interrupts
391.25 ? 31% interrupts.CPU123.NMI:Non-maskable_interrupts
391.25 ? 31% interrupts.CPU123.PMI:Performance_monitoring_interrupts
2097 ? 3% interrupts.CPU124.CAL:Function_call_interrupts
382.50 ? 37% interrupts.CPU124.NMI:Non-maskable_interrupts
382.50 ? 37% interrupts.CPU124.PMI:Performance_monitoring_interrupts
376.00 ? 31% interrupts.CPU126.NMI:Non-maskable_interrupts
376.00 ? 31% interrupts.CPU126.PMI:Performance_monitoring_interrupts
456.25 ? 26% interrupts.CPU135.NMI:Non-maskable_interrupts
456.25 ? 26% interrupts.CPU135.PMI:Performance_monitoring_interrupts
384.25 ? 27% interrupts.CPU136.NMI:Non-maskable_interrupts
384.25 ? 27% interrupts.CPU136.PMI:Performance_monitoring_interrupts
382.75 ? 40% interrupts.CPU139.NMI:Non-maskable_interrupts
382.75 ? 40% interrupts.CPU139.PMI:Performance_monitoring_interrupts
505.50 ? 12% interrupts.CPU14.NMI:Non-maskable_interrupts
505.50 ? 12% interrupts.CPU14.PMI:Performance_monitoring_interrupts
381.50 ? 41% interrupts.CPU141.NMI:Non-maskable_interrupts
381.50 ? 41% interrupts.CPU141.PMI:Performance_monitoring_interrupts
353.75 ? 30% interrupts.CPU142.NMI:Non-maskable_interrupts
353.75 ? 30% interrupts.CPU142.PMI:Performance_monitoring_interrupts
370.50 ?142% interrupts.CPU144.RES:Rescheduling_interrupts
2114 ? 10% interrupts.CPU145.CAL:Function_call_interrupts
79.50 ? 71% interrupts.CPU145.RES:Rescheduling_interrupts
473.50 ? 22% interrupts.CPU146.NMI:Non-maskable_interrupts
473.50 ? 22% interrupts.CPU146.PMI:Performance_monitoring_interrupts
2107 ? 6% interrupts.CPU152.CAL:Function_call_interrupts
468.25 ? 22% interrupts.CPU158.NMI:Non-maskable_interrupts
468.25 ? 22% interrupts.CPU158.PMI:Performance_monitoring_interrupts
464.50 ? 23% interrupts.CPU159.NMI:Non-maskable_interrupts
464.50 ? 23% interrupts.CPU159.PMI:Performance_monitoring_interrupts
554.25 ? 34% interrupts.CPU160.NMI:Non-maskable_interrupts
554.25 ? 34% interrupts.CPU160.PMI:Performance_monitoring_interrupts
469.25 ? 19% interrupts.CPU161.NMI:Non-maskable_interrupts
469.25 ? 19% interrupts.CPU161.PMI:Performance_monitoring_interrupts
476.25 ? 19% interrupts.CPU162.NMI:Non-maskable_interrupts
476.25 ? 19% interrupts.CPU162.PMI:Performance_monitoring_interrupts
2084 ? 7% interrupts.CPU165.CAL:Function_call_interrupts
37.50 ? 54% interrupts.CPU175.RES:Rescheduling_interrupts
33.75 ? 59% interrupts.CPU177.RES:Rescheduling_interrupts
31.25 ? 79% interrupts.CPU179.RES:Rescheduling_interrupts
542.00 ? 8% interrupts.CPU2.NMI:Non-maskable_interrupts
542.00 ? 8% interrupts.CPU2.PMI:Performance_monitoring_interrupts
515.50 ? 10% interrupts.CPU22.NMI:Non-maskable_interrupts
515.50 ? 10% interrupts.CPU22.PMI:Performance_monitoring_interrupts
2207 ? 5% interrupts.CPU27.CAL:Function_call_interrupts
378.50 ? 37% interrupts.CPU28.NMI:Non-maskable_interrupts
378.50 ? 37% interrupts.CPU28.PMI:Performance_monitoring_interrupts
521.25 ? 8% interrupts.CPU3.NMI:Non-maskable_interrupts
521.25 ? 8% interrupts.CPU3.PMI:Performance_monitoring_interrupts
376.50 ? 27% interrupts.CPU38.NMI:Non-maskable_interrupts
376.50 ? 27% interrupts.CPU38.PMI:Performance_monitoring_interrupts
442.75 ? 31% interrupts.CPU39.NMI:Non-maskable_interrupts
442.75 ? 31% interrupts.CPU39.PMI:Performance_monitoring_interrupts
363.50 ? 30% interrupts.CPU41.NMI:Non-maskable_interrupts
363.50 ? 30% interrupts.CPU41.PMI:Performance_monitoring_interrupts
412.00 ? 26% interrupts.CPU42.NMI:Non-maskable_interrupts
412.00 ? 26% interrupts.CPU42.PMI:Performance_monitoring_interrupts
417.25 ? 26% interrupts.CPU44.NMI:Non-maskable_interrupts
417.25 ? 26% interrupts.CPU44.PMI:Performance_monitoring_interrupts
479.75 ? 19% interrupts.CPU49.NMI:Non-maskable_interrupts
479.75 ? 19% interrupts.CPU49.PMI:Performance_monitoring_interrupts
501.50 ? 7% interrupts.CPU5.NMI:Non-maskable_interrupts
501.50 ? 7% interrupts.CPU5.PMI:Performance_monitoring_interrupts
136.00 ? 55% interrupts.CPU54.RES:Rescheduling_interrupts
52.25 ?133% interrupts.CPU54.TLB:TLB_shootdowns
85.25 ? 78% interrupts.CPU57.RES:Rescheduling_interrupts
470.00 ? 20% interrupts.CPU61.NMI:Non-maskable_interrupts
470.00 ? 20% interrupts.CPU61.PMI:Performance_monitoring_interrupts
459.75 ? 39% interrupts.CPU64.NMI:Non-maskable_interrupts
459.75 ? 39% interrupts.CPU64.PMI:Performance_monitoring_interrupts
471.00 ? 15% interrupts.CPU65.NMI:Non-maskable_interrupts
471.00 ? 15% interrupts.CPU65.PMI:Performance_monitoring_interrupts
2156 ? 5% interrupts.CPU70.CAL:Function_call_interrupts
413.75 ? 30% interrupts.CPU70.NMI:Non-maskable_interrupts
413.75 ? 30% interrupts.CPU70.PMI:Performance_monitoring_interrupts
498.00 ? 6% interrupts.CPU8.NMI:Non-maskable_interrupts
498.00 ? 6% interrupts.CPU8.PMI:Performance_monitoring_interrupts
35.00 ? 77% interrupts.CPU84.RES:Rescheduling_interrupts
532.75 ? 14% interrupts.CPU96.NMI:Non-maskable_interrupts
532.75 ? 14% interrupts.CPU96.PMI:Performance_monitoring_interrupts
512.75 ? 10% interrupts.CPU97.NMI:Non-maskable_interrupts
512.75 ? 10% interrupts.CPU97.PMI:Performance_monitoring_interrupts
564.00 ? 11% interrupts.CPU98.NMI:Non-maskable_interrupts
564.00 ? 11% interrupts.CPU98.PMI:Performance_monitoring_interrupts
508.50 ? 7% interrupts.CPU99.NMI:Non-maskable_interrupts
508.50 ? 7% interrupts.CPU99.PMI:Performance_monitoring_interrupts
24008 ? 7% softirqs.CPU0.RCU
18707 ? 3% softirqs.CPU100.RCU
18502 ? 3% softirqs.CPU101.RCU
18675 ? 2% softirqs.CPU102.RCU
18045 softirqs.CPU104.RCU
18650 ? 3% softirqs.CPU105.RCU
17689 ? 11% softirqs.CPU106.RCU
19020 ? 4% softirqs.CPU107.RCU
18536 ? 3% softirqs.CPU108.RCU
18755 ? 6% softirqs.CPU109.RCU
18205 ? 2% softirqs.CPU110.RCU
18513 ? 2% softirqs.CPU111.RCU
18073 softirqs.CPU112.RCU
18027 ? 2% softirqs.CPU113.RCU
17849 softirqs.CPU114.RCU
17884 softirqs.CPU115.RCU
17930 ? 2% softirqs.CPU117.RCU
17833 softirqs.CPU118.RCU
17947 softirqs.CPU119.RCU
119921 ? 16% softirqs.CPU124.NET_RX
18019 ? 2% softirqs.CPU128.RCU
18029 ? 3% softirqs.CPU129.RCU
18096 ? 3% softirqs.CPU130.RCU
18236 ? 3% softirqs.CPU131.RCU
18002 ? 2% softirqs.CPU133.RCU
17794 ? 2% softirqs.CPU134.RCU
18152 ? 4% softirqs.CPU136.RCU
18424 softirqs.CPU137.RCU
17947 ? 2% softirqs.CPU138.RCU
18123 ? 3% softirqs.CPU139.RCU
17605 ? 3% softirqs.CPU140.RCU
17974 ? 3% softirqs.CPU141.RCU
17908 ? 3% softirqs.CPU142.RCU
17509 ? 8% softirqs.CPU143.RCU
17213 ? 7% softirqs.CPU149.RCU
18222 ? 6% softirqs.CPU157.RCU
18000 softirqs.CPU16.RCU
17636 ? 5% softirqs.CPU160.RCU
17939 softirqs.CPU17.RCU
16398 ? 3% softirqs.CPU171.RCU
16471 ? 3% softirqs.CPU174.RCU
16332 ? 4% softirqs.CPU175.RCU
16971 ? 2% softirqs.CPU177.RCU
17066 ? 3% softirqs.CPU179.RCU
17779 softirqs.CPU18.RCU
17085 ? 2% softirqs.CPU180.RCU
16873 ? 2% softirqs.CPU184.RCU
16891 ? 3% softirqs.CPU189.RCU
17740 softirqs.CPU19.RCU
17799 softirqs.CPU20.RCU
17849 softirqs.CPU21.RCU
17970 softirqs.CPU22.RCU
18434 ? 3% softirqs.CPU23.RCU
121520 ? 16% softirqs.CPU24.NET_RX
18069 ? 2% softirqs.CPU24.RCU
17990 ? 2% softirqs.CPU25.RCU
119557 ? 17% softirqs.CPU26.NET_RX
18175 ? 2% softirqs.CPU26.RCU
18042 ? 2% softirqs.CPU27.RCU
121158 ? 17% softirqs.CPU28.NET_RX
18021 ? 2% softirqs.CPU28.RCU
17932 ? 2% softirqs.CPU29.RCU
118430 ? 18% softirqs.CPU30.NET_RX
18209 ? 2% softirqs.CPU30.RCU
119326 ? 17% softirqs.CPU31.NET_RX
18199 softirqs.CPU31.RCU
18492 ? 2% softirqs.CPU32.RCU
19277 ? 2% softirqs.CPU34.RCU
19211 ? 3% softirqs.CPU35.RCU
18440 ? 2% softirqs.CPU38.RCU
18845 ? 3% softirqs.CPU41.RCU
18867 ? 2% softirqs.CPU42.RCU
19035 ? 3% softirqs.CPU43.RCU
18776 ? 6% softirqs.CPU44.RCU
18714 ? 3% softirqs.CPU48.RCU
18693 ? 4% softirqs.CPU49.RCU
18542 ? 3% softirqs.CPU50.RCU
18566 ? 3% softirqs.CPU51.RCU
18595 ? 3% softirqs.CPU52.RCU
17532 ? 6% softirqs.CPU53.RCU
18324 ? 3% softirqs.CPU54.RCU
18358 ? 4% softirqs.CPU55.RCU
18381 ? 3% softirqs.CPU56.RCU
18105 ? 6% softirqs.CPU57.RCU
18699 ? 4% softirqs.CPU58.RCU
18598 ? 4% softirqs.CPU59.RCU
18562 ? 4% softirqs.CPU60.RCU
18413 ? 3% softirqs.CPU61.RCU
18541 ? 4% softirqs.CPU62.RCU
18720 ? 5% softirqs.CPU63.RCU
18323 ? 3% softirqs.CPU64.RCU
18475 ? 4% softirqs.CPU66.RCU
18508 ? 4% softirqs.CPU67.RCU
18171 ? 4% softirqs.CPU69.RCU
18317 ? 4% softirqs.CPU70.RCU
18503 ? 4% softirqs.CPU71.RCU
17711 ? 3% softirqs.CPU72.RCU
17352 ? 3% softirqs.CPU74.RCU
17417 ? 3% softirqs.CPU75.RCU
17381 ? 4% softirqs.CPU76.RCU
17325 ? 4% softirqs.CPU77.RCU
17310 ? 3% softirqs.CPU78.RCU
18160 ? 3% softirqs.CPU80.RCU
17948 ? 2% softirqs.CPU81.RCU
18245 ? 2% softirqs.CPU82.RCU
18307 ? 2% softirqs.CPU83.RCU
18126 ? 2% softirqs.CPU84.RCU
18393 ? 2% softirqs.CPU86.RCU
17788 ? 2% softirqs.CPU87.RCU
18174 ? 2% softirqs.CPU88.RCU
18175 ? 2% softirqs.CPU89.RCU
18108 ? 2% softirqs.CPU90.RCU
18213 ? 3% softirqs.CPU91.RCU
17961 ? 2% softirqs.CPU92.RCU
17844 ? 2% softirqs.CPU93.RCU
18154 ? 2% softirqs.CPU94.RCU
18683 ? 2% softirqs.CPU95.RCU
18564 ? 3% softirqs.CPU96.RCU
18520 ? 4% softirqs.CPU97.RCU
18764 ? 6% softirqs.CPU98.RCU
18466 ? 2% softirqs.CPU99.RCU
-17.1% 3506041 softirqs.RCU
1.19 ? 19% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_on_stack
0.93 ? 18% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.93 ? 17% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
0.27 ?100% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
1.03 ? 13% perf-profile.calltrace.cycles-pp.mul_short.mul_short
0.28 ?100% perf-profile.calltrace.cycles-pp.semop
0.84 ? 16% perf-profile.calltrace.cycles-pp.mul_long.mul_long
0.83 ? 13% perf-profile.calltrace.cycles-pp.mul_int.mul_int
0.63 ? 20% perf-profile.calltrace.cycles-pp.ret_from_fork
0.63 ? 20% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
0.61 ? 15% perf-profile.calltrace.cycles-pp.random_r
2.05 ? 5% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
2.27 ? 4% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.05 ? 5% perf-profile.calltrace.cycles-pp.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
2.05 ? 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt
0.60 ? 6% perf-profile.calltrace.cycles-pp.ip_make_skb.udp_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
1.73 ? 14% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.72 ? 14% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
1.84 ? 13% perf-profile.calltrace.cycles-pp.__open64_nocancel
1.82 ? 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel
1.82 ? 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
1.81 ? 13% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
1.81 ? 13% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
1.01 ? 17% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_on_stack
1.22 ? 15% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu
1.18 ? 16% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_on_stack.do_softirq_own_stack
0.84 ? 20% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__socket
0.84 ? 20% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__socket
0.84 ? 20% perf-profile.calltrace.cycles-pp.__socket
0.84 ? 20% perf-profile.calltrace.cycles-pp.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe.__socket
0.83 ? 19% perf-profile.calltrace.cycles-pp.__sys_socket.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe.__socket
1.23 ? 18% perf-profile.calltrace.cycles-pp.__alloc_file.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2
1.23 ? 17% perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2.do_sys_open
1.11 ? 18% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open
3.69 ? 10% perf-profile.calltrace.cycles-pp.kmem_cache_free.skb_free_datagram.unix_dgram_recvmsg.sock_read_iter.new_sync_read
5.25 ? 9% perf-profile.calltrace.cycles-pp.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
4.63 ? 10% perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
4.62 ? 10% perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_dgram_sendmsg
2.45 ? 16% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
10.75 ? 12% perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
11.30 ? 12% perf-profile.calltrace.cycles-pp.unix_dgram_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
10.49 ? 12% perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
10.25 ? 12% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
10.45 ? 12% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
10.94 ? 12% perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_dgram_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
10.90 ? 12% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_dgram_sendmsg.sock_sendmsg.sock_write_iter
10.90 ? 12% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_dgram_sendmsg.sock_sendmsg
27.36 ? 8% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.61 ? 9% perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
26.52 ? 9% perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
2.25 ? 10% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.50 ? 13% perf-profile.children.cycles-pp.tick_sched_timer
1.23 ? 11% perf-profile.children.cycles-pp.tick_sched_handle
1.22 ? 10% perf-profile.children.cycles-pp.update_process_times
0.10 ? 24% perf-profile.children.cycles-pp.rcu_sched_clock_irq
1.03 ? 13% perf-profile.children.cycles-pp.mul_short
0.75 ? 9% perf-profile.children.cycles-pp.scheduler_tick
0.84 ? 16% perf-profile.children.cycles-pp.mul_long
0.83 ? 13% perf-profile.children.cycles-pp.mul_int
0.63 ? 20% perf-profile.children.cycles-pp.ret_from_fork
0.63 ? 20% perf-profile.children.cycles-pp.kthread
0.79 ? 6% perf-profile.children.cycles-pp._raw_spin_lock
0.62 ? 15% perf-profile.children.cycles-pp.random_r
0.53 ? 12% perf-profile.children.cycles-pp.console_unlock
0.58 ? 8% perf-profile.children.cycles-pp.vprintk_emit
0.85 ? 10% perf-profile.children.cycles-pp.security_file_permission
0.74 ? 8% perf-profile.children.cycles-pp.common_file_perm
0.46 ? 6% perf-profile.children.cycles-pp.memcpy_erms
0.45 ? 14% perf-profile.children.cycles-pp.serial8250_console_write
0.50 ? 8% perf-profile.children.cycles-pp.worker_thread
0.43 ? 14% perf-profile.children.cycles-pp.uart_console_write
0.45 ? 7% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
0.48 ? 7% perf-profile.children.cycles-pp.process_one_work
0.42 ? 14% perf-profile.children.cycles-pp.wait_for_xmitr
0.55 ? 9% perf-profile.children.cycles-pp.semop
0.40 ? 15% perf-profile.children.cycles-pp.serial8250_console_putchar
0.39 ? 8% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.24 ? 16% perf-profile.children.cycles-pp.load_balance
0.36 ? 14% perf-profile.children.cycles-pp.io_serial_in
0.42 ? 21% perf-profile.children.cycles-pp.do_semtimedop
0.23 ? 27% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.16 ? 17% perf-profile.children.cycles-pp.update_rq_clock
0.17 ? 30% perf-profile.children.cycles-pp.run_local_timers
0.22 ? 3% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.18 ? 18% perf-profile.children.cycles-pp.find_busiest_group
0.15 ? 16% perf-profile.children.cycles-pp.rcu_idle_exit
0.49 ? 4% perf-profile.children.cycles-pp.asm_exc_page_fault
0.18 ? 16% perf-profile.children.cycles-pp.update_sd_lb_stats
0.23 ? 17% perf-profile.children.cycles-pp.srandom_r
0.47 ? 5% perf-profile.children.cycles-pp.exc_page_fault
0.22 ? 15% perf-profile.children.cycles-pp.__skb_datagram_iter
0.47 ? 4% perf-profile.children.cycles-pp.do_user_addr_fault
0.39 ? 7% perf-profile.children.cycles-pp.__handle_mm_fault
0.42 ? 6% perf-profile.children.cycles-pp.handle_mm_fault
0.08 ? 13% perf-profile.children.cycles-pp.update_irq_load_avg
0.12 ? 13% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.09 ? 21% perf-profile.children.cycles-pp.rcu_eqs_exit
0.24 ? 14% perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.05 ? 61% perf-profile.children.cycles-pp.delay_tsc
0.12 ? 10% perf-profile.children.cycles-pp.dput
0.17 ? 13% perf-profile.children.cycles-pp.newton_raphson
0.16 ? 13% perf-profile.children.cycles-pp._copy_to_iter
0.16 ? 16% perf-profile.children.cycles-pp.copyout
0.23 ? 12% perf-profile.children.cycles-pp.pipe_write
0.12 ? 15% perf-profile.children.cycles-pp.__libc_fork
0.16 ? 9% perf-profile.children.cycles-pp.security_file_alloc
0.07 ? 24% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.15 ? 9% perf-profile.children.cycles-pp.apparmor_file_alloc_security
0.10 ? 8% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.09 ? 14% perf-profile.children.cycles-pp.skb_queue_tail
0.17 ? 10% perf-profile.children.cycles-pp.irq_work_run_list
0.11 ? 16% perf-profile.children.cycles-pp.__do_sys_clone
0.11 ? 16% perf-profile.children.cycles-pp._do_fork
0.08 ? 19% perf-profile.children.cycles-pp.vt_console_print
0.03 ?102% perf-profile.children.cycles-pp.__check_heap_object
0.19 ? 2% perf-profile.children.cycles-pp.do_fault
0.10 ? 24% perf-profile.children.cycles-pp.copy_user_generic_unrolled
0.07 ? 22% perf-profile.children.cycles-pp.fbcon_putcs
0.07 ? 22% perf-profile.children.cycles-pp.bit_putcs
0.13 ? 6% perf-profile.children.cycles-pp.filemap_map_pages
0.18 ? 14% perf-profile.children.cycles-pp.netif_rx
0.06 ? 20% perf-profile.children.cycles-pp.sys_imageblit
0.06 ? 20% perf-profile.children.cycles-pp.drm_fb_helper_sys_imageblit
0.07 ? 11% perf-profile.children.cycles-pp.unix_stream_read_actor
0.09 ? 20% perf-profile.children.cycles-pp.unmap_vmas
0.07 ? 20% perf-profile.children.cycles-pp.__fget_light
0.09 ? 15% perf-profile.children.cycles-pp.copy_process
0.07 ? 22% perf-profile.children.cycles-pp._copy_from_iter
0.18 ? 14% perf-profile.children.cycles-pp.netif_rx_internal
0.10 ? 14% perf-profile.children.cycles-pp.do_wp_page
0.09 ? 20% perf-profile.children.cycles-pp.unmap_page_range
0.07 ? 24% perf-profile.children.cycles-pp.unix_peer_get
0.09 ? 16% perf-profile.children.cycles-pp.zap_pte_range
0.04 ? 58% perf-profile.children.cycles-pp.num_rtns_1
0.06 ? 20% perf-profile.children.cycles-pp.matrix_rtns
0.09 ? 12% perf-profile.children.cycles-pp.wp_page_copy
0.23 ? 5% perf-profile.children.cycles-pp.release_sock
0.07 ? 7% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.06 ? 20% perf-profile.children.cycles-pp.__wake_up_common_lock
0.07 ? 6% perf-profile.children.cycles-pp.__inet_lookup_established
0.09 ? 4% perf-profile.children.cycles-pp.tcp_connect
0.13 ? 7% perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_skb
0.09 ? 7% perf-profile.children.cycles-pp.security_sock_rcv_skb
0.06 perf-profile.children.cycles-pp.__tcp_select_window
0.07 ? 20% perf-profile.children.cycles-pp.__put_anon_vma
0.09 ? 18% perf-profile.children.cycles-pp.alloc_pipe_info
0.10 ? 23% perf-profile.children.cycles-pp.rwsem_spin_on_owner
0.12 ? 13% perf-profile.children.cycles-pp.accept
0.16 ? 2% perf-profile.children.cycles-pp.newseg
0.16 ? 4% perf-profile.children.cycles-pp.validate_xmit_skb
0.07 ? 6% perf-profile.children.cycles-pp.skb_clone_tx_timestamp
0.19 ? 21% perf-profile.children.cycles-pp.shmat
0.14 ? 3% perf-profile.children.cycles-pp.__x64_sys_accept
0.14 ? 3% perf-profile.children.cycles-pp.__sys_accept4
0.14 ? 3% perf-profile.children.cycles-pp.__sys_accept4_file
0.14 ? 3% perf-profile.children.cycles-pp.__shmem_file_setup
0.23 ? 10% perf-profile.children.cycles-pp.unix_create
0.23 ? 9% perf-profile.children.cycles-pp.inet_create
0.21 ? 20% perf-profile.children.cycles-pp.unlink_anon_vmas
0.24 ? 20% perf-profile.children.cycles-pp.free_pgtables
0.27 ? 11% perf-profile.children.cycles-pp.unix_create1
0.10 ? 7% perf-profile.children.cycles-pp.netif_skb_features
0.22 ? 24% perf-profile.children.cycles-pp.shmget
0.25 ? 17% perf-profile.children.cycles-pp.__x64_sys_pipe
0.25 ? 17% perf-profile.children.cycles-pp.do_pipe2
0.25 ? 18% perf-profile.children.cycles-pp.__do_pipe_flags
0.26 ? 12% perf-profile.children.cycles-pp.ipcget
0.42 ? 10% perf-profile.children.cycles-pp.__ip_append_data
0.24 ? 12% perf-profile.children.cycles-pp.__x64_sys_shmget
0.25 ? 18% perf-profile.children.cycles-pp.create_pipe_files
0.26 ? 8% perf-profile.children.cycles-pp.remove_vma
0.67 ? 9% perf-profile.children.cycles-pp.ip_make_skb
0.18 ? 31% perf-profile.children.cycles-pp.osq_lock
0.28 ? 28% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
0.28 ? 28% perf-profile.children.cycles-pp.rwsem_optimistic_spin
0.47 ? 11% perf-profile.children.cycles-pp.sk_alloc
0.46 ? 11% perf-profile.children.cycles-pp.sk_prot_alloc
0.49 ? 7% perf-profile.children.cycles-pp.sock_alloc
0.47 ? 7% perf-profile.children.cycles-pp.sock_alloc_inode
0.42 ? 11% perf-profile.children.cycles-pp.d_alloc_pseudo
0.57 ? 6% perf-profile.children.cycles-pp.alloc_file
0.45 ? 9% perf-profile.children.cycles-pp.__d_alloc
0.59 ? 7% perf-profile.children.cycles-pp.new_inode_pseudo
0.58 ? 7% perf-profile.children.cycles-pp.alloc_inode
0.75 ? 7% perf-profile.children.cycles-pp.__sys_socketpair
0.76 ? 8% perf-profile.children.cycles-pp.__x64_sys_socketpair
0.99 ? 7% perf-profile.children.cycles-pp.__sock_create
0.84 ? 20% perf-profile.children.cycles-pp.__socket
1.48 ? 6% perf-profile.children.cycles-pp.rcu_core
0.75 ? 8% perf-profile.children.cycles-pp.sock_alloc_file
0.93 ? 8% perf-profile.children.cycles-pp.__x64_sys_socket
0.92 ? 8% perf-profile.children.cycles-pp.__sys_socket
1.43 ? 5% perf-profile.children.cycles-pp.rcu_do_batch
0.92 ? 7% perf-profile.children.cycles-pp.alloc_file_pseudo
1.85 ? 13% perf-profile.children.cycles-pp.__open64_nocancel
1.96 ? 3% perf-profile.children.cycles-pp.do_filp_open
1.96 ? 3% perf-profile.children.cycles-pp.path_openat
2.06 ? 2% perf-profile.children.cycles-pp.do_sys_open
2.06 ? 2% perf-profile.children.cycles-pp.do_sys_openat2
1.93 ? 7% perf-profile.children.cycles-pp.__alloc_file
1.94 ? 7% perf-profile.children.cycles-pp.alloc_empty_file
3.31 ? 8% perf-profile.children.cycles-pp.kmem_cache_alloc
9.93 ? 7% perf-profile.children.cycles-pp.kmem_cache_free
10.75 ? 12% perf-profile.children.cycles-pp.unix_stream_sendmsg
11.30 ? 12% perf-profile.children.cycles-pp.unix_dgram_sendmsg
9.37 ? 10% perf-profile.children.cycles-pp.kmem_cache_alloc_node
5.21 ? 7% perf-profile.children.cycles-pp.get_obj_cgroup_from_current
27.93 ? 8% perf-profile.children.cycles-pp.ksys_write
27.85 ? 8% perf-profile.children.cycles-pp.vfs_write
27.36 ? 8% perf-profile.children.cycles-pp.new_sync_write
26.62 ? 9% perf-profile.children.cycles-pp.sock_write_iter
26.52 ? 9% perf-profile.children.cycles-pp.sock_sendmsg
21.48 ? 12% perf-profile.children.cycles-pp.__alloc_skb
21.53 ? 12% perf-profile.children.cycles-pp.alloc_skb_with_frags
21.68 ? 12% perf-profile.children.cycles-pp.sock_alloc_send_pskb
58.11 ? 7% perf-profile.children.cycles-pp.do_syscall_64
60.07 ? 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
1.02 ? 14% perf-profile.self.cycles-pp.mul_short
0.84 ? 15% perf-profile.self.cycles-pp.mul_long
0.83 ? 13% perf-profile.self.cycles-pp.mul_int
0.78 ? 7% perf-profile.self.cycles-pp._raw_spin_lock
0.55 ? 14% perf-profile.self.cycles-pp.random_r
0.69 ? 8% perf-profile.self.cycles-pp.common_file_perm
0.45 ? 6% perf-profile.self.cycles-pp.memcpy_erms
0.38 ? 9% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.36 ? 14% perf-profile.self.cycles-pp.io_serial_in
0.09 ? 9% perf-profile.self.cycles-pp.sock_write_iter
0.05 perf-profile.self.cycles-pp.udp_sendmsg
0.23 ? 27% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.24 ? 17% perf-profile.self.cycles-pp.aa_sk_perm
0.17 ? 29% perf-profile.self.cycles-pp.run_local_timers
0.08 ? 10% perf-profile.self.cycles-pp.new_sync_write
0.22 ? 16% perf-profile.self.cycles-pp.srandom_r
0.08 ? 13% perf-profile.self.cycles-pp.update_irq_load_avg
0.13 ? 22% perf-profile.self.cycles-pp.update_sd_lb_stats
0.12 ? 13% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.06 ? 15% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.05 ? 61% perf-profile.self.cycles-pp.delay_tsc
0.17 ? 11% perf-profile.self.cycles-pp.newton_raphson
0.06 ? 6% perf-profile.self.cycles-pp.rebalance_domains
0.09 ? 15% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.08 ? 16% perf-profile.self.cycles-pp.vfs_write
0.10 ? 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.03 ?102% perf-profile.self.cycles-pp.__check_heap_object
0.09 ? 20% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.06 ? 20% perf-profile.self.cycles-pp.sys_imageblit
0.12 ? 13% perf-profile.self.cycles-pp.skb_release_data
0.05 ? 58% perf-profile.self.cycles-pp.net_rx_action
0.04 ? 58% perf-profile.self.cycles-pp.do_syscall_64
0.08 ? 8% perf-profile.self.cycles-pp.filemap_map_pages
0.12 ? 6% perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_skb
0.06 ? 15% perf-profile.self.cycles-pp.__ip_finish_output
0.06 ? 7% perf-profile.self.cycles-pp.__tcp_select_window
0.07 ? 5% perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.10 ? 23% perf-profile.self.cycles-pp.rwsem_spin_on_owner
0.11 ? 11% perf-profile.self.cycles-pp.unix_dgram_recvmsg
0.07 ? 6% perf-profile.self.cycles-pp.skb_clone_tx_timestamp
0.08 ? 6% perf-profile.self.cycles-pp.__ip_append_data
0.08 ? 11% perf-profile.self.cycles-pp.netif_skb_features
0.17 ? 31% perf-profile.self.cycles-pp.osq_lock
0.42 ? 27% perf-profile.self.cycles-pp.page_counter_uncharge
0.30 ? 17% perf-profile.self.cycles-pp.irqtime_account_irq
1.73 ? 8% perf-profile.self.cycles-pp.kmem_cache_alloc
4.04 ? 12% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
4.55 ? 8% perf-profile.self.cycles-pp.kmem_cache_free
4.44 ? 8% perf-profile.self.cycles-pp.kfree
5.14 ? 7% perf-profile.self.cycles-pp.get_obj_cgroup_from_current

reaim.jobs_per_min

-------------------------------------------------+
:: |
: : |
+ + : + + |
+ :+ + + : : :: .+ |
: : + +. : : : +.+.+ : |
+ : : : + : : : : :|
: : .+ : + : : + : + : : ::|
: + + : +.+ :+ : : : : + |
O + :: : |
+ O O O O + + |
O O O O O O |
O |
O |
-------------------------------------------------+

************************************************************************
threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
==============================================================
config/rootfs/runtime/tbox_group/test/testcase/ucode:
rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap4/lru-file-mmap-read-rand/vm-scalability/0x5003003
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
%stddev
\
0.10 vm-scalability.free_time
8645 vm-scalability.median
-2.8% 1671023 vm-scalability.throughput
+2.2% 382.04 vm-scalability.time.elapsed_time
+2.2% 382.04 vm-scalability.time.elapsed_time.max
-2.8% 48740460 vm-scalability.time.major_page_faults
-20.4% 2613088 ? 20% vm-scalability.time.maximum_resident_set_size
-7.4% 836535 vm-scalability.time.minor_page_faults
58756 vm-scalability.time.system_time
30.74 ? 2% vm-scalability.time.user_time
569304 ? 6% meminfo.Active(anon)
0.24 ? 4% mpstat.cpu.all.soft%
-11.5% 67201649 ? 3% numa-numastat.node2.numa_foreign
8133 ? 3% numa-meminfo.node0.KernelStack
423920 ? 24% numa-meminfo.node3.Shmem
7248 ? 9% slabinfo.eventpoll_pwq.active_objs
7248 ? 9% slabinfo.eventpoll_pwq.num_objs
311459 ?157% cpuidle.C1.usage
477088 ?140% cpuidle.POLL.time
53583 ? 39% cpuidle.POLL.usage
8126 ? 2% numa-vmstat.node0.nr_kernel_stack
106647 ? 24% numa-vmstat.node3.nr_shmem
-7.6% 3014749 ? 6% numa-vmstat.node3.workingset_nodereclaim
17.00 vmstat.cpu.id
82.00 vmstat.cpu.sy
847.25 ? 7% vmstat.memory.buff
5684 vmstat.system.cs
15343 ? 2% softirqs.CPU157.RCU
14422 ? 5% softirqs.CPU16.RCU
14826 ? 2% softirqs.CPU165.RCU
13273 softirqs.CPU38.RCU
11140 ? 3% softirqs.CPU8.SCHED
136.00 ? 11% interrupts.CPU120.RES:Rescheduling_interrupts
380.75 ? 10% interrupts.CPU13.RES:Rescheduling_interrupts
72.50 ? 27% interrupts.CPU139.RES:Rescheduling_interrupts
82.00 ? 20% interrupts.CPU173.RES:Rescheduling_interrupts
91.00 ? 11% interrupts.CPU179.RES:Rescheduling_interrupts
89.00 ? 20% interrupts.CPU185.RES:Rescheduling_interrupts
98.00 ? 10% interrupts.CPU189.RES:Rescheduling_interrupts
250.25 ? 58% interrupts.CPU191.RES:Rescheduling_interrupts
440.75 ? 7% interrupts.CPU2.RES:Rescheduling_interrupts
190.50 interrupts.CPU64.RES:Rescheduling_interrupts
190.25 ? 3% interrupts.CPU66.RES:Rescheduling_interrupts
-9.1% 320400 interrupts.CPU67.CAL:Function_call_interrupts
173.75 ? 5% interrupts.CPU67.RES:Rescheduling_interrupts
170.00 ? 9% interrupts.CPU68.RES:Rescheduling_interrupts
-9.1% 318025 ? 2% interrupts.CPU70.CAL:Function_call_interrupts
5255 ? 33% interrupts.CPU71.NMI:Non-maskable_interrupts
5255 ? 33% interrupts.CPU71.PMI:Performance_monitoring_interrupts
100.25 ? 13% interrupts.CPU87.RES:Rescheduling_interrupts
1.64 ? 90% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.do_task_dead.do_exit.do_group_exit
10.18 ?115% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.do_wait.kernel_wait4.__do_sys_wait4
0.10 ? 37% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.balance_pgdat
0.64 ? 97% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.do_user_addr_fault
73.50 ? 98% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.kswapd.kthread.ret_from_fork
3.26 ? 96% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.do_user_addr_fault
0.16 ?144% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.rmap_walk_file
3.78 ? 9% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.__alloc_pages_nodemask
45.93 ?173% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.process_one_work
10513 ? 63% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.do_wait.kernel_wait4.__do_sys_wait4
190.92 ? 8% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.__alloc_pages_slowpath
1622 ? 95% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.shrink_lruvec
3.78 ? 9% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.__alloc_pages_nodemask
17.78 ? 96% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.__flush_work
3.10 ? 21% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.dput
22.28 ?139% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.generic_perform_write
87.87 ? 64% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.process_one_work
6.89 ? 39% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.rmap_walk_file
190.92 ? 8% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.__alloc_pages_slowpath
760.99 ?163% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.generic_perform_write
90.14 ? 70% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.rmap_walk_file
1124 ? 64% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.preempt_schedule_common._cond_resched.shrink_lruvec
169.39 ? 23% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.schedule_timeout.kswapd.kthread
1472 ? 35% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.worker_thread.kthread.ret_from_fork
-2.3% 188669 proc-vmstat.allocstall_movable
934.75 ? 2% proc-vmstat.kswapd_low_wmark_hit_quickly
142606 ? 6% proc-vmstat.nr_active_anon
142609 ? 6% proc-vmstat.nr_zone_active_anon
-3.5% 2.814e+08 proc-vmstat.numa_foreign
-2.6% 1.262e+09 proc-vmstat.numa_hit
-2.6% 1.261e+09 proc-vmstat.numa_local
-3.5% 2.814e+08 proc-vmstat.numa_miss
-3.5% 2.816e+08 proc-vmstat.numa_other
3436 ? 2% proc-vmstat.pageoutrun
-2.7% 1.534e+09 proc-vmstat.pgalloc_normal
-2.4% 717357 ? 2% proc-vmstat.pgdeactivate
1e+08 proc-vmstat.pgfault
-2.7% 1.549e+09 proc-vmstat.pgfree
-2.8% 48694010 proc-vmstat.pgmajfault
-2.4% 717357 ? 2% proc-vmstat.pgrefill
-2.3% 2.079e+09 proc-vmstat.pgscan_direct
-2.5% 2.221e+09 proc-vmstat.pgscan_file
-6.1% 1.425e+08 ? 2% proc-vmstat.pgscan_kswapd
-2.6% 1.402e+09 proc-vmstat.pgsteal_direct
-2.8% 1.49e+09 proc-vmstat.pgsteal_file
-6.8% 88192320 ? 2% proc-vmstat.pgsteal_kswapd
-8.4% 74310955 ? 2% proc-vmstat.slabs_scanned
-3.8% 17213022 proc-vmstat.workingset_nodereclaim
-1.4% 4747661 proc-vmstat.workingset_nodes
-3.3% 1.465e+08 proc-vmstat.workingset_refault_file
5.37 ? 20% perf-stat.i.MPKI
-1.0% 1.315e+10 perf-stat.i.branch-instructions
+3.6% 1.124e+08 perf-stat.i.cache-misses
+3.2% 2.387e+08 ? 3% perf-stat.i.cache-references
5585 perf-stat.i.context-switches
7.29 perf-stat.i.cpi
+1.0% 4.781e+11 perf-stat.i.cpu-cycles
3918 perf-stat.i.cycles-between-cache-misses
-1.4% 1.495e+10 perf-stat.i.dTLB-loads
-4.3% 3.878e+09 perf-stat.i.dTLB-stores
-5.5% 6765423 ? 3% perf-stat.i.iTLB-load-misses
-1.4% 5.875e+10 perf-stat.i.instructions
8407 ? 2% perf-stat.i.instructions-per-iTLB-miss
0.17 perf-stat.i.ipc
-4.8% 124567 perf-stat.i.major-faults
-1.6% 167.78 perf-stat.i.metric.M/sec
79.15 perf-stat.i.node-load-miss-rate%
+7.7% 25800856 perf-stat.i.node-load-misses
-4.0% 11162780 perf-stat.i.node-stores
-4.7% 131161 perf-stat.i.page-faults
4.04 ? 2% perf-stat.overall.MPKI
8.15 perf-stat.overall.cpi
4289 perf-stat.overall.cycles-between-cache-misses
8824 ? 3% perf-stat.overall.instructions-per-iTLB-miss
0.12 perf-stat.overall.ipc
80.11 perf-stat.overall.node-load-miss-rate%
60.79 perf-stat.overall.node-store-miss-rate%
-1.2% 1.343e+10 perf-stat.ps.branch-instructions
+3.6% 1.141e+08 perf-stat.ps.cache-misses
+2.9% 2.422e+08 ? 2% perf-stat.ps.cache-references
5648 perf-stat.ps.context-switches
-1.6% 1.527e+10 perf-stat.ps.dTLB-loads
-4.3% 3.948e+09 perf-stat.ps.dTLB-stores
-5.5% 6810312 ? 3% perf-stat.ps.iTLB-load-misses
-9.6% 814873 ? 9% perf-stat.ps.iTLB-loads
-1.5% 6.002e+10 perf-stat.ps.instructions
-4.8% 127225 perf-stat.ps.major-faults
+7.5% 26227147 perf-stat.ps.node-load-misses
-3.9% 11392306 perf-stat.ps.node-stores
-4.7% 133788 perf-stat.ps.page-faults
3.68 ? 15% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded.filemap_fault.__xfs_filemap_fault
3.56 ? 16% perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded.filemap_fault
3.42 ? 17% perf-profile.calltrace.cycles-pp.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded
3.04 ? 19% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
3.06 ? 19% perf-profile.calltrace.cycles-pp._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
4.09 ? 4% perf-profile.calltrace.cycles-pp.iomap_readahead_actor.iomap_apply.iomap_readahead.read_pages.page_cache_readahead_unbounded
3.94 ? 4% perf-profile.calltrace.cycles-pp.iomap_readpage_actor.iomap_readahead_actor.iomap_apply.iomap_readahead.read_pages
4.12 ? 4% perf-profile.calltrace.cycles-pp.read_pages.page_cache_readahead_unbounded.filemap_fault.__xfs_filemap_fault.__do_fault
4.11 ? 4% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readahead.read_pages.page_cache_readahead_unbounded.filemap_fault
4.12 ? 4% perf-profile.calltrace.cycles-pp.iomap_readahead.read_pages.page_cache_readahead_unbounded.filemap_fault.__xfs_filemap_fault
2.12 ? 4% perf-profile.calltrace.cycles-pp.memset_erms.iomap_readpage_actor.iomap_readahead_actor.iomap_apply.iomap_readahead
1.68 ? 3% perf-profile.calltrace.cycles-pp.iomap_set_range_uptodate.iomap_readpage_actor.iomap_readahead_actor.iomap_apply.iomap_readahead
4.19 ? 13% perf-profile.children.cycles-pp._raw_spin_lock
3.89 ? 14% perf-profile.children.cycles-pp.get_page_from_freelist
3.75 ? 15% perf-profile.children.cycles-pp.rmqueue
3.59 ? 16% perf-profile.children.cycles-pp.rmqueue_bulk
4.11 ? 4% perf-profile.children.cycles-pp.iomap_apply
4.09 ? 4% perf-profile.children.cycles-pp.iomap_readahead_actor
4.12 ? 4% perf-profile.children.cycles-pp.read_pages
4.12 ? 4% perf-profile.children.cycles-pp.iomap_readahead
3.95 ? 4% perf-profile.children.cycles-pp.iomap_readpage_actor
2.15 ? 4% perf-profile.children.cycles-pp.memset_erms
0.82 ? 2% perf-profile.children.cycles-pp.free_unref_page_list
1.68 ? 3% perf-profile.children.cycles-pp.iomap_set_range_uptodate
0.95 ? 2% perf-profile.children.cycles-pp.__list_del_entry_valid
0.69 ? 2% perf-profile.children.cycles-pp.isolate_lru_pages
0.56 perf-profile.children.cycles-pp.page_referenced
0.15 ? 2% perf-profile.children.cycles-pp.down_read
0.15 perf-profile.children.cycles-pp.xa_load
0.11 perf-profile.children.cycles-pp.page_remove_rmap
0.21 perf-profile.children.cycles-pp.unaccount_page_cache_page
0.06 perf-profile.children.cycles-pp.lru_note_cost
0.09 ? 7% perf-profile.children.cycles-pp.propagate_protected_usage
0.24 ? 4% perf-profile.children.cycles-pp.page_counter_try_charge
0.34 ? 3% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.35 ? 8% perf-profile.children.cycles-pp.page_counter_cancel
0.30 ? 2% perf-profile.children.cycles-pp.uncharge_batch
0.37 ? 8% perf-profile.children.cycles-pp.page_counter_uncharge
0.34 ? 4% perf-profile.children.cycles-pp.try_charge
0.77 ? 4% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.36 ? 2% perf-profile.children.cycles-pp.workingset_age_nonresident
0.09 ? 5% perf-profile.self.cycles-pp.workingset_eviction
2.14 ? 4% perf-profile.self.cycles-pp.memset_erms
1.66 ? 3% perf-profile.self.cycles-pp.iomap_set_range_uptodate
0.95 ? 2% perf-profile.self.cycles-pp.__list_del_entry_valid
0.39 ? 2% perf-profile.self.cycles-pp.xas_create
0.31 ? 3% perf-profile.self.cycles-pp.free_pcppages_bulk
0.12 perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.10 perf-profile.self.cycles-pp.try_charge
0.15 ? 19% perf-profile.self.cycles-pp.kmem_cache_free
0.06 perf-profile.self.cycles-pp.lru_note_cost
0.09 ? 4% perf-profile.self.cycles-pp.propagate_protected_usage
0.29 ? 8% perf-profile.self.cycles-pp.page_counter_cancel
0.21 ? 3% perf-profile.self.cycles-pp.page_counter_try_charge
0.33 ? 5% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.36 ? 2% perf-profile.self.cycles-pp.workingset_age_nonresident
************************************************************************
threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
==============================================================
config/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-hsw-4ex1/page_fault2/will-it-scale/0x16
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
fail:runs
|
4:4 perf-profile.calltrace.cycles-pp.error_entry.testcase
4:4 perf-profile.children.cycles-pp.error_entry
3:4 perf-profile.self.cycles-pp.error_entry
%stddev
\
+14.5% 13642560 will-it-scale.72.processes
189479 will-it-scale.per_process_ops
+14.5% 13642560 will-it-scale.workload
+68.4% 9.683e+09 ? 16% cpuidle.C1E.time
4.64 mpstat.cpu.all.usr%
3801 ? 11% syscalls.sys_write.med
58685 ? 3% slabinfo.filp.active_objs
58899 ? 3% slabinfo.filp.num_objs
11745 ? 6% slabinfo.pde_opener.active_objs
11745 ? 6% slabinfo.pde_opener.num_objs
26912 ? 9% numa-meminfo.node1.KReclaimable
5392 ? 5% numa-meminfo.node1.KernelStack
26912 ? 9% numa-meminfo.node1.SReclaimable
47269 ? 14% numa-meminfo.node1.SUnreclaim
74181 ? 10% numa-meminfo.node1.Slab
51174 ? 10% numa-meminfo.node3.SUnreclaim
86149 ? 12% numa-meminfo.node3.Slab
+14.4% 1.042e+09 numa-numastat.node0.local_node
+14.4% 1.042e+09 numa-numastat.node0.numa_hit
+14.0% 1.027e+09 numa-numastat.node1.local_node
+14.0% 1.027e+09 numa-numastat.node1.numa_hit
+14.6% 1.024e+09 numa-numastat.node2.local_node
+14.6% 1.024e+09 numa-numastat.node2.numa_hit
+14.9% 1.019e+09 numa-numastat.node3.local_node
+14.9% 1.019e+09 numa-numastat.node3.numa_hit
14317 ?102% numa-numastat.node3.other_node
-1.0% 1196811 proc-vmstat.nr_anon_pages
32990 proc-vmstat.nr_slab_reclaimable
54099 proc-vmstat.nr_slab_unreclaimable
50.50 ? 29% proc-vmstat.numa_hint_faults_local
+14.5% 4.11e+09 proc-vmstat.numa_hit
+14.5% 4.11e+09 proc-vmstat.numa_local
6395 ? 48% proc-vmstat.numa_pages_migrated
22714 ? 2% proc-vmstat.numa_pte_updates
+14.4% 4.112e+09 proc-vmstat.pgalloc_normal
+14.5% 4.1e+09 proc-vmstat.pgfault
+14.5% 4.109e+09 proc-vmstat.pgfree
6395 ? 48% proc-vmstat.pgmigrate_success
+14.4% 5.191e+08 numa-vmstat.node0.numa_hit
+14.4% 5.19e+08 numa-vmstat.node0.numa_local
5394 ? 5% numa-vmstat.node1.nr_kernel_stack
6727 ? 9% numa-vmstat.node1.nr_slab_reclaimable
11817 ? 14% numa-vmstat.node1.nr_slab_unreclaimable
+13.8% 5.108e+08 numa-vmstat.node1.numa_hit
+13.8% 5.107e+08 numa-vmstat.node1.numa_local
+14.4% 5.092e+08 numa-vmstat.node2.numa_hit
+14.4% 5.091e+08 numa-vmstat.node2.numa_local
12793 ? 10% numa-vmstat.node3.nr_slab_unreclaimable
+14.8% 5.071e+08 numa-vmstat.node3.numa_hit
+14.8% 5.07e+08 numa-vmstat.node3.numa_local
103615 ? 14% numa-vmstat.node3.numa_other
5937 ? 47% sched_debug.cfs_rq:/.exec_clock.min
34362 ? 9% sched_debug.cfs_rq:/.exec_clock.stddev
454176 ? 46% sched_debug.cfs_rq:/.min_vruntime.min
-27.2% 2548272 ? 9% sched_debug.cfs_rq:/.min_vruntime.stddev
17.19 ? 2% sched_debug.cfs_rq:/.nr_spread_over.avg
45.96 ? 16% sched_debug.cfs_rq:/.nr_spread_over.max
-27.2% 2548274 ? 9% sched_debug.cfs_rq:/.spread0.stddev
-15.7% 1117832 ? 5% sched_debug.cpu.avg_idle.max
29035 ? 26% sched_debug.cpu.nr_switches.max
3338 ? 11% sched_debug.cpu.nr_switches.stddev
26560 ? 28% sched_debug.cpu.sched_count.max
694.00 ? 3% sched_debug.cpu.sched_count.min
13114 ? 28% sched_debug.cpu.sched_goidle.max
28.46 ? 27% sched_debug.cpu.sched_goidle.min
1522 ? 14% sched_debug.cpu.sched_goidle.stddev
13967 ? 17% sched_debug.cpu.ttwu_count.max
271.50 ? 5% sched_debug.cpu.ttwu_count.min
1578 ? 9% sched_debug.cpu.ttwu_count.stddev
239.38 ? 4% sched_debug.cpu.ttwu_local.min
27986 ? 26% softirqs.CPU1.SCHED
7646 ? 4% softirqs.CPU100.RCU
25311 ? 18% softirqs.CPU103.SCHED
17571 ? 15% softirqs.CPU11.SCHED
19452 ? 14% softirqs.CPU121.SCHED
7842 ? 7% softirqs.CPU137.RCU
8527 ? 6% softirqs.CPU14.RCU
7848 ? 15% softirqs.CPU18.RCU
8707 ? 6% softirqs.CPU19.RCU
8128 ? 2% softirqs.CPU20.RCU
7871 ? 3% softirqs.CPU22.RCU
25339 ? 19% softirqs.CPU22.SCHED
24880 ? 29% softirqs.CPU24.SCHED
7858 ? 6% softirqs.CPU25.RCU
8416 ? 5% softirqs.CPU26.RCU
8114 ? 4% softirqs.CPU27.RCU
8326 ? 8% softirqs.CPU3.RCU
8484 ? 7% softirqs.CPU30.RCU
8370 ? 2% softirqs.CPU36.RCU
8209 ? 3% softirqs.CPU37.RCU
8513 ? 5% softirqs.CPU38.RCU
8100 ? 5% softirqs.CPU4.RCU
8026 ? 6% softirqs.CPU40.RCU
8525 ? 5% softirqs.CPU43.RCU
7996 ? 2% softirqs.CPU47.RCU
23154 ? 11% softirqs.CPU49.SCHED
8204 ? 5% softirqs.CPU73.RCU
15044 ? 43% softirqs.CPU73.SCHED
16051 ? 18% softirqs.CPU94.SCHED
17981 ? 43% softirqs.CPU96.SCHED
-9.9% 1156401 softirqs.RCU
112.25 ? 49% interrupts.CPU1.RES:Rescheduling_interrupts
7922 ? 2% interrupts.CPU12.NMI:Non-maskable_interrupts
7922 ? 2% interrupts.CPU12.PMI:Performance_monitoring_interrupts
6241 ? 31% interrupts.CPU121.NMI:Non-maskable_interrupts
6241 ? 31% interrupts.CPU121.PMI:Performance_monitoring_interrupts
+9.0% 722.50 ? 4% interrupts.CPU123.CAL:Function_call_interrupts
39.50 ?113% interrupts.CPU128.RES:Rescheduling_interrupts
39.25 ?111% interrupts.CPU133.RES:Rescheduling_interrupts
3735 ? 42% interrupts.CPU137.NMI:Non-maskable_interrupts
3735 ? 42% interrupts.CPU137.PMI:Performance_monitoring_interrupts
6835 ? 18% interrupts.CPU14.NMI:Non-maskable_interrupts
6835 ? 18% interrupts.CPU14.PMI:Performance_monitoring_interrupts
4624 ? 38% interrupts.CPU15.NMI:Non-maskable_interrupts
4624 ? 38% interrupts.CPU15.PMI:Performance_monitoring_interrupts
3913 ? 34% interrupts.CPU3.NMI:Non-maskable_interrupts
3913 ? 34% interrupts.CPU3.PMI:Performance_monitoring_interrupts
5453 ? 30% interrupts.CPU37.NMI:Non-maskable_interrupts
5453 ? 30% interrupts.CPU37.PMI:Performance_monitoring_interrupts
3836 ? 7% interrupts.CPU42.NMI:Non-maskable_interrupts
3836 ? 7% interrupts.CPU42.PMI:Performance_monitoring_interrupts
3143 ? 33% interrupts.CPU50.NMI:Non-maskable_interrupts
3143 ? 33% interrupts.CPU50.PMI:Performance_monitoring_interrupts
938.75 ? 21% interrupts.CPU56.CAL:Function_call_interrupts
654.25 interrupts.CPU81.CAL:Function_call_interrupts
653.50 ? 3% interrupts.CPU82.CAL:Function_call_interrupts
2858 ? 61% interrupts.CPU84.NMI:Non-maskable_interrupts
2858 ? 61% interrupts.CPU84.PMI:Performance_monitoring_interrupts
3105 ? 17% interrupts.CPU9.NMI:Non-maskable_interrupts
3105 ? 17% interrupts.CPU9.PMI:Performance_monitoring_interrupts
6284 ? 33% interrupts.CPU92.NMI:Non-maskable_interrupts
6284 ? 33% interrupts.CPU92.PMI:Performance_monitoring_interrupts
+8.8% 720.75 ? 8% interrupts.CPU95.CAL:Function_call_interrupts
15.98 perf-stat.i.MPKI
+20.2% 1.34e+10 perf-stat.i.branch-instructions
0.49 perf-stat.i.branch-miss-rate%
+13.4% 63617962 perf-stat.i.branch-misses
37.40 perf-stat.i.cache-miss-rate%
+15.6% 4.128e+08 perf-stat.i.cache-misses
+10.1% 1.101e+09 perf-stat.i.cache-references
3.02 perf-stat.i.cpi
520.12 perf-stat.i.cycles-between-cache-misses
+18.4% 2.086e+10 perf-stat.i.dTLB-loads
+14.0% 96553382 perf-stat.i.dTLB-store-misses
+14.2% 1.239e+10 perf-stat.i.dTLB-stores
92.09 perf-stat.i.iTLB-load-miss-rate%
+14.8% 40782994 perf-stat.i.iTLB-load-misses
+18.8% 6.877e+10 perf-stat.i.instructions
1695 perf-stat.i.instructions-per-iTLB-miss
0.33 perf-stat.i.ipc
335.20 perf-stat.i.metric.M/sec
+14.5% 13580221 perf-stat.i.minor-faults
+15.4% 13682892 perf-stat.i.node-load-misses
+15.4% 3.344e+08 perf-stat.i.node-loads
18.32 perf-stat.i.node-store-miss-rate%
+28.0% 11615910 perf-stat.i.node-store-misses
+16.4% 51885017 perf-stat.i.node-stores
+14.5% 13580224 perf-stat.i.page-faults
16.00 perf-stat.overall.MPKI
0.47 perf-stat.overall.branch-miss-rate%
37.51 perf-stat.overall.cache-miss-rate%
3.02 perf-stat.overall.cpi
503.24 perf-stat.overall.cycles-between-cache-misses
92.25 perf-stat.overall.iTLB-load-miss-rate%
1686 perf-stat.overall.instructions-per-iTLB-miss
0.33 perf-stat.overall.ipc
18.29 perf-stat.overall.node-store-miss-rate%
+3.9% 1519305 perf-stat.overall.path-length
+20.1% 1.335e+10 perf-stat.ps.branch-instructions
+13.4% 63379750 perf-stat.ps.branch-misses
+15.5% 4.113e+08 perf-stat.ps.cache-misses
+10.0% 1.096e+09 perf-stat.ps.cache-references
+18.4% 2.078e+10 perf-stat.ps.dTLB-loads
+13.9% 96194702 perf-stat.ps.dTLB-store-misses
+14.1% 1.235e+10 perf-stat.ps.dTLB-stores
+14.8% 40634390 perf-stat.ps.iTLB-load-misses
+18.8% 6.852e+10 perf-stat.ps.instructions
+14.5% 13530862 perf-stat.ps.minor-faults
+15.4% 13631265 perf-stat.ps.node-load-misses
+15.4% 3.332e+08 perf-stat.ps.node-loads
+28.0% 11574692 perf-stat.ps.node-store-misses
+16.3% 51699346 perf-stat.ps.node-stores
+14.5% 13530865 perf-stat.ps.page-faults
+18.9% 2.073e+13 perf-stat.total.instructions
7.14 ? 11% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
2.81 ? 9% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_mm.mem_cgroup_charge.do_fault.__handle_mm_fault.handle_mm_fault
0.31 ?100% perf-profile.calltrace.cycles-pp.mem_cgroup_charge_statistics.mem_cgroup_charge.do_fault.__handle_mm_fault.handle_mm_fault
0.30 ?100% perf-profile.calltrace.cycles-pp.__count_memcg_events.mem_cgroup_charge_statistics.mem_cgroup_charge.do_fault.__handle_mm_fault
0.68 ? 15% perf-profile.calltrace.cycles-pp.__count_memcg_events.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.66 ? 17% perf-profile.calltrace.cycles-pp.__mod_memcg_state.__mod_memcg_lruvec_state.page_add_new_anon_rmap.alloc_set_pte.finish_fault
1.11 ? 16% perf-profile.calltrace.cycles-pp.page_add_new_anon_rmap.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
0.94 ? 17% perf-profile.calltrace.cycles-pp.__mod_memcg_lruvec_state.page_add_new_anon_rmap.alloc_set_pte.finish_fault.do_fault
0.86 ? 15% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap
0.90 ? 15% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
0.69 ? 12% perf-profile.calltrace.cycles-pp._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
0.68 ? 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
0.64 ? 14% perf-profile.calltrace.cycles-pp.__mod_memcg_lruvec_state.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
1.74 ? 13% perf-profile.calltrace.cycles-pp.try_charge.mem_cgroup_charge.do_fault.__handle_mm_fault.handle_mm_fault
1.23 ? 13% perf-profile.calltrace.cycles-pp.page_counter_try_charge.try_charge.mem_cgroup_charge.do_fault.__handle_mm_fault
3.49 ? 15% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.zap_pte_range
3.51 ? 15% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range
6.65 ? 13% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas
7.08 ? 13% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region
5.61 ? 14% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_cache_add.alloc_set_pte
5.67 ? 14% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_cache_add.alloc_set_pte.finish_fault
7.48 ? 14% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.lru_cache_add.alloc_set_pte.finish_fault.do_fault
7.62 ? 14% perf-profile.calltrace.cycles-pp.lru_cache_add.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
7.15 ? 11% perf-profile.children.cycles-pp.mem_cgroup_charge
2.82 ? 9% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
1.21 ? 16% perf-profile.children.cycles-pp.__count_memcg_events
1.63 ? 13% perf-profile.children.cycles-pp.native_irq_return_iret
1.84 ? 14% perf-profile.children.cycles-pp.__mod_memcg_state
1.11 ? 16% perf-profile.children.cycles-pp.page_add_new_anon_rmap
0.54 ? 17% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.14 ? 11% perf-profile.children.cycles-pp.lock_page_memcg
0.12 ? 11% perf-profile.children.cycles-pp.uncharge_page
0.12 ? 16% perf-profile.children.cycles-pp.memcg_check_events
0.10 ? 14% perf-profile.children.cycles-pp.uncharge_batch
0.06 ? 14% perf-profile.children.cycles-pp.page_counter_cancel
0.06 ? 16% perf-profile.children.cycles-pp.page_counter_uncharge
0.10 ? 15% perf-profile.children.cycles-pp.propagate_protected_usage
0.90 ? 15% perf-profile.children.cycles-pp.tlb_finish_mmu
1.24 ? 13% perf-profile.children.cycles-pp.page_counter_try_charge
1.75 ? 13% perf-profile.children.cycles-pp.try_charge
7.68 ? 14% perf-profile.children.cycles-pp.release_pages
7.98 ? 13% perf-profile.children.cycles-pp.tlb_flush_mmu
7.50 ? 14% perf-profile.children.cycles-pp.pagevec_lru_move_fn
7.62 ? 14% perf-profile.children.cycles-pp.lru_cache_add
9.64 ? 14% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
10.75 ? 14% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
1.95 ? 8% perf-profile.self.cycles-pp.mem_cgroup_charge
2.79 ? 9% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
1.21 ? 16% perf-profile.self.cycles-pp.__count_memcg_events
1.62 ? 13% perf-profile.self.cycles-pp.native_irq_return_iret
1.82 ? 14% perf-profile.self.cycles-pp.__mod_memcg_state
0.14 ? 15% perf-profile.self.cycles-pp.lock_page_memcg
0.11 ? 9% perf-profile.self.cycles-pp.uncharge_page
0.19 ? 12% perf-profile.self.cycles-pp.shmem_fault
0.07 ? 17% perf-profile.self.cycles-pp.memcg_check_events
0.10 ? 18% perf-profile.self.cycles-pp.propagate_protected_usage
0.51 ? 14% perf-profile.self.cycles-pp.try_charge
1.02 ? 15% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
1.15 ? 13% perf-profile.self.cycles-pp.page_counter_try_charge
10.74 ? 14% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
************************************************************************
threads Skylake with 192G memory
==============================================================
config/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/malloc1/will-it-scale/0x2006906
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
fail:runs
|
0:3 perf-profile.children.cycles-pp.error_entry
0:3 perf-profile.self.cycles-pp.error_entry
%stddev
\
-13.0% 1059732 ? 2% will-it-scale.52.processes
20379 ? 2% will-it-scale.per_process_ops
-13.0% 1059732 ? 2% will-it-scale.workload
-12.5% 1898524 ? 9% cpuidle.C1.time
-26.6% 4.848e+09 ? 86% cpuidle.C6.time
21774 ? 4% cpuidle.POLL.time
3512 ? 2% cpuidle.POLL.usage
-15.9% 3.065e+08 ? 10% numa-numastat.node0.local_node
-15.9% 3.065e+08 ? 10% numa-numastat.node0.numa_hit
6764 ?140% numa-numastat.node0.other_node
26964 ? 34% numa-numastat.node1.other_node
1241 ? 9% slabinfo.Acpi-Parse.active_objs
1241 ? 9% slabinfo.Acpi-Parse.num_objs
4330 slabinfo.skbuff_head_cache.active_objs
4330 slabinfo.skbuff_head_cache.num_objs
1208 proc-vmstat.nr_page_table_pages
2640 ?120% proc-vmstat.numa_hint_faults
-13.1% 6.371e+08 ? 2% proc-vmstat.numa_hit
-13.1% 6.371e+08 ? 2% proc-vmstat.numa_local
2518 ?120% proc-vmstat.numa_pages_migrated
11696 ? 75% proc-vmstat.numa_pte_updates
-13.1% 6.382e+08 ? 2% proc-vmstat.pgalloc_normal
-13.1% 3.196e+08 ? 2% proc-vmstat.pgfault
-13.1% 6.382e+08 ? 2% proc-vmstat.pgfree
2518 ?120% proc-vmstat.pgmigrate_success
711.00 ? 7% numa-vmstat.node0.nr_active_anon
4622 ? 12% numa-vmstat.node0.nr_mapped
725.67 ? 10% numa-vmstat.node0.nr_page_table_pages
2244 ? 40% numa-vmstat.node0.nr_shmem
12685 ? 2% numa-vmstat.node0.nr_slab_reclaimable
711.00 ? 7% numa-vmstat.node0.nr_zone_active_anon
-15.7% 1.536e+08 ? 10% numa-vmstat.node0.numa_hit
-15.6% 1.536e+08 ? 10% numa-vmstat.node0.numa_local
12124 ? 78% numa-vmstat.node0.numa_other
4145 ? 13% numa-vmstat.node1.nr_mapped
480.67 ? 14% numa-vmstat.node1.nr_page_table_pages
11176 numa-vmstat.node1.nr_slab_reclaimable
10.84 ?141% sched_debug.cfs_rq:/.MIN_vruntime.avg
975.57 ?141% sched_debug.cfs_rq:/.MIN_vruntime.max
102.26 ?141% sched_debug.cfs_rq:/.MIN_vruntime.stddev
10.84 ?141% sched_debug.cfs_rq:/.max_vruntime.avg
975.57 ?141% sched_debug.cfs_rq:/.max_vruntime.max
102.26 ?141% sched_debug.cfs_rq:/.max_vruntime.stddev
1.04 ? 19% sched_debug.cfs_rq:/.nr_spread_over.avg
5.04 ? 9% sched_debug.cfs_rq:/.removed.load_avg.avg
29.34 sched_debug.cfs_rq:/.removed.load_avg.stddev
29.80 ? 10% sched_debug.cpu.nr_uninterruptible.max
-24.67 sched_debug.cpu.nr_uninterruptible.min
9463 ? 11% sched_debug.cpu.ttwu_local.max
2946 ? 8% numa-meminfo.node0.Active
2844 ? 7% numa-meminfo.node0.Active(anon)
50742 ? 2% numa-meminfo.node0.KReclaimable
17961 ? 12% numa-meminfo.node0.Mapped
2911 ? 10% numa-meminfo.node0.PageTables
50742 ? 2% numa-meminfo.node0.SReclaimable
8978 ? 40% numa-meminfo.node0.Shmem
150653 ? 5% numa-meminfo.node0.Slab
44704 numa-meminfo.node1.KReclaimable
16451 ? 14% numa-meminfo.node1.Mapped
-9.0% 1216509 ? 6% numa-meminfo.node1.MemUsed
1922 ? 14% numa-meminfo.node1.PageTables
44704 numa-meminfo.node1.SReclaimable
3182 softirqs.CPU12.SCHED
3118 softirqs.CPU14.SCHED
31646 ? 26% softirqs.CPU15.SCHED
3133 softirqs.CPU16.SCHED
3069 ? 3% softirqs.CPU18.SCHED
2831 ? 14% softirqs.CPU26.SCHED
5520 ? 63% softirqs.CPU28.SCHED
3195 ? 3% softirqs.CPU30.SCHED
38070 softirqs.CPU33.SCHED
3134 softirqs.CPU34.SCHED
10927 ?100% softirqs.CPU36.SCHED
10241 ? 98% softirqs.CPU38.SCHED
3241 ? 2% softirqs.CPU4.SCHED
3063 softirqs.CPU42.SCHED
3137 ? 3% softirqs.CPU44.SCHED
12843 ? 54% softirqs.CPU46.SCHED
14604 ?111% softirqs.CPU50.SCHED
37906 softirqs.CPU56.SCHED
10749 ?100% softirqs.CPU6.SCHED
14468 ?111% softirqs.CPU61.SCHED
3122 ? 2% softirqs.CPU65.SCHED
37781 softirqs.CPU66.SCHED
9505 ? 90% softirqs.CPU67.SCHED
35116 ? 9% softirqs.CPU80.SCHED
5126 ? 55% softirqs.CPU83.SCHED
3173 ? 3% softirqs.CPU85.SCHED
3111 ? 2% softirqs.CPU89.SCHED
3152 softirqs.CPU91.SCHED
4952 ? 53% softirqs.CPU95.SCHED
29249 ? 28% softirqs.CPU98.SCHED
23348 ? 62% softirqs.CPU99.SCHED
21.56 perf-stat.i.MPKI
-12.5% 5.404e+09 ? 2% perf-stat.i.branch-instructions
0.48 perf-stat.i.branch-miss-rate%
-11.0% 25947096 perf-stat.i.branch-misses
26.49 perf-stat.i.cache-miss-rate%
+4.7% 1.412e+08 perf-stat.i.cache-misses
5.91 ? 2% perf-stat.i.cpi
1037 perf-stat.i.cycles-between-cache-misses
-12.2% 4269936 ? 2% perf-stat.i.dTLB-load-misses
-12.5% 6.512e+09 ? 2% perf-stat.i.dTLB-loads
-13.0% 4239459 ? 2% perf-stat.i.dTLB-store-misses
-12.5% 2.785e+09 ? 2% perf-stat.i.dTLB-stores
4.63 perf-stat.i.iTLB-load-miss-rate%
-9.4% 4481880 ? 2% perf-stat.i.iTLB-load-misses
-12.0% 92511341 perf-stat.i.iTLB-loads
-12.5% 2.474e+10 ? 2% perf-stat.i.instructions
5618 perf-stat.i.instructions-per-iTLB-miss
0.17 ? 2% perf-stat.i.ipc
147.85 ? 2% perf-stat.i.metric.M/sec
-13.0% 1059028 ? 2% perf-stat.i.minor-faults
+25.6% 22487001 perf-stat.i.node-load-misses
+27.6% 2626451 ? 3% perf-stat.i.node-loads
+3.4% 17708330 perf-stat.i.node-store-misses
-13.0% 1059029 ? 2% perf-stat.i.page-faults
21.56 perf-stat.overall.MPKI
0.48 perf-stat.overall.branch-miss-rate%
26.49 ? 2% perf-stat.overall.cache-miss-rate%
5.92 ? 2% perf-stat.overall.cpi
1035 perf-stat.overall.cycles-between-cache-misses
4.62 perf-stat.overall.iTLB-load-miss-rate%
5521 perf-stat.overall.instructions-per-iTLB-miss
0.17 ? 2% perf-stat.overall.ipc
-12.5% 5.386e+09 ? 2% perf-stat.ps.branch-instructions
-10.9% 25835386 perf-stat.ps.branch-misses
+4.7% 1.408e+08 perf-stat.ps.cache-misses
-12.2% 4255712 ? 2% perf-stat.ps.dTLB-load-misses
-12.4% 6.49e+09 ? 2% perf-stat.ps.dTLB-loads
-13.0% 4225561 ? 2% perf-stat.ps.dTLB-store-misses
-12.5% 2.776e+09 ? 2% perf-stat.ps.dTLB-stores
-9.4% 4467205 ? 2% perf-stat.ps.iTLB-load-misses
-12.0% 92206666 perf-stat.ps.iTLB-loads
-12.5% 2.466e+10 ? 2% perf-stat.ps.instructions
-13.0% 1055544 ? 2% perf-stat.ps.minor-faults
+25.7% 22412778 perf-stat.ps.node-load-misses
+27.6% 2617791 ? 3% perf-stat.ps.node-loads
+3.4% 17649933 perf-stat.ps.node-store-misses
-13.0% 1055545 ? 2% perf-stat.ps.page-faults
-12.3% 7.458e+12 ? 2% perf-stat.total.instructions
719.33 ? 72% interrupts.38:PCI-MSI.67633153-edge.eth0-TxRx-0
3220 ? 15% interrupts.CPU101.NMI:Non-maskable_interrupts
3220 ? 15% interrupts.CPU101.PMI:Performance_monitoring_interrupts
3066 ? 9% interrupts.CPU11.NMI:Non-maskable_interrupts
3066 ? 9% interrupts.CPU11.PMI:Performance_monitoring_interrupts
20.33 ? 79% interrupts.CPU11.RES:Rescheduling_interrupts
6562 ? 28% interrupts.CPU12.NMI:Non-maskable_interrupts
6562 ? 28% interrupts.CPU12.PMI:Performance_monitoring_interrupts
17.67 ? 30% interrupts.CPU13.RES:Rescheduling_interrupts
7875 interrupts.CPU14.NMI:Non-maskable_interrupts
7875 interrupts.CPU14.PMI:Performance_monitoring_interrupts
2893 interrupts.CPU15.NMI:Non-maskable_interrupts
2893 interrupts.CPU15.PMI:Performance_monitoring_interrupts
105.67 ? 62% interrupts.CPU18.RES:Rescheduling_interrupts
58.33 ? 72% interrupts.CPU2.RES:Rescheduling_interrupts
103.00 ? 64% interrupts.CPU20.RES:Rescheduling_interrupts
16.00 ? 77% interrupts.CPU21.RES:Rescheduling_interrupts
258.00 ? 4% interrupts.CPU27.CAL:Function_call_interrupts
-5.4% 450.67 interrupts.CPU28.CAL:Function_call_interrupts
7876 interrupts.CPU28.NMI:Non-maskable_interrupts
7876 interrupts.CPU28.PMI:Performance_monitoring_interrupts
12.67 ? 61% interrupts.CPU29.RES:Rescheduling_interrupts
719.33 ? 72% interrupts.CPU30.38:PCI-MSI.67633153-edge.eth0-TxRx-0
2867 interrupts.CPU33.NMI:Non-maskable_interrupts
2867 interrupts.CPU33.PMI:Performance_monitoring_interrupts
449.33 interrupts.CPU34.CAL:Function_call_interrupts
26.33 ? 70% interrupts.CPU35.RES:Rescheduling_interrupts
2863 interrupts.CPU37.NMI:Non-maskable_interrupts
2863 interrupts.CPU37.PMI:Performance_monitoring_interrupts
17.33 ? 86% interrupts.CPU37.RES:Rescheduling_interrupts
105.67 ? 62% interrupts.CPU38.RES:Rescheduling_interrupts
2388 ? 28% interrupts.CPU39.NMI:Non-maskable_interrupts
2388 ? 28% interrupts.CPU39.PMI:Performance_monitoring_interrupts
12.67 ? 9% interrupts.CPU39.RES:Rescheduling_interrupts
6560 ? 28% interrupts.CPU4.NMI:Non-maskable_interrupts
6560 ? 28% interrupts.CPU4.PMI:Performance_monitoring_interrupts
101.33 ? 64% interrupts.CPU40.RES:Rescheduling_interrupts
102.33 ? 64% interrupts.CPU42.RES:Rescheduling_interrupts
102.67 ? 62% interrupts.CPU44.RES:Rescheduling_interrupts
4058 ? 67% interrupts.CPU5.NMI:Non-maskable_interrupts
4058 ? 67% interrupts.CPU5.PMI:Performance_monitoring_interrupts
68.33 ?104% interrupts.CPU50.RES:Rescheduling_interrupts
2390 ? 28% interrupts.CPU56.NMI:Non-maskable_interrupts
2390 ? 28% interrupts.CPU56.PMI:Performance_monitoring_interrupts
211.67 ?136% interrupts.CPU56.TLB:TLB_shootdowns
330.00 ? 80% interrupts.CPU57.RES:Rescheduling_interrupts
4055 ? 68% interrupts.CPU58.NMI:Non-maskable_interrupts
4055 ? 68% interrupts.CPU58.PMI:Performance_monitoring_interrupts
115.33 ? 40% interrupts.CPU59.RES:Rescheduling_interrupts
140.67 ? 18% interrupts.CPU6.RES:Rescheduling_interrupts
2.67 ? 88% interrupts.CPU60.TLB:TLB_shootdowns
112.00 ? 45% interrupts.CPU61.RES:Rescheduling_interrupts
7668 ? 3% interrupts.CPU63.NMI:Non-maskable_interrupts
7668 ? 3% interrupts.CPU63.PMI:Performance_monitoring_interrupts
26.67 ? 32% interrupts.CPU64.RES:Rescheduling_interrupts
7874 interrupts.CPU65.NMI:Non-maskable_interrupts
7874 interrupts.CPU65.PMI:Performance_monitoring_interrupts
2393 ? 28% interrupts.CPU66.NMI:Non-maskable_interrupts
2393 ? 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts
18.33 ? 68% interrupts.CPU66.RES:Rescheduling_interrupts
7873 interrupts.CPU67.NMI:Non-maskable_interrupts
7873 interrupts.CPU67.PMI:Performance_monitoring_interrupts
1038 ? 66% interrupts.CPU68.CAL:Function_call_interrupts
4056 ? 68% interrupts.CPU7.NMI:Non-maskable_interrupts
4056 ? 68% interrupts.CPU7.PMI:Performance_monitoring_interrupts
99.67 ?136% interrupts.CPU7.TLB:TLB_shootdowns
1921 ? 34% interrupts.CPU70.NMI:Non-maskable_interrupts
1921 ? 34% interrupts.CPU70.PMI:Performance_monitoring_interrupts
75.67 ? 71% interrupts.CPU72.RES:Rescheduling_interrupts
1125 ? 65% interrupts.CPU74.CAL:Function_call_interrupts
12.33 ? 45% interrupts.CPU78.RES:Rescheduling_interrupts
112.33 ? 43% interrupts.CPU79.RES:Rescheduling_interrupts
2861 interrupts.CPU80.NMI:Non-maskable_interrupts
2861 interrupts.CPU80.PMI:Performance_monitoring_interrupts
7.67 ? 40% interrupts.CPU80.RES:Rescheduling_interrupts
100.67 ? 58% interrupts.CPU81.RES:Rescheduling_interrupts
20.00 ? 91% interrupts.CPU82.RES:Rescheduling_interrupts
451.33 interrupts.CPU83.CAL:Function_call_interrupts
7876 interrupts.CPU85.NMI:Non-maskable_interrupts
7876 interrupts.CPU85.PMI:Performance_monitoring_interrupts
452.67 interrupts.CPU87.CAL:Function_call_interrupts
4339 ? 48% interrupts.CPU88.NMI:Non-maskable_interrupts
4339 ? 48% interrupts.CPU88.PMI:Performance_monitoring_interrupts
451.33 interrupts.CPU89.CAL:Function_call_interrupts
92.67 ? 64% interrupts.CPU89.RES:Rescheduling_interrupts
92.67 ? 61% interrupts.CPU91.RES:Rescheduling_interrupts
3222 ? 15% interrupts.CPU92.NMI:Non-maskable_interrupts
3222 ? 15% interrupts.CPU92.PMI:Performance_monitoring_interrupts
2390 ? 28% interrupts.CPU94.NMI:Non-maskable_interrupts
2390 ? 28% interrupts.CPU94.PMI:Performance_monitoring_interrupts
3.67 ?105% interrupts.CPU94.RES:Rescheduling_interrupts
91.67 ? 64% interrupts.CPU95.RES:Rescheduling_interrupts
2402 ? 28% interrupts.CPU96.NMI:Non-maskable_interrupts
2402 ? 28% interrupts.CPU96.PMI:Performance_monitoring_interrupts
9.33 ?105% interrupts.CPU96.RES:Rescheduling_interrupts
92.00 ? 63% interrupts.CPU97.RES:Rescheduling_interrupts
4417 ? 59% interrupts.CPU98.NMI:Non-maskable_interrupts
4417 ? 59% interrupts.CPU98.PMI:Performance_monitoring_interrupts
730.67 ? 39% interrupts.TLB:TLB_shootdowns
10.55 ? 2% perf-profile.calltrace.cycles-pp.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap
10.45 ? 2% perf-profile.calltrace.cycles-pp.flush_tlb_func_common.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu.unmap_region
10.34 ? 2% perf-profile.calltrace.cycles-pp.native_flush_tlb_one_user.flush_tlb_func_common.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu
2.02 ? 15% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.27 ? 15% perf-profile.calltrace.cycles-pp.uncharge_page.mem_cgroup_uncharge_list.release_pages.tlb_flush_mmu.tlb_finish_mmu
0.46 ? 72% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_mm.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault
0.44 ? 72% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_mm.__memcg_kmem_charge_page.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc
1.74 perf-profile.calltrace.cycles-pp.__mmap
1.39 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__mmap
1.81 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
1.73 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
1.12 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
1.09 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
1.10 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
0.94 perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.79 perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
0.52 perf-profile.calltrace.cycles-pp.free_pgtables.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
0.69 ? 3% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__anon_vma_prepare.do_anonymous_page.__handle_mm_fault.handle_mm_fault
0.89 perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
12.65 perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_cancel.page_counter_uncharge.uncharge_batch.mem_cgroup_uncharge_list
33.91 perf-profile.calltrace.cycles-pp.mem_cgroup_uncharge_list.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region
36.16 ? 2% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap
31.35 perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.uncharge_batch.mem_cgroup_uncharge_list.release_pages
32.34 perf-profile.calltrace.cycles-pp.page_counter_uncharge.uncharge_batch.mem_cgroup_uncharge_list.release_pages.tlb_flush_mmu
32.62 perf-profile.calltrace.cycles-pp.uncharge_batch.mem_cgroup_uncharge_list.release_pages.tlb_flush_mmu.tlb_finish_mmu
4.32 ? 3% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.__memcg_kmem_charge.__memcg_kmem_charge_page.__alloc_pages_nodemask
10.55 ? 2% perf-profile.children.cycles-pp.flush_tlb_mm_range
10.46 ? 2% perf-profile.children.cycles-pp.flush_tlb_func_common
10.36 ? 2% perf-profile.children.cycles-pp.native_flush_tlb_one_user
2.03 ? 15% perf-profile.children.cycles-pp.mem_cgroup_charge
1.20 ? 20% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
1.27 ? 15% perf-profile.children.cycles-pp.uncharge_page
1.77 perf-profile.children.cycles-pp.__mmap
1.81 perf-profile.children.cycles-pp.unmap_vmas
1.77 perf-profile.children.cycles-pp.unmap_page_range
0.47 ? 2% perf-profile.children.cycles-pp.___might_sleep
1.09 perf-profile.children.cycles-pp.vm_mmap_pgoff
0.27 perf-profile.children.cycles-pp.free_pgd_range
1.10 perf-profile.children.cycles-pp.ksys_mmap_pgoff
0.34 ? 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.77 ? 3% perf-profile.children.cycles-pp.try_charge
0.34 ? 2% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.94 perf-profile.children.cycles-pp.do_mmap
0.29 perf-profile.children.cycles-pp.get_page_from_freelist
0.14 ? 3% perf-profile.children.cycles-pp.__irqentry_text_end
0.37 ? 2% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.21 ? 3% perf-profile.children.cycles-pp.alloc_pages_vma
0.31 perf-profile.children.cycles-pp._cond_resched
0.80 perf-profile.children.cycles-pp.mmap_region
0.17 ? 2% perf-profile.children.cycles-pp.perf_event_mmap
0.10 ? 4% perf-profile.children.cycles-pp.perf_iterate_sb
0.17 ? 2% perf-profile.children.cycles-pp.rcu_all_qs
0.52 perf-profile.children.cycles-pp.free_pgtables
0.09 ? 5% perf-profile.children.cycles-pp.rmqueue
0.07 ? 7% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size
0.14 ? 3% perf-profile.children.cycles-pp.sync_regs
0.15 perf-profile.children.cycles-pp.clear_page_erms
0.08 perf-profile.children.cycles-pp._raw_spin_lock
0.17 perf-profile.children.cycles-pp.prep_new_page
0.08 ? 16% perf-profile.children.cycles-pp.__list_del_entry_valid
0.50 perf-profile.children.cycles-pp.unlink_anon_vmas
0.08 ? 5% perf-profile.children.cycles-pp.__list_add_valid
0.06 ? 8% perf-profile.children.cycles-pp.__perf_sw_event
0.12 perf-profile.children.cycles-pp.free_unref_page_list
0.06 perf-profile.children.cycles-pp.malloc
0.06 perf-profile.children.cycles-pp.down_write_killable
0.08 perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
0.07 perf-profile.children.cycles-pp.obj_cgroup_charge
0.07 perf-profile.children.cycles-pp.find_vma
0.43 perf-profile.children.cycles-pp.vm_area_alloc
0.09 perf-profile.children.cycles-pp.mem_cgroup_page_lruvec
0.16 ? 5% perf-profile.children.cycles-pp.page_remove_rmap
0.09 ? 9% perf-profile.children.cycles-pp.lock_page_memcg
0.37 ? 5% perf-profile.children.cycles-pp.remove_vma
0.38 ? 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.13 ? 3% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.89 ? 2% perf-profile.children.cycles-pp.zap_pte_range
1.10 ? 2% perf-profile.children.cycles-pp.kmem_cache_alloc
0.41 perf-profile.children.cycles-pp.__count_memcg_events
0.78 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.98 ? 3% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.83 perf-profile.children.cycles-pp.__mod_memcg_state
33.92 perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
36.47 ? 2% perf-profile.children.cycles-pp.release_pages
31.39 perf-profile.children.cycles-pp.page_counter_cancel
32.36 perf-profile.children.cycles-pp.page_counter_uncharge
32.63 perf-profile.children.cycles-pp.uncharge_batch
17.26 perf-profile.children.cycles-pp.propagate_protected_usage
10.34 perf-profile.self.cycles-pp.native_flush_tlb_one_user
8.27 ? 3% perf-profile.self.cycles-pp.page_counter_try_charge
1.20 ? 21% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
1.27 ? 15% perf-profile.self.cycles-pp.uncharge_page
0.92 ? 22% perf-profile.self.cycles-pp.mem_cgroup_charge
0.46 ? 2% perf-profile.self.cycles-pp.___might_sleep
0.25 perf-profile.self.cycles-pp.free_pgd_range
0.34 ? 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.48 ? 2% perf-profile.self.cycles-pp.release_pages
0.12 ? 6% perf-profile.self.cycles-pp.try_charge
0.31 perf-profile.self.cycles-pp.unmap_page_range
0.30 ? 2% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.27 ? 4% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.14 ? 3% perf-profile.self.cycles-pp.__irqentry_text_end
0.12 ? 4% perf-profile.self.cycles-pp.flush_tlb_func_common
0.11 ? 4% perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.07 ? 7% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size
0.15 perf-profile.self.cycles-pp.clear_page_erms
0.08 perf-profile.self.cycles-pp._raw_spin_lock
0.13 ? 3% perf-profile.self.cycles-pp.rcu_all_qs
0.14 ? 3% perf-profile.self.cycles-pp.free_pages_and_swap_cache
0.14 ? 3% perf-profile.self.cycles-pp.sync_regs
0.13 ? 3% perf-profile.self.cycles-pp._cond_resched
0.06 perf-profile.self.cycles-pp.rmqueue
0.08 perf-profile.self.cycles-pp.__list_add_valid
0.06 ? 8% perf-profile.self.cycles-pp.perf_iterate_sb
0.06 perf-profile.self.cycles-pp.free_unref_page_list
0.09 perf-profile.self.cycles-pp.mem_cgroup_page_lruvec
0.65 perf-profile.self.cycles-pp.zap_pte_range
0.28 ? 4% perf-profile.self.cycles-pp.get_obj_cgroup_from_current
0.09 ? 9% perf-profile.self.cycles-pp.lock_page_memcg
0.38 ? 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.41 perf-profile.self.cycles-pp.__count_memcg_events
0.97 ? 7% perf-profile.self.cycles-pp.page_counter_uncharge
0.77 perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.83 perf-profile.self.cycles-pp.__mod_memcg_state
18.63 ? 2% perf-profile.self.cycles-pp.page_counter_cancel
17.18 perf-profile.self.cycles-pp.propagate_protected_usage
************************************************************************
Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
==============================================================
pc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
10/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp4/hackbench/0x4003003
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
fail:runs
|
1:2 last_state.booting
2:2 last_state.is_incomplete_run
************************************************************************
threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
==============================================================
config/rootfs/runtime/tbox_group/test/testcase/ucode:
rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap4/lru-file-mmap-read-rand/vm-scalability/0x5002f01
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
%stddev
\
0.10 vm-scalability.free_time
8680 vm-scalability.median
-2.3% 1676446 vm-scalability.throughput
+1.4% 381.66 vm-scalability.time.elapsed_time
+1.4% 381.66 vm-scalability.time.elapsed_time.max
-1.8% 777639 vm-scalability.time.involuntary_context_switches
-3.6% 48602585 vm-scalability.time.major_page_faults
-11.2% 2881590 ? 4% vm-scalability.time.maximum_resident_set_size
-8.4% 838899 vm-scalability.time.minor_page_faults
58704 vm-scalability.time.system_time
31.02 vm-scalability.time.user_time
5529 vmstat.system.cs
0.24 ? 3% mpstat.cpu.all.soft%
0.08 mpstat.cpu.all.usr%
-8.9% 72179734 ? 2% numa-numastat.node1.numa_miss
-9.0% 72199079 ? 2% numa-numastat.node1.other_node
11852 ? 4% softirqs.CPU171.SCHED
12224 ? 3% softirqs.CPU179.SCHED
16568 ? 8% softirqs.CPU22.RCU
50030 ? 94% numa-meminfo.node0.AnonPages
54912 ? 84% numa-meminfo.node0.Inactive(anon)
19202 ? 15% numa-meminfo.node0.Shmem
183671 ? 23% numa-meminfo.node1.AnonPages
212628 ? 36% numa-meminfo.node1.Inactive(anon)
-15.1% 4482709 ? 9% numa-meminfo.node2.MemFree
12509 ? 94% numa-vmstat.node0.nr_anon_pages
13726 ? 84% numa-vmstat.node0.nr_inactive_anon
4815 ? 15% numa-vmstat.node0.nr_shmem
13736 ? 84% numa-vmstat.node0.nr_zone_inactive_anon
45909 ? 24% numa-vmstat.node1.nr_anon_pages
53130 ? 36% numa-vmstat.node1.nr_inactive_anon
53134 ? 36% numa-vmstat.node1.nr_zone_inactive_anon
9.50 ? 24% numa-vmstat.node1.workingset_restore_file
-13.7% 1158149 ? 8% numa-vmstat.node2.nr_free_pages
+7.3% 371.50 ? 2% numa-vmstat.node2.nr_isolated_file
117.00 ? 23% numa-vmstat.node3.nr_mlock
697781 ? 17% sched_debug.cfs_rq:/.load.max
74893 ? 10% sched_debug.cfs_rq:/.load.stddev
6695 ? 5% sched_debug.cpu.nr_switches.avg
-27.60 sched_debug.cpu.nr_uninterruptible.min
5436 ? 6% sched_debug.cpu.sched_count.avg
3751 ? 6% sched_debug.cpu.sched_count.min
2351 ? 6% sched_debug.cpu.ttwu_count.avg
1508 ? 7% sched_debug.cpu.ttwu_count.min
-9.1% 879.05 ? 5% sched_debug.cpu.ttwu_count.stddev
2010 ? 6% sched_debug.cpu.ttwu_local.avg
5729 ? 10% sched_debug.cpu.ttwu_local.max
1456 ? 6% sched_debug.cpu.ttwu_local.min
613.47 ? 7% sched_debug.cpu.ttwu_local.stddev
-2.4% 65125700 interrupts.CAL:Function_call_interrupts
92.75 ? 33% interrupts.CPU110.RES:Rescheduling_interrupts
72.50 ? 26% interrupts.CPU164.RES:Rescheduling_interrupts
70.75 ? 23% interrupts.CPU167.RES:Rescheduling_interrupts
112.25 ? 17% interrupts.CPU172.RES:Rescheduling_interrupts
8485 interrupts.CPU174.NMI:Non-maskable_interrupts
8485 interrupts.CPU174.PMI:Performance_monitoring_interrupts
76.00 ? 8% interrupts.CPU177.RES:Rescheduling_interrupts
401.00 ? 3% interrupts.CPU25.RES:Rescheduling_interrupts
384.00 ? 8% interrupts.CPU27.RES:Rescheduling_interrupts
339.50 ? 7% interrupts.CPU46.RES:Rescheduling_interrupts
223.75 ? 8% interrupts.CPU61.RES:Rescheduling_interrupts
220.50 ? 11% interrupts.CPU63.RES:Rescheduling_interrupts
170.50 ? 13% interrupts.CPU67.RES:Rescheduling_interrupts
182.25 ? 18% interrupts.CPU68.RES:Rescheduling_interrupts
162.00 ? 8% interrupts.CPU70.RES:Rescheduling_interrupts
148.75 ? 5% interrupts.CPU82.RES:Rescheduling_interrupts
150.50 ? 8% interrupts.CPU83.RES:Rescheduling_interrupts
-3.1% 187482 proc-vmstat.allocstall_movable
2627 ? 3% proc-vmstat.compact_daemon_wake
-17.1% 10099416 ? 11% proc-vmstat.compact_migrate_scanned
380.00 ? 4% proc-vmstat.kswapd_high_wmark_hit_quickly
917.25 proc-vmstat.kswapd_low_wmark_hit_quickly
-5.5% 4668260 ? 2% proc-vmstat.nr_free_pages
1435 proc-vmstat.nr_isolated_file
33262 proc-vmstat.nr_kernel_stack
457.50 proc-vmstat.nr_mlock
11328 ? 32% proc-vmstat.numa_hint_faults
3270 ? 70% proc-vmstat.numa_hint_faults_local
-3.3% 1.266e+09 proc-vmstat.numa_hit
-3.3% 1.266e+09 proc-vmstat.numa_local
3468 proc-vmstat.pageoutrun
-4.6% 752437 proc-vmstat.pgactivate
-3.5% 10194445 proc-vmstat.pgalloc_dma32
-3.5% 1.529e+09 proc-vmstat.pgalloc_normal
-3.9% 709594 proc-vmstat.pgdeactivate
-3.6% 99444010 proc-vmstat.pgfault
-3.6% 1.545e+09 proc-vmstat.pgfree
-3.6% 48555761 proc-vmstat.pgmajfault
-3.9% 709594 proc-vmstat.pgrefill
-2.9% 2.078e+09 proc-vmstat.pgscan_direct
-3.3% 2.214e+09 proc-vmstat.pgscan_file
-10.1% 1.362e+08 ? 3% proc-vmstat.pgscan_kswapd
-3.2% 1.401e+09 proc-vmstat.pgsteal_direct
-3.7% 1.485e+09 proc-vmstat.pgsteal_file
-10.8% 84151622 ? 2% proc-vmstat.pgsteal_kswapd
-7.9% 75987203 proc-vmstat.slabs_scanned
91871 ? 2% proc-vmstat.workingset_activate_file
-4.2% 17225500 proc-vmstat.workingset_nodereclaim
-1.7% 4750078 proc-vmstat.workingset_nodes
-3.8% 1.474e+08 proc-vmstat.workingset_refault_file
-1.4% 1.306e+10 perf-stat.i.branch-instructions
+2.6% 1.112e+08 perf-stat.i.cache-misses
5442 perf-stat.i.context-switches
7.30 perf-stat.i.cpi
3955 perf-stat.i.cycles-between-cache-misses
-1.8% 1.485e+10 perf-stat.i.dTLB-loads
-4.7% 3.851e+09 perf-stat.i.dTLB-stores
-7.5% 6465869 ? 8% perf-stat.i.iTLB-load-misses
855188 ? 22% perf-stat.i.iTLB-loads
-1.8% 5.838e+10 perf-stat.i.instructions
0.17 perf-stat.i.ipc
-5.1% 124241 perf-stat.i.major-faults
-2.0% 166.70 perf-stat.i.metric.M/sec
5755 perf-stat.i.minor-faults
79.53 perf-stat.i.node-load-miss-rate%
+6.1% 25290246 perf-stat.i.node-load-misses
-2.1% 6194197 perf-stat.i.node-loads
-5.1% 11104510 perf-stat.i.node-stores
-5.0% 129997 perf-stat.i.page-faults
4.03 perf-stat.overall.MPKI
8.18 perf-stat.overall.cpi
4327 perf-stat.overall.cycles-between-cache-misses
0.12 perf-stat.overall.ipc
80.52 perf-stat.overall.node-load-miss-rate%
60.73 perf-stat.overall.node-store-miss-rate%
-1.5% 1.335e+10 perf-stat.ps.branch-instructions
+2.7% 1.128e+08 perf-stat.ps.cache-misses
5495 perf-stat.ps.context-switches
-1.9% 1.517e+10 perf-stat.ps.dTLB-loads
-4.7% 3.919e+09 perf-stat.ps.dTLB-stores
-7.5% 6499674 ? 8% perf-stat.ps.iTLB-load-misses
770946 ? 21% perf-stat.ps.iTLB-loads
-1.8% 5.963e+10 perf-stat.ps.instructions
-5.0% 126858 perf-stat.ps.major-faults
5724 perf-stat.ps.minor-faults
+6.1% 25700170 perf-stat.ps.node-load-misses
-1.9% 6216984 perf-stat.ps.node-loads
-5.0% 11328815 perf-stat.ps.node-stores
-5.0% 132583 perf-stat.ps.page-faults
4.23 ? 18% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
2.75 ? 28% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
2.70 ? 29% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list
14.38 ? 4% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
2.37 ? 11% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded.filemap_fault.__xfs_filemap_fault
1.56 ? 16% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded.filemap_fault
39.76 ? 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node
39.35 ? 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
4.36 ? 18% perf-profile.children.cycles-pp.__remove_mapping
14.97 ? 5% perf-profile.children.cycles-pp.shrink_page_list
77.22 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
3.95 ? 15% perf-profile.children.cycles-pp._raw_spin_lock
0.90 ? 9% perf-profile.children.cycles-pp.free_pcppages_bulk
0.15 ? 43% perf-profile.children.cycles-pp.drain_local_pages_wq
0.15 ? 43% perf-profile.children.cycles-pp.drain_pages_zone
0.83 ? 6% perf-profile.children.cycles-pp.free_unref_page_list
0.98 perf-profile.children.cycles-pp.__list_del_entry_valid
0.49 ? 3% perf-profile.children.cycles-pp.xas_create
0.57 ? 2% perf-profile.children.cycles-pp.page_referenced
0.33 perf-profile.children.cycles-pp.page_referenced_one
0.11 ? 4% perf-profile.children.cycles-pp.page_remove_rmap
0.21 ? 2% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.15 ? 12% perf-profile.children.cycles-pp.refill_obj_stock
0.14 ? 13% perf-profile.children.cycles-pp.drain_obj_stock
0.09 ? 20% perf-profile.children.cycles-pp.__memcg_kmem_uncharge
0.22 ? 10% perf-profile.children.cycles-pp.run_ksoftirqd
0.06 ? 7% perf-profile.children.cycles-pp.lru_note_cost
0.22 ? 11% perf-profile.children.cycles-pp.smpboot_thread_fn
0.29 ? 9% perf-profile.children.cycles-pp.rcu_do_batch
0.30 ? 9% perf-profile.children.cycles-pp.__softirqentry_text_start
0.10 ? 7% perf-profile.children.cycles-pp.propagate_protected_usage
0.36 ? 3% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.32 ? 4% perf-profile.children.cycles-pp.uncharge_batch
0.37 ? 7% perf-profile.children.cycles-pp.page_counter_cancel
0.38 ? 6% perf-profile.children.cycles-pp.page_counter_uncharge
0.30 ? 6% perf-profile.children.cycles-pp.page_counter_try_charge
0.48 ? 20% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.40 ? 6% perf-profile.children.cycles-pp.try_charge
0.37 ? 2% perf-profile.children.cycles-pp.workingset_age_nonresident
0.86 ? 3% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
1.56 ? 16% perf-profile.children.cycles-pp.mem_cgroup_charge
2.38 ? 10% perf-profile.children.cycles-pp.__add_to_page_cache_locked
77.22 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.08 ? 5% perf-profile.self.cycles-pp.workingset_eviction
0.11 ? 7% perf-profile.self.cycles-pp.try_charge
0.06 ? 7% perf-profile.self.cycles-pp.lru_note_cost
0.10 ? 8% perf-profile.self.cycles-pp.propagate_protected_usage
0.51 ? 4% perf-profile.self.cycles-pp.__mod_memcg_state
0.31 ? 6% perf-profile.self.cycles-pp.page_counter_cancel
0.25 ? 4% perf-profile.self.cycles-pp.page_counter_try_charge
0.35 ? 3% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.48 ? 20% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.37 ? 2% perf-profile.self.cycles-pp.workingset_age_nonresident
************************************************************************
Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
==============================================================
r/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
m/xfs/libaio/x86_64-rhel-8.3/50%/debian-10.4-x86_64-20200603.cgz/200s/randread/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01
warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()")
reduce memory footprint of percpu_ref in fast path")
cfb19d10f9a82910b
-----------------
%stddev
\
0.03 ? 9% fio.latency_10ms%
0.88 fio.latency_20ms%
16478 fio.read_bw_MBps
+5.0% 352820 fio.read_clat_mean_us
+12.9% 6781430 ? 4% fio.read_clat_stddev
-4.8% 4218471 fio.read_iops
9332 fio.read_slat_mean_us
+9.1% 214657 fio.read_slat_stddev
-4.7% 6.757e+09 fio.time.file_system_inputs
-3.6% 317010 fio.time.involuntary_context_switches
-9.3% 729.76 fio.time.user_time
-4.7% 8.447e+08 fio.workload
3.74 iostat.cpu.user
3.77 mpstat.cpu.all.usr%
2462 ? 9% sched_debug.cpu.nr_switches.min
-4.9% 16521561 vmstat.io.bi
5272 vmstat.system.cs
7940 ? 11% softirqs.CPU13.RCU
13810 ? 26% softirqs.CPU21.SCHED
13735 ? 12% softirqs.CPU22.SCHED
12200 ? 14% softirqs.CPU42.SCHED
11244 ? 12% softirqs.CPU44.SCHED
2212 ? 3% slabinfo.PING.active_objs
2212 ? 3% slabinfo.PING.num_objs
0.00 slabinfo.dio.active_objs
0.00 slabinfo.dio.num_objs
4766 ? 4% slabinfo.kmalloc-rcl-64.active_objs
4766 ? 4% slabinfo.kmalloc-rcl-64.num_objs
2585 ? 4% slabinfo.kmalloc-rcl-96.active_objs
2585 ? 4% slabinfo.kmalloc-rcl-96.num_objs
1008 ? 5% slabinfo.kmem_cache_node.active_objs
-4.7% 249875 ? 2% proc-vmstat.allocstall_movable
3878 ? 10% proc-vmstat.compact_daemon_wake
11773 ? 3% proc-vmstat.kswapd_low_wmark_hit_quickly
-6.5% 299875 proc-vmstat.nr_free_pages
-5.1% 7.798e+08 ? 2% proc-vmstat.numa_hit
-5.1% 7.798e+08 ? 2% proc-vmstat.numa_local
11817 ? 2% proc-vmstat.pageoutrun
-4.8% 30804742 proc-vmstat.pgalloc_dma32
-4.7% 8.149e+08 proc-vmstat.pgalloc_normal
-4.8% 8.36e+08 proc-vmstat.pgfree
-4.7% 3.379e+09 proc-vmstat.pgpgin
-4.7% 6.292e+08 ? 2% proc-vmstat.pgscan_direct
-4.3% 6.818e+08 proc-vmstat.pgscan_file
-4.7% 6.292e+08 ? 2% proc-vmstat.pgsteal_direct
-4.3% 6.818e+08 proc-vmstat.pgsteal_file
638.50 ?169% interrupts.66:PCI-MSI.31981599-edge.i40e-eth0-TxRx-30
63316 interrupts.CAL:Function_call_interrupts
294.00 ? 19% interrupts.CPU13.RES:Rescheduling_interrupts
235.25 ? 18% interrupts.CPU2.RES:Rescheduling_interrupts
+7.7% 743.25 ? 4% interrupts.CPU29.CAL:Function_call_interrupts
6868 ? 12% interrupts.CPU33.NMI:Non-maskable_interrupts
6868 ? 12% interrupts.CPU33.PMI:Performance_monitoring_interrupts
770.00 ? 4% interrupts.CPU34.CAL:Function_call_interrupts
88.50 ? 14% interrupts.CPU41.TLB:TLB_shootdowns
322.75 ? 8% interrupts.CPU42.RES:Rescheduling_interrupts
6928 ? 6% interrupts.CPU5.NMI:Non-maskable_interrupts
6928 ? 6% interrupts.CPU5.PMI:Performance_monitoring_interrupts
5879 ? 21% interrupts.CPU60.NMI:Non-maskable_interrupts
5879 ? 21% interrupts.CPU60.PMI:Performance_monitoring_interrupts
236.75 ? 12% interrupts.CPU8.RES:Rescheduling_interrupts
5578 ? 17% interrupts.CPU91.NMI:Non-maskable_interrupts
5578 ? 17% interrupts.CPU91.PMI:Performance_monitoring_interrupts
5438 ? 39% interrupts.CPU93.NMI:Non-maskable_interrupts
5438 ? 39% interrupts.CPU93.PMI:Performance_monitoring_interrupts
7196 ? 4% interrupts.TLB:TLB_shootdowns
-3.9% 1.518e+10 perf-stat.i.branch-instructions
0.57 perf-stat.i.branch-miss-rate%
-7.5% 84974573 perf-stat.i.branch-misses
-3.7% 6.931e+08 perf-stat.i.cache-misses
5271 perf-stat.i.context-switches
1.75 perf-stat.i.cpi
+2.4% 130.38 perf-stat.i.cpu-migrations
+2.6% 202.49 perf-stat.i.cycles-between-cache-misses
-4.0% 1.995e+10 perf-stat.i.dTLB-loads
-4.7% 1.158e+10 perf-stat.i.dTLB-stores
-15.7% 67439011 ? 3% perf-stat.i.iTLB-load-misses
-18.0% 4545198 ? 5% perf-stat.i.iTLB-loads
-4.1% 7.733e+10 perf-stat.i.instructions
1152 ? 3% perf-stat.i.instructions-per-iTLB-miss
0.57 perf-stat.i.ipc
-4.2% 498.65 perf-stat.i.metric.M/sec
-6.6% 1.037e+08 ? 3% perf-stat.i.node-stores
0.56 perf-stat.overall.branch-miss-rate%
1.75 perf-stat.overall.cpi
+4.1% 194.90 perf-stat.overall.cycles-between-cache-misses
1148 ? 3% perf-stat.overall.instructions-per-iTLB-miss
0.57 perf-stat.overall.ipc
-4.0% 1.51e+10 perf-stat.ps.branch-instructions
-7.5% 84544220 perf-stat.ps.branch-misses
-3.7% 6.896e+08 perf-stat.ps.cache-misses
5243 perf-stat.ps.context-switches
+2.4% 129.98 perf-stat.ps.cpu-migrations
-4.1% 1.985e+10 perf-stat.ps.dTLB-loads
-4.7% 1.152e+10 perf-stat.ps.dTLB-stores
-15.7% 67094648 ? 3% perf-stat.ps.iTLB-load-misses
-18.0% 4522626 ? 5% perf-stat.ps.iTLB-loads
-4.1% 7.694e+10 perf-stat.ps.instructions
-6.7% 1.032e+08 ? 3% perf-stat.ps.node-stores
-4.1% 1.556e+13 perf-stat.total.instructions
11.79 ? 11% perf-profile.calltrace.cycles-pp.read_pages.page_cache_readahead_unbounded.generic_file_buffered_read.xfs_file_buffered_aio_read.xfs_file_read_iter
11.70 ? 11% perf-profile.calltrace.cycles-pp.iomap_readahead.read_pages.page_cache_readahead_unbounded.generic_file_buffered_read.xfs_file_buffered_aio_read
9.62 ? 11% perf-profile.calltrace.cycles-pp.submit_bio.iomap_readahead.read_pages.page_cache_readahead_unbounded.generic_file_buffered_read
9.58 ? 11% perf-profile.calltrace.cycles-pp.submit_bio_noacct.submit_bio.iomap_readahead.read_pages.page_cache_readahead_unbounded
5.75 ? 11% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
3.09 ? 11% perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_buffered_read.xfs_file_buffered_aio_read.xfs_file_read_iter.aio_read
3.05 ? 11% perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.generic_file_buffered_read.xfs_file_buffered_aio_read.xfs_file_read_iter
2.95 ? 11% perf-profile.calltrace.cycles-pp.xas_load.find_get_entry.pagecache_get_page.generic_file_buffered_read.xfs_file_buffered_aio_read
3.50 ? 7% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read.xfs_file_buffered_aio_read
0.27 ?100% perf-profile.calltrace.cycles-pp.submit_bio_checks.submit_bio_noacct.submit_bio.iomap_readahead.read_pages
1.78 ? 11% perf-profile.calltrace.cycles-pp.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list
1.72 ? 11% perf-profile.calltrace.cycles-pp.xas_create.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list
1.08 ? 17% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
0.77 ? 19% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list
0.74 ? 7% perf-profile.calltrace.cycles-pp.workingset_eviction.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
1.04 ? 9% perf-profile.calltrace.cycles-pp.do_io_getevents.__x64_sys_io_getevents.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.11 ? 8% perf-profile.calltrace.cycles-pp.__x64_sys_io_getevents.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.89 ? 9% perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_buffered_read.xfs_file_buffered_aio_read.xfs_file_read_iter.aio_read
1.03 ? 10% perf-profile.calltrace.cycles-pp.iomap_readahead_actor.iomap_apply.iomap_readahead.read_pages.page_cache_readahead_unbounded
0.76 ? 10% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.generic_file_buffered_read.xfs_file_buffered_aio_read.xfs_file_read_iter
0.75 ? 13% perf-profile.calltrace.cycles-pp.xfs_read_iomap_begin.iomap_apply.iomap_readahead.read_pages.page_cache_readahead_unbounded
0.74 ? 8% perf-profile.calltrace.cycles-pp.read_events.do_io_getevents.__x64_sys_io_getevents.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.72 ? 10% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.generic_file_buffered_read.xfs_file_buffered_aio_read
0.83 ? 8% perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.lru_cache_add.add_to_page_cache_lru.page_cache_readahead_unbounded
0.68 ? 8% perf-profile.calltrace.cycles-pp.aio_read_events.read_events.do_io_getevents.__x64_sys_io_getevents.do_syscall_64
0.62 ? 12% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
0.60 ? 6% perf-profile.calltrace.cycles-pp.try_charge.mem_cgroup_charge.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded
11.79 ? 11% perf-profile.children.cycles-pp.read_pages
11.71 ? 11% perf-profile.children.cycles-pp.iomap_readahead
9.62 ? 11% perf-profile.children.cycles-pp.submit_bio
9.59 ? 11% perf-profile.children.cycles-pp.submit_bio_noacct
5.10 ? 11% perf-profile.children.cycles-pp.__remove_mapping
6.35 ? 12% perf-profile.children.cycles-pp.shrink_page_list
3.42 ? 10% perf-profile.children.cycles-pp.xas_load
3.10 ? 11% perf-profile.children.cycles-pp.pagecache_get_page
3.06 ? 11% perf-profile.children.cycles-pp.find_get_entry
3.51 ? 7% perf-profile.children.cycles-pp.__add_to_page_cache_locked
2.66 ? 9% perf-profile.children.cycles-pp.xas_store
2.78 ? 10% perf-profile.children.cycles-pp.__delete_from_page_cache
1.97 ? 11% perf-profile.children.cycles-pp.xas_create
1.71 ? 10% perf-profile.children.cycles-pp.__list_del_entry_valid
0.41 ? 7% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.82 ? 6% perf-profile.children.cycles-pp.workingset_eviction
0.51 ? 6% perf-profile.children.cycles-pp.__count_memcg_events
1.05 ? 9% perf-profile.children.cycles-pp.do_io_getevents
0.93 ? 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.11 ? 8% perf-profile.children.cycles-pp.__x64_sys_io_getevents
0.90 ? 9% perf-profile.children.cycles-pp.copy_page_to_iter
0.68 ? 5% perf-profile.children.cycles-pp.__mod_memcg_state
0.96 ? 7% perf-profile.children.cycles-pp.free_unref_page_list
1.03 ? 10% perf-profile.children.cycles-pp.iomap_readahead_actor
0.50 ? 10% perf-profile.children.cycles-pp.submit_bio_checks
0.76 ? 10% perf-profile.children.cycles-pp.copyout
0.75 ? 8% perf-profile.children.cycles-pp.read_events
0.76 ? 13% perf-profile.children.cycles-pp.xfs_read_iomap_begin
0.84 ? 8% perf-profile.children.cycles-pp.free_pcppages_bulk
0.84 ? 8% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.69 ? 8% perf-profile.children.cycles-pp.aio_read_events
0.49 ? 10% perf-profile.children.cycles-pp.lookup_ioctx
0.32 ? 13% perf-profile.children.cycles-pp.remove_mapping
0.47 ? 8% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.35 ? 9% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.42 ? 9% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.28 ? 7% perf-profile.children.cycles-pp.__mod_lruvec_state
0.35 ? 8% perf-profile.children.cycles-pp.hrtimer_interrupt
0.39 ? 8% perf-profile.children.cycles-pp.asm_call_on_stack
0.24 ? 6% perf-profile.children.cycles-pp.__mod_node_page_state
0.20 ? 13% perf-profile.children.cycles-pp.blk_throtl_bio
0.20 ? 14% perf-profile.children.cycles-pp.ktime_get
0.27 ? 8% perf-profile.children.cycles-pp.__might_fault
0.18 ? 10% perf-profile.children.cycles-pp.pagevec_lookup_entries
0.18 ? 10% perf-profile.children.cycles-pp.find_get_entries
0.28 ? 7% perf-profile.children.cycles-pp.___might_sleep
0.20 ? 12% perf-profile.children.cycles-pp._copy_from_user
0.31 ? 10% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.17 ? 7% perf-profile.children.cycles-pp.__get_user_4
0.09 ? 4% perf-profile.children.cycles-pp.bio_endio
0.36 ? 7% perf-profile.children.cycles-pp.workingset_update_node
0.14 ? 12% perf-profile.children.cycles-pp.xas_start
0.24 ? 8% perf-profile.children.cycles-pp.__might_sleep
0.17 ? 9% perf-profile.children.cycles-pp._copy_to_user
0.23 ? 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.11 ? 14% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.15 ? 8% perf-profile.children.cycles-pp.__fget_files
0.20 ? 7% perf-profile.children.cycles-pp._cond_resched
0.10 ? 10% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.10 ? 8% perf-profile.children.cycles-pp.mutex_lock
0.04 ? 57% perf-profile.children.cycles-pp.mem_cgroup_page_lruvec
0.14 ? 8% perf-profile.children.cycles-pp.__check_object_size
0.09 ? 7% perf-profile.children.cycles-pp.mark_page_accessed
0.06 ? 6% perf-profile.children.cycles-pp.lru_note_cost
0.15 ? 12% perf-profile.children.cycles-pp.propagate_protected_usage
0.60 ? 7% perf-profile.children.cycles-pp.try_charge
0.42 ? 5% perf-profile.children.cycles-pp.page_counter_try_charge
0.61 ? 7% perf-profile.children.cycles-pp.workingset_age_nonresident
0.75 ? 11% perf-profile.children.cycles-pp.page_counter_cancel
0.92 ? 10% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.77 ? 12% perf-profile.children.cycles-pp.page_counter_uncharge
0.82 ? 10% perf-profile.children.cycles-pp.uncharge_batch
3.25 ? 10% perf-profile.self.cycles-pp.xas_load
0.21 ? 6% perf-profile.self.cycles-pp.workingset_eviction
1.92 ? 12% perf-profile.self.cycles-pp.xas_create
1.71 ? 10% perf-profile.self.cycles-pp.__list_del_entry_valid
0.51 ? 6% perf-profile.self.cycles-pp.__count_memcg_events
0.92 ? 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.67 ? 5% perf-profile.self.cycles-pp.__mod_memcg_state
0.34 ? 11% perf-profile.self.cycles-pp.io_submit_one
0.43 ? 7% perf-profile.self.cycles-pp.free_pcppages_bulk
0.34 ? 15% perf-profile.self.cycles-pp.mem_cgroup_charge
0.30 ? 9% perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.28 ? 14% perf-profile.self.cycles-pp.get_page_from_freelist
0.24 ? 5% perf-profile.self.cycles-pp.__mod_node_page_state
0.23 ? 14% perf-profile.self.cycles-pp.lookup_ioctx
0.17 ? 7% perf-profile.self.cycles-pp.__get_user_4
0.16 ? 8% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.27 ? 8% perf-profile.self.cycles-pp.___might_sleep
0.14 ? 3% perf-profile.self.cycles-pp.release_pages
0.15 ? 11% perf-profile.self.cycles-pp.find_get_entries
0.06 ? 7% perf-profile.self.cycles-pp.bio_endio
0.22 ? 8% perf-profile.self.cycles-pp.__might_sleep
0.17 ? 11% perf-profile.self.cycles-pp.pmem_submit_bio
0.19 ? 4% perf-profile.self.cycles-pp.xas_store
0.15 ? 7% perf-profile.self.cycles-pp.__fget_files
0.12 ? 9% perf-profile.self.cycles-pp.__alloc_pages_nodemask
0.13 ? 13% perf-profile.self.cycles-pp.xas_start
0.16 ? 4% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.10 ? 17% perf-profile.self.cycles-pp.blk_throtl_bio
0.16 ? 4% perf-profile.self.cycles-pp.kmem_cache_free
0.10 ? 8% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.11 ? 11% perf-profile.self.cycles-pp.read_tsc
0.10 ? 10% perf-profile.self.cycles-pp.find_get_entry
0.08 ? 11% perf-profile.self.cycles-pp.__put_user_4
0.08 ? 5% perf-profile.self.cycles-pp.mark_page_accessed
0.07 ? 12% perf-profile.self.cycles-pp.aio_read
0.07 ? 5% perf-profile.self.cycles-pp.submit_bio_noacct
0.08 ? 10% perf-profile.self.cycles-pp.lru_cache_add
0.08 ? 8% perf-profile.self.cycles-pp._cond_resched
0.07 ? 10% perf-profile.self.cycles-pp.copy_page_to_iter
0.07 ? 6% perf-profile.self.cycles-pp.__x64_sys_io_getevents
0.06 ? 6% perf-profile.self.cycles-pp.lru_note_cost
0.41 ? 6% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.15 ? 12% perf-profile.self.cycles-pp.propagate_protected_usage
0.38 ? 6% perf-profile.self.cycles-pp.page_counter_try_charge
0.64 ? 11% perf-profile.self.cycles-pp.page_counter_cancel
0.61 ? 8% perf-profile.self.cycles-pp.workingset_age_nonresident
estimated based on internal Intel analysis and are provided
purposes only. Any difference in system hardware or software
may affect actual performance.

Attachments:

(No filename) (175.02 kB)
job-script (7.59 kB)
job.yaml (5.22 kB)
reproduce (14.26 kB)
Download all attachments

2021-01-11 10:04:50

by Ming Lei

[permalink] [raw]

Subject: Re: [percpu_ref] 2b0d3d3e4f: reaim.jobs_per_min -18.4% regression

On Sun, Jan 10, 2021 at 10:32:47PM +0800, kernel test robot wrote:
>
> Greeting,
>
> FYI, we noticed a -18.4% regression of reaim.jobs_per_min due to commit:
>
>
> commit: 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e ("percpu_ref: reduce memory footprint of percpu_ref in fast path")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
>
> in testcase: reaim
> on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
> with following parameters:
>
> runtime: 300s
> nr_task: 100%
> test: short
> cpufreq_governor: performance
> ucode: 0x5002f01
>
> test-description: REAIM is an updated and improved version of AIM 7 benchmark.
> test-url: https://sourceforge.net/projects/re-aim-7/
>
> In addition to that, the commit also has significant impact on the following tests:
>
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | vm-scalability: vm-scalability.throughput -2.8% regression |
> | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
> | test parameters | cpufreq_governor=performance |
> | | runtime=300s |
> | | test=lru-file-mmap-read-rand |
> | | ucode=0x5003003 |
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | will-it-scale: will-it-scale.per_process_ops 14.5% improvement |
> | test machine | 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory |
> | test parameters | cpufreq_governor=performance |
> | | mode=process |
> | | nr_task=50% |
> | | test=page_fault2 |
> | | ucode=0x16 |
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | will-it-scale: will-it-scale.per_process_ops -13.0% regression |
> | test machine | 104 threads Skylake with 192G memory |
> | test parameters | cpufreq_governor=performance |
> | | mode=process |
> | | nr_task=50% |
> | | test=malloc1 |
> | | ucode=0x2006906 |
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | vm-scalability: vm-scalability.throughput -2.3% regression |
> | test machine | 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
> | test parameters | cpufreq_governor=performance |
> | | runtime=300s |
> | | test=lru-file-mmap-read-rand |
> | | ucode=0x5002f01 |
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | fio-basic: fio.read_iops -4.8% regression |
> | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
> | test parameters | bs=4k |
> | | cpufreq_governor=performance |
> | | disk=2pmem |
> | | fs=xfs |
> | | ioengine=libaio |
> | | nr_task=50% |
> | | runtime=200s |
> | | rw=randread |
> | | test_size=200G |
> | | time_based=tb |
> | | ucode=0x5002f01 |
> +------------------+---------------------------------------------------------------------------+
> | testcase: change | stress-ng: stress-ng.stackmmap.ops_per_sec -45.4% regression |
> | test machine | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
> | test parameters | class=memory |
> | | cpufreq_governor=performance |
> | | disk=1HDD |
> | | nr_threads=100% |
> | | testtime=10s |
> | | ucode=0x5002f01 |
> +------------------+---------------------------------------------------------------------------+

Just run a quick test of the last two on 2b0d3d3e4fcf ("percpu_ref: reduce memory footprint of
percpu_ref in fast path) and cf785af19319 ("block: warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()").

Not see difference in the two kernel(fio on null_blk with 224 hw queues,
and 'stress-ng --stackmmap-ops') on one 224 cores, dual sockets system.

BTW this patch itself doesn't touch fast path code, so it is supposed to
not affect performance.

Can you double check if the test itself is good?

Note: cf785af19319 is 2b0d3d3e4fcf^

Thanks,
Ming

2021-01-19 05:56:18

by Xing, Zhengjun

[permalink] [raw]

Subject: Re: [LKP] Re: [percpu_ref] 2b0d3d3e4f: reaim.jobs_per_min -18.4% regression

On 1/11/2021 5:58 PM, Ming Lei wrote:
> On Sun, Jan 10, 2021 at 10:32:47PM +0800, kernel test robot wrote:
>> Greeting,
>>
>> FYI, we noticed a -18.4% regression of reaim.jobs_per_min due to commit:
>>
>>
>> commit: 2b0d3d3e4fcfb19d10f9a82910b8f0f05c56ee3e ("percpu_ref: reduce memory footprint of percpu_ref in fast path")
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>
>>
>> in testcase: reaim
>> on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
>> with following parameters:
>>
>> runtime: 300s
>> nr_task: 100%
>> test: short
>> cpufreq_governor: performance
>> ucode: 0x5002f01
>>
>> test-description: REAIM is an updated and improved version of AIM 7 benchmark.
>> test-url: https://sourceforge.net/projects/re-aim-7/
>>
>> In addition to that, the commit also has significant impact on the following tests:
>>
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | vm-scalability: vm-scalability.throughput -2.8% regression |
>> | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
>> | test parameters | cpufreq_governor=performance |
>> | | runtime=300s |
>> | | test=lru-file-mmap-read-rand |
>> | | ucode=0x5003003 |
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | will-it-scale: will-it-scale.per_process_ops 14.5% improvement |
>> | test machine | 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory |
>> | test parameters | cpufreq_governor=performance |
>> | | mode=process |
>> | | nr_task=50% |
>> | | test=page_fault2 |
>> | | ucode=0x16 |
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | will-it-scale: will-it-scale.per_process_ops -13.0% regression |
>> | test machine | 104 threads Skylake with 192G memory |
>> | test parameters | cpufreq_governor=performance |
>> | | mode=process |
>> | | nr_task=50% |
>> | | test=malloc1 |
>> | | ucode=0x2006906 |
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | vm-scalability: vm-scalability.throughput -2.3% regression |
>> | test machine | 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
>> | test parameters | cpufreq_governor=performance |
>> | | runtime=300s |
>> | | test=lru-file-mmap-read-rand |
>> | | ucode=0x5002f01 |
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | fio-basic: fio.read_iops -4.8% regression |
>> | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
>> | test parameters | bs=4k |
>> | | cpufreq_governor=performance |
>> | | disk=2pmem |
>> | | fs=xfs |
>> | | ioengine=libaio |
>> | | nr_task=50% |
>> | | runtime=200s |
>> | | rw=randread |
>> | | test_size=200G |
>> | | time_based=tb |
>> | | ucode=0x5002f01 |
>> +------------------+---------------------------------------------------------------------------+
>> | testcase: change | stress-ng: stress-ng.stackmmap.ops_per_sec -45.4% regression |
>> | test machine | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
>> | test parameters | class=memory |
>> | | cpufreq_governor=performance |
>> | | disk=1HDD |
>> | | nr_threads=100% |
>> | | testtime=10s |
>> | | ucode=0x5002f01 |
>> +------------------+---------------------------------------------------------------------------+
> Just run a quick test of the last two on 2b0d3d3e4fcf ("percpu_ref: reduce memory footprint of
> percpu_ref in fast path) and cf785af19319 ("block: warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx()").
>
> Not see difference in the two kernel(fio on null_blk with 224 hw queues,
> and 'stress-ng --stackmmap-ops') on one 224 cores, dual sockets system.
>
> BTW this patch itself doesn't touch fast path code, so it is supposed to
> not affect performance.
>
> Can you double check if the test itself is good?
I re-test the "fio-basic: fio.read_iops -4.8% regression" for more than
5 times, the average regression is -2.3%.
For "stress_ng", normally, it tests a lot of cases one by one. Command
'stress-ng --stackmmap-ops' only test "stackmmap" case.
I also tried only test "stackmmap" case, the regression is -7.8%.

But for here, it mainly reports "reaim.jobs_per_min -18.4% regression",
I re-test "reaim" case, the result is almost the same.
>
> Note: cf785af19319 is 2b0d3d3e4fcf^
>
>
>
> Thanks,
> Ming
> _______________________________________________
> LKP mailing list -- [email protected]
> To unsubscribe send an email to [email protected]

--
Zhengjun Xing