LinuxLists.cc - [linus:master] [sched] af7f588d8f: will-it-scale.per_thread

2023-05-15 07:35:39

Subject: [linus:master] [sched] af7f588d8f: will-it-scale.per_thread_ops -13.9% regression

Hello,

kernel test robot
commit: af7f588d8f7355bc42 In addition to that,
+------------------+------ | testcase: change | test machine | test parameters | | mode=thread | | nr_task=16 | +------------------+------ | testcase: change | test machine | test parameters | | mode=thread | | nr_task=16 | +------------------+------ | testcase: change | test machine | test parameters | | mode=thread | | nr_task=50% | +------------------+------ | testcase: change | test machine | test parameters | | runtime=300s | | size=2T | +------------------+------
FYI, we noticed that regression introduced will-it-scale context_switch1 fix commit.

Furthermore, we applied the will-it-scale patch.

[1]
Details are as below:

========================== compiler/cpufreq_governor/ gcc-11/performance/x86_6
commit:
99babd04b2 ("selftests/rseq: af7f588d8f ("sched:
99babd04b2505471 ---------------- %stddev %change \ | 629260 629260 613314 70477120 5615798 ? 4% 24.94 ? 2% -90.0% 1.057e+08 56978825 7.49 -73.8% 1.41e+08 95368601 1.082e+08 35238560 8809640 ? 2% 68.53 328629 0.68 +3.2% 4.074e+08 ? 12% 17476 1623 -15.7% 6.135e+09 4.584e+08 4.804e+09 0.12 ? 4% -0.0 22357676 59747111 433036 ? 12% 572995 ? 11% 8059420 8122292 2.202e+08 ? 5% 0.93 ? 5% -0.8 1.101e+08 ? 2% 3.22 -0.5 231050 +1893.9% 230970 +1894.6% 219559 ? 5% 572890 21995823 25221015 19440153 19440011 214830 25424026 214830 19098913 25832239 110043 ? 15% -88.5% 2804771 ? 2% 504968 ? 9% -82.6% 110043 ? 15% -88.5% 2804771 ? 2% 504968 ? 9% -82.6% 854772 ? 2% 73087 ? 59% +734.2% 861849 ? 2% 322.78 ? 2% 470921 ? 4% 10146 ? 89% +197.5% 166.82 ? 15% -82.9% 0.00 ? 15% -78.3% 29334462 9400046 1494 ? 50% +1.6e+05% 1459 ? 53% +1.7e+05% 193090 ? 23% +315.9% 193026 ? 23% +316.0% 9933 ? 9% +50.4% 6124 ? 15% +49221.4% 229095 229051 145346 ? 16% -45.5% 383868 ? 12% 19095555 19201636 19201557 102686 ? 9% -34.7% 2789340 ? 3% 1.113e+08 20783078 16570 ? 5% -38.1% 102686 ? 9% -34.7% 19047079 282546 ? 6% 364.56 ? 53% +1.7e+05% 48203 ? 23% +317.2% 2477 ? 10% +50.9% 1530 ? 15% +49298.5% 364.56 ? 53% +1.7e+05% 48203 ? 23% +317.2% 573038 ? 11% 433108 ? 12% 57253 +843.2% 95911 ? 12% -49.6% 4773715 27815434 4800160 -98.0% 697703 ? 3% 4135 ? 5% -37.9% 4761596 25670 ? 9% -34.7% 57253 +843.2% 4800159 -98.0% 8122528 8059656 57680 +1900.0% 143171 5919284 11853042 5493638 59564263 4854706 6640 -4.7% 4769410 53693 -14.7% 57680 +1900.0% 4854706 683375 ? 10% 519396 ? 17% 8698455 938.89 ? 18% 8495696 1313649 ? 5% 61945 +1240.4% 8928138 2256846 ? 2% 1520379 ? 2% 0.38 ? 4% +115.9% 7.631e+10 0.67 -0.1 4.989e+08 6418393 50457351 ? 5% 60968060 2.14 ? 2% +13.1% 4.35e+11 1058 ? 14% 385252 ? 4% 0.01 ? 22% -0.0 2628303 ? 7% 1.104e+11 0.00 ? 13% +0.0 242451 ? 5% 6.626e+10 3.816e+11 0.80 -6.6% 1.94 +1.7% 504.83 ? 2% 1129 -6.2% 6259 ? 2% -11.2% 58.32 ? 3% +29.9 1117525 458656 ? 5% 6260 ? 2% -11.2% 0.13 ? 5% +61.1% 12.66 ? 5% -4.9 1.15 +7.5% 67823 +10.3% 0.00 ? 7% -0.0 0.87 -7.0% 69.79 ? 2% +21.7 260155 7.514e+10 4.914e+08 50614911 ? 5% 60049636 4.331e+11 1048 ? 13% 2706759 ? 7% 1.087e+11 242516 ? 5% 6.523e+10 3.757e+11 6291 ? 2% -11.8% 1103763 478062 ? 5% 6292 ? 2% -11.8% 1.192e+14 20.06 ? 2% -20.1 22.16 ? 2% -17.8 22.02 ? 2% -17.7 20.05 ? 2% -16.1 58.25 -9.4 11.88 ? 2% -9.3 11.73 -9.2 10.32 ? 2% -9.1 11.28 -8.8 10.60 ? 2% -8.5 10.69 -8.4 10.59 ? 2% -8.3 10.62 -8.3 10.49 -8.3 10.36 ? 2% -8.2 10.61 ? 3% -8.1 8.71 ? 2% -8.1 56.47 -8.0 56.28 -7.8 9.88 -7.8 9.76 -7.7 9.76 -7.7 9.48 ? 2% -7.5 55.87 -7.5 55.45 -7.2 8.86 ? 2% -7.0 8.63 ? 2% -6.7 8.41 ? 2% -6.6 8.42 ? 2% -6.6 54.70 -6.5 8.30 ? 2% -6.5 8.18 ? 2% -6.5 8.10 ? 2% -6.4 8.20 ? 2% -6.4 6.10 ? 2% -6.1 7.43 ? 2% -6.0 6.92 ? 2% -5.4 6.85 ? 2% -5.4 6.68 ? 2% -5.3 6.70 ? 2% -5.2 6.50 ? 2% -5.1 6.47 ? 2% -5.1 6.40 ? 2% -5.0 5.69 ? 2% -4.9 5.85 ? 2% -4.7 5.83 ? 2% -4.6 5.66 ? 2% -4.4 52.13 -4.4 5.50 ? 2% -4.3 51.86 -4.1 3.00 ? 6% -2.4 3.00 ? 6% -2.4 28.01 +19.1 27.98 +19.1 27.90 +19.2 28.25 +19.3 13.04 +30.9 12.84 +31.1 0.00 +41.3 0.00 +41.5 0.00 +41.6 0.00 +41.8 0.00 +42.0 0.00 +42.1 27.66 ? 2% -21.6 26.84 ? 2% -21.0 26.57 ? 2% -20.8 21.19 ? 2% -20.4 24.08 ? 2% -19.2 22.32 ? 2% -17.8 22.19 ? 2% -17.7 68.01 -16.5 67.76 -16.3 20.43 ? 2% -16.3 16.08 ? 2% -12.8 12.85 ? 2% -10.0 58.48 -9.6 11.84 -9.3 12.30 ? 2% -9.1 10.55 ? 2% -8.4 11.30 ? 2% -8.4 10.60 ? 2% -8.3 10.42 ? 2% -8.2 10.93 ? 2% -8.1 9.96 -7.9 9.92 -7.8 9.80 -7.7 55.91 -7.5 9.65 ? 2% -7.5 55.47 -7.2 8.95 ? 2% -7.0 8.65 ? 2% -6.7 54.85 -6.6 8.43 ? 2% -6.6 8.35 ? 2% -6.5 8.23 ? 2% -6.4 6.86 ? 2% -5.4 5.79 ? 2% -4.9 52.14 -4.3 3.81 ? 5% -3.7 4.46 ? 2% -3.5 3.03 ? 6% -2.4 3.03 ? 6% -2.4 2.85 ? 2% -2.3 2.44 ? 2% -2.0 2.08 ? 2% -1.7 2.09 ? 2% -1.6 2.00 -1.5 1.87 -1.5 1.53 ? 2% -1.2 1.37 ? 2% -1.0 1.49 -1.0 1.43 -1.0 1.26 -0.9 1.13 ? 2% -0.9 1.24 -0.9 1.53 ? 2% -0.8 0.96 ? 3% -0.8 0.94 -0.7 0.79 ? 3% -0.6 0.76 ? 3% -0.6 0.75 ? 2% -0.6 0.91 ? 4% -0.6 0.72 -0.6 1.16 ? 2% -0.6 0.70 -0.6 0.74 ? 2% -0.6 0.67 ? 2% -0.5 0.65 -0.5 1.01 -0.5 0.69 ? 2% -0.5 0.59 -0.5 0.55 ? 2% -0.5 0.67 ? 6% -0.4 0.52 -0.4 0.50 ? 2% -0.4 0.58 ? 4% -0.4 0.49 ? 2% -0.4 0.42 -0.3 0.45 ? 5% -0.3 0.43 ? 2% -0.3 0.52 ? 8% -0.3 0.42 ? 5% -0.3 0.43 ? 5% -0.3 0.48 ? 4% -0.3 0.59 ? 3% -0.3 0.38 ? 2% -0.3 0.38 ? 5% -0.3 0.36 ? 2% -0.3 0.35 ? 4% -0.3 0.42 ? 3% -0.3 0.34 ? 7% -0.3 0.32 -0.3 0.39 ? 2% -0.3 0.31 ? 2% -0.3 0.29 ? 5% -0.2 0.31 ? 2% -0.2 0.29 ? 2% -0.2 0.30 ? 2% -0.2 0.29 ? 2% -0.2 0.29 ? 2% -0.2 0.28 ? 4% -0.2 0.27 ? 3% -0.2 0.26 ? 2% -0.2 0.26 ? 3% -0.2 0.24 ? 4% -0.2 0.26 ? 2% -0.2 0.35 ? 11% -0.2 0.79 ? 11% -0.2 0.22 -0.2 0.26 ? 3% -0.2 0.26 ? 3% -0.2 0.79 ? 11% -0.2 0.79 ? 11% -0.1 0.22 ? 3% -0.1 0.21 ? 3% -0.1 0.22 ? 6% -0.1 0.14 ? 6% -0.1 0.16 ? 9% -0.1 0.14 ? 11% -0.1 0.16 ? 2% -0.1 0.13 ? 5% -0.1 0.13 ? 3% -0.1 0.10 ? 4% -0.0 0.11 ? 12% -0.0 0.12 ? 6% +0.0 0.15 ? 11% +0.1 0.15 ? 6% +0.1 0.15 ? 7% +0.1 0.00 +0.1 0.24 ? 16% +0.2 0.24 ? 16% +0.2 0.24 ? 16% +0.2 0.00 +0.9 0.00 +0.9 0.00 +0.9 0.00 +1.0 0.16 ? 4% +1.0 28.01 +19.1 28.25 +19.3 28.25 +19.3 28.19 +19.4 65.01 +27.1 13.19 +31.2 0.00 +83.5 0.42 ? 2% +83.7 0.36 ? 2% +83.8 20.28 ? 2% -19.9 14.57 -14.2 6.94 ? 2% -5.5 5.60 ? 2% -4.8 3.77 ? 5% -3.6 4.25 ? 2% -3.4 3.03 ? 6% -2.4 2.31 ? 2% -1.8 1.87 ? 2% -1.5 0.97 ? 2% -0.8 0.94 ? 2% -0.7 0.75 ? 2% -0.6 0.69 ? 2% -0.6 0.67 -0.5 0.64 -0.5 0.70 -0.5 0.51 -0.4 0.55 ? 2% -0.4 0.65 ? 6% -0.4 0.52 ? 3% -0.4 0.47 ? 2% -0.4 0.48 ? 2% -0.4 0.47 ? 2% -0.4 0.45 ? 2% -0.4 0.45 ? 3% -0.4 0.43 ? 2% -0.3 0.40 ? 3% -0.3 0.43 -0.3 0.36 ? 2% -0.3 0.36 ? 2% -0.3 0.37 ? 5% -0.3 0.38 ? 2% -0.3 0.55 ? 4% -0.3 0.33 ? 2% -0.3 0.33 -0.3 0.33 ? 3% -0.3 0.33 ? 6% -0.3 0.31 ? 3% -0.3 0.40 ? 4% -0.2 0.41 ? 3% -0.2 0.30 ? 2% -0.2 0.31 ? 7% -0.2 0.48 -0.2 0.29 ? 2% -0.2 0.30 ? 2% -0.2 0.28 -0.2 0.26 ? 2% -0.2 0.26 -0.2 0.26 ? 3% -0.2 0.22 ? 2% -0.2 0.25 ? 4% -0.2 0.33 ? 3% -0.2 0.22 ? 4% -0.2 0.22 ? 10% -0.2 0.21 ? 2% -0.2 0.18 ? 2% -0.1 0.19 ? 5% -0.1 0.20 ? 5% -0.1 0.20 -0.1 0.21 ? 3% -0.1 0.14 ? 4% -0.1 0.15 ? 4% -0.1 0.13 ? 5% -0.1 0.12 ? 2% -0.1 0.12 ? 5% -0.1 0.14 ? 11% +0.1 0.33 +0.3 0.00 +0.7 0.00 +83.5

************************** lkp-spr-r02: 224 ========================== compiler/cpufreq_governor/ gcc-11/performance/x86_6
commit:
99babd04b2 ("selftests/rseq: af7f588d8f ("sched:
99babd04b2505471 ---------------- %stddev %change \ | 9168456 573027 9168456 2.864e+09 6.10 +1.0 1.13 ? 3% -0.9 22292871 ? 7% 19.00 ? 3% -14.7% 18144385 483366 1555224 ? 5% 87537260 ? 26% 47.94 ? 62% +27.4 0.59 -66.0% 1.468e+08 2.746e+09 1.67 ? 2% -1.0 34.20 433.23 ? 2% 33394 ? 5% +814.1% 55976 ? 9% -22.7% 4483729 ? 33% 28119032 ? 5% 4336878 ? 31% -97.7% 1758120 ? 22% 4166 ? 30% -75.4% 4314309 ? 32% 33394 ? 5% +814.1% 4336876 ? 31% -97.7% 134596 ? 6% 134504 ? 6% 223873 ? 9% 17932980 ? 33% 17344798 ? 31% 17344618 ? 31% 7027715 ? 22% 1.125e+08 ? 5% 19565762 ? 30% 16664 ? 30% -75.4% 17255300 ? 32% 421445 ? 2% 22061123 ? 7% 20088958 ? 8% 19170771 ? 11% 19170591 ? 11% 213051 7430615 ? 13% 25328866 ? 7% 21777 ? 8% -63.7% 213051 19164273 ? 8% 26692881 65901 ? 13% -98.2% 1327262 ? 10% -93.4% 265536 ? 3% -96.2% 65901 ? 13% -98.2% 1327262 ? 10% -93.4% 265536 ? 3% -96.2% 33780 ? 13% +39.7% 979.30 ? 2% 978.91 ? 2% 860.31 ? 2% 1736 ? 3% +506.1% 0.09 ? 4% -13.4% 10630744 ? 12% 74675547 ? 12% 26031282 ? 12% 105357 ? 2% -16.6% 5921091 11856660 5515701 ? 7% 59582345 4793162 ? 11% 1858672 ? 13% 5443 ? 8% -63.7% 4791488 ? 8% 53263 -18.3% 103557 4793162 ? 11% 260965 ? 3% -73.8% 210545 ? 4% -91.7% 8964679 1384 ? 19% 8648086 1033314 ? 14% 8998915 1704072 ? 2% 0.45 ? 8% +168.1% 2.07e+10 1.334e+08 8.65 ? 7% -4.0 7550889 ? 5% 18312903 0.99 +203.5% 9.998e+10 122169 ? 13% +148.0% 0.02 ? 3% -0.0 5554209 ? 3% 2.912e+10 0.00 ? 13% -0.0 316990 ? 13% -85.2% 1.752e+10 1.016e+11 1.02 -66.2% 0.45 -1.4% 299.03 ? 6% 300.64 -68.1% 4904 -27.1% 52.06 ? 27% +42.6 1131145 ? 23% 1002828 ? 32% -96.5% 4904 -27.1% 0.46 ? 8% +168.7% 16.33 ? 4% -7.0 0.98 +197.4% 13283 ? 6% +94.7% 0.02 ? 3% -0.0 0.00 ? 13% -0.0 1.02 -66.2% 53.52 ? 19% +40.4 3330788 2.063e+10 1.33e+08 7528642 ? 5% 18252001 9.965e+10 5544297 ? 3% 2.903e+10 316419 ? 13% -85.2% 1.746e+10 1.013e+11 4885 -26.8% 1127461 ? 23% 1001303 ? 32% -96.4% 4885 -26.8% 3.054e+13 34.69 -20.3 34.45 -20.1 31.23 ? 2% -18.2 20.47 ? 2% -12.3 18.88 -11.5 17.86 -10.9 17.75 -10.8 17.53 -10.6 19.27 ? 2% -10.6 16.91 ? 2% -10.3 16.38 -9.9 16.36 ? 2% -9.5 14.54 -8.7 15.76 -8.6 14.21 -8.5 14.11 -8.4 15.42 -8.4 13.97 -8.3 13.81 ? 2% -8.0 14.00 -7.6 11.27 ? 2% -7.5 10.99 ? 2% -7.3 12.19 -7.3 10.72 ? 2% -7.2 11.91 -7.1 13.13 -7.0 12.84 -6.9 11.43 ? 2% -6.8 12.56 -6.7 11.32 ? 2% -6.7 12.46 -6.7 9.69 ? 2% -5.8 10.64 -5.7 10.56 ? 2% -5.6 10.24 ? 2% -5.4 9.23 ? 2% -5.3 9.97 -5.3 14.74 -4.4 11.02 -2.5 10.67 -2.4 2.87 -2.1 5.01 -1.9 3.13 ? 2% -1.9 2.76 -1.7 2.92 ? 4% -1.5 2.07 ? 2% -1.3 1.96 ? 2% -1.2 1.60 ? 2% -1.0 1.30 -1.0 1.68 ? 8% -0.8 1.46 -0.8 1.37 ? 16% -0.5 0.44 ? 50% +0.6 0.43 ? 50% +0.6 0.00 +0.7 41.34 +1.9 38.47 +3.6 38.19 +3.8 37.53 +4.3 36.83 +4.7 35.95 +5.3 32.14 +7.7 31.80 +7.9 34.12 +10.2 34.07 +10.3 33.94 +10.3 35.09 +10.5 14.13 +17.8 13.80 +18.0 0.00 +24.3 0.00 +25.1 0.00 +25.3 0.00 +25.5 0.00 +25.9 0.00 +26.1 46.54 -27.0 45.22 -26.2 44.82 -26.0 37.76 -22.0 35.11 -20.4 34.91 -20.3 32.01 -18.6 25.16 ? 2% -14.5 22.28 ? 2% -13.4 21.63 ? 2% -11.9 19.92 ? 2% -10.9 19.32 ? 2% -10.6 16.44 -9.9 16.82 -9.8 14.58 -8.7 15.79 -8.6 14.23 -8.5 15.53 -8.4 14.13 -8.4 14.03 -8.4 14.25 -7.7 58.80 -7.5 12.22 -7.3 58.45 -7.3 13.24 -7.1 15.07 -4.6 6.83 -3.9 5.92 -3.5 4.58 -2.7 4.53 -2.6 11.21 -2.6 11.14 -2.5 3.96 -2.4 5.46 -2.2 3.78 -2.2 3.24 ? 2% -2.0 3.27 -2.0 3.31 ? 2% -1.9 3.20 -1.9 3.04 ? 4% -1.6 2.43 -1.5 2.32 -1.4 2.29 -1.4 2.32 -1.3 2.02 -1.2 2.04 ? 2% -1.2 1.83 -1.1 1.77 -0.9 1.72 -0.9 1.75 ? 8% -0.9 1.45 -0.9 1.26 ? 2% -0.8 1.20 ? 2% -0.7 1.35 -0.7 1.14 -0.7 1.03 -0.6 1.46 ? 10% -0.6 1.06 ? 2% -0.6 1.00 ? 2% -0.6 0.85 ? 2% -0.6 0.99 ? 2% -0.6 1.06 ? 7% -0.6 0.88 ? 2% -0.5 1.14 ? 13% -0.5 0.71 ? 2% -0.5 1.22 ? 13% -0.5 0.81 ? 2% -0.5 0.78 ? 4% -0.5 0.65 ? 2% -0.4 0.76 ? 2% -0.4 0.73 ? 8% -0.4 0.72 -0.4 0.68 ? 8% -0.4 0.66 ? 5% -0.4 0.72 -0.4 0.55 ? 2% -0.4 0.70 ? 6% -0.4 1.78 ? 12% -0.4 0.55 -0.3 0.62 -0.3 0.60 ? 8% -0.3 0.56 ? 5% -0.3 0.51 ? 3% -0.3 0.51 ? 3% -0.3 0.51 -0.3 0.53 -0.3 0.93 ? 14% -0.3 0.46 -0.3 0.48 ? 2% -0.3 0.53 ? 2% -0.3 0.92 ? 18% -0.3 0.47 ? 2% -0.3 0.54 ? 7% -0.3 0.45 -0.3 0.48 ? 2% -0.3 0.46 -0.3 0.44 ? 4% -0.3 0.35 ? 3% -0.3 0.42 -0.2 0.51 ? 2% -0.2 0.39 ? 2% -0.2 0.35 ? 3% -0.2 0.36 ? 6% -0.2 0.46 ? 2% -0.2 0.37 ? 2% -0.2 0.35 ? 5% -0.2 0.34 ? 3% -0.2 0.35 ? 2% -0.2 0.34 ? 5% -0.2 0.32 -0.2 0.30 ? 5% -0.2 0.31 ? 2% -0.2 0.36 ? 2% -0.2 0.28 ? 2% -0.2 0.28 -0.2 0.29 ? 5% -0.2 0.32 ? 3% -0.2 0.28 ? 3% -0.2 0.28 ? 4% -0.2 0.26 ? 2% -0.2 0.26 ? 2% -0.2 0.28 ? 2% -0.2 0.27 -0.2 0.27 ? 4% -0.2 0.37 ? 2% -0.2 0.25 ? 5% -0.2 0.65 ? 2% -0.2 0.24 ? 10% -0.1 0.24 ? 3% -0.1 0.24 ? 2% -0.1 0.25 -0.1 0.24 ? 4% -0.1 0.23 ? 6% -0.1 0.22 ? 6% -0.1 0.24 ? 3% -0.1 0.24 ? 2% -0.1 0.23 ? 3% -0.1 0.22 ? 4% -0.1 0.18 ? 2% -0.1 0.23 ? 3% -0.1 0.59 ? 15% -0.1 0.21 ? 6% -0.1 0.19 ? 7% -0.1 0.17 ? 2% -0.1 0.17 ? 3% -0.1 0.17 ? 4% -0.1 0.18 ? 4% -0.1 0.16 ? 2% -0.1 0.18 ? 2% -0.1 0.21 ? 3% -0.1 0.18 ? 4% -0.1 0.15 ? 2% -0.1 0.15 ? 6% -0.1 0.15 ? 2% -0.1 0.16 ? 4% -0.1 0.15 ? 2% -0.1 0.13 ? 5% -0.1 0.15 ? 4% -0.1 0.13 ? 4% -0.1 0.16 ? 4% -0.1 0.16 ? 4% -0.1 0.13 ? 8% -0.1 0.12 ? 8% -0.1 0.12 ? 14% -0.1 0.34 ? 4% -0.1 0.12 ? 6% -0.1 0.14 ? 7% -0.1 0.19 ? 14% -0.1 0.11 ? 5% -0.1 0.20 ? 15% -0.1 0.12 ? 3% -0.1 0.14 ? 2% -0.1 0.17 ? 12% -0.1 0.13 ? 5% -0.1 0.13 ? 5% -0.1 0.13 ? 3% -0.1 0.12 ? 4% -0.1 0.11 ? 4% -0.1 0.20 ? 16% -0.1 0.16 ? 12% -0.1 0.13 ? 4% -0.1 0.16 ? 16% -0.1 0.20 ? 4% -0.1 0.14 ? 16% -0.1 0.09 ? 15% -0.1 0.10 ? 6% -0.0 0.11 ? 19% -0.0 0.12 ? 16% -0.0 0.12 ? 16% -0.0 0.10 ? 16% -0.0 0.13 ? 3% -0.0 0.05 ? 50% +0.0 0.10 ? 14% +0.0 0.10 ? 16% +0.0 0.10 ? 16% +0.0 0.04 ? 50% +0.0 0.06 ? 8% +0.1 0.06 ? 6% +0.1 0.00 +0.1 0.08 ? 10% +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.2 0.00 +0.2 0.22 ? 2% +0.5 41.73 +1.7 37.62 +4.2 36.89 +4.7 36.13 +5.2 32.16 +7.7 34.12 +10.2 35.09 +10.5 35.09 +10.5 34.97 +10.6 14.68 +18.3 46.43 +26.2 0.00 +49.6 0.81 ? 2% +50.9 0.58 ? 2% +50.9 10.88 ? 2% -6.3 6.51 -3.7 3.68 -2.1 2.97 ? 2% -1.7 4.42 ? 2% -1.4 2.09 -1.3 1.91 -1.1 2.28 ? 10% -1.0 1.58 -0.9 1.50 ? 2% -0.9 1.19 ? 2% -0.7 1.21 -0.7 1.29 ? 2% -0.7 1.14 ? 3% -0.7 1.01 ? 2% -0.6 1.08 -0.6 1.43 ? 11% -0.6 0.91 ? 2% -0.5 1.47 ? 15% -0.5 0.96 -0.5 0.74 ? 2% -0.5 0.73 ? 3% -0.4 0.75 ? 3% -0.4 0.65 ? 2% -0.4 0.72 ? 2% -0.4 0.70 -0.4 0.74 ? 2% -0.4 0.71 ? 2% -0.4 0.72 ? 2% -0.4 0.61 ? 2% -0.4 0.60 -0.3 0.62 ? 5% -0.3 0.58 ? 2% -0.3 0.58 ? 9% -0.3 0.63 -0.3 0.51 ? 3% -0.3 0.50 ? 3% -0.3 0.51 ? 2% -0.3 0.50 ? 3% -0.3 0.49 ? 2% -0.3 0.50 -0.3 0.49 ? 5% -0.3 0.49 -0.3 0.49 ? 3% -0.3 0.44 ? 2% -0.3 0.44 -0.3 0.47 ? 2% -0.3 0.37 ? 3% -0.2 0.43 ? 2% -0.2 0.39 ? 2% -0.2 0.37 ? 4% -0.2 0.44 ? 2% -0.2 0.41 ? 6% -0.2 0.34 ? 7% -0.2 0.35 ? 2% -0.2 0.31 ? 5% -0.2 0.32 -0.2 0.29 ? 6% -0.2 0.32 ? 5% -0.2 0.27 ? 3% -0.2 0.30 ? 2% -0.2 0.29 ? 2% -0.2 0.31 ? 3% -0.2 0.31 ? 4% -0.2 0.27 ? 3% -0.2 0.27 ? 4% -0.2 0.27 ? 5% -0.2 0.29 ? 3% -0.2 0.29 ? 4% -0.2 0.25 ? 3% -0.2 0.26 ? 3% -0.2 0.26 ? 2% -0.2 0.36 ? 3% -0.2 0.24 -0.2 0.34 ? 2% -0.2 0.23 ? 3% -0.1 0.25 ? 2% -0.1 0.29 -0.1 0.23 ? 4% -0.1 0.63 ? 2% -0.1 0.22 ? 2% -0.1 0.18 ? 2% -0.1 0.22 ? 11% -0.1 0.28 ? 22% -0.1 0.22 -0.1 0.23 ? 2% -0.1 0.54 ? 3% -0.1 0.20 ? 2% -0.1 0.21 ? 6% -0.1 0.22 ? 2% -0.1 0.20 ? 3% -0.1 0.20 ? 3% -0.1 0.21 ? 3% -0.1 0.22 ? 2% -0.1 0.20 ? 2% -0.1 0.20 ? 2% -0.1 0.19 ? 5% -0.1 0.18 ? 2% -0.1 0.19 ? 8% -0.1 0.18 ? 2% -0.1 0.17 ? 2% -0.1 0.16 ? 3% -0.1 0.23 ? 3% -0.1 0.30 ? 2% -0.1 0.18 ? 2% -0.1 0.13 ? 3% -0.1 0.15 ? 3% -0.1 0.15 ? 3% -0.1 0.16 ? 5% -0.1 0.15 ? 3% -0.1 0.17 ? 2% -0.1 0.17 ? 5% -0.1 0.15 ? 3% -0.1 0.15 ? 7% -0.1 0.13 ? 7% -0.1 0.13 ? 3% -0.1 0.15 ? 6% -0.1 0.15 ? 5% -0.1 0.16 -0.1 0.15 ? 5% -0.1 0.34 ? 4% -0.1 0.14 ? 5% -0.1 0.12 ? 3% -0.1 0.11 ? 3% -0.1 0.14 ? 5% -0.1 0.12 ? 4% -0.1 0.12 ? 15% -0.1 0.13 ? 3% -0.1 0.13 ? 3% -0.1 0.14 ? 2% -0.1 0.13 ? 7% -0.1 0.12 ? 4% -0.1 0.13 ? 5% -0.1 0.11 ? 3% -0.1 0.19 ? 17% -0.1 0.13 ? 15% -0.1 0.20 ? 5% -0.1 0.10 ? 4% -0.0 0.01 ?200% +0.1 0.00 +0.2 0.21 ? 2% +0.5 0.54 +1.3 0.00 +49.6

************************** lkp-skl-fpga01: 104 ========================== compiler/cpufreq_governor/ gcc-11/performance/x86_6
commit:
99babd04b2 ("selftests/rseq: af7f588d8f ("sched:
99babd04b2505471 ---------------- fail:runs %reproduction | | 1:6 -17% 1:6 -17% 1:6 -17% %stddev %change \ | 4324658 ? 3% 270290 ? 3% 4324658 ? 3% 1.329e+09 ? 3% 12.36 ? 2% +2.9 5.59 -3.0 19.17 -11.3% 8547612 ? 2% 233179 158734 ? 25% +182.8% 0.14 ? 5% -48.8% 70953881 1.286e+09 ? 3% 1.67 ? 6% -0.7 252.48 114167 ? 20% -93.5% 1525472 387653 ? 11% -87.1% 0.39 ? 6% -24.3% 22932 ? 5% +13.0% 59500 ? 4% +55.5% 114167 ? 20% -93.5% 1525472 387653 ? 11% -87.1% 671631 ? 3% 1542612 708750 0.37 ? 6% -24.7% -848558 -1486548 708763 214.42 3058 ? 4% +113.4% 1063 ? 2% 0.20 ? 4% -13.3% 12311145 ? 3% 40179926 ? 3% 18457770 ? 3% 1.48 ? 33% +393.8% 9.306e+09 1.571e+08 ? 2% 67978805 ? 32% 8635700 ? 3% 2.00 +112.1% 152.95 ? 4% 9807863 ? 8% 1.313e+10 7.867e+09 9070094 ? 3% 11272936 ? 7% 4.592e+10 0.50 -51.9% 660.44 ? 31% -85.8% 291.58 1.48 ? 32% +375.8% 1.99 +108.2% 0.50 -51.9% 3197936 ? 3% 9.276e+09 1.566e+08 ? 2% 67767931 ? 32% 8606808 ? 3% 152.61 ? 4% 9780257 ? 8% 1.309e+10 7.842e+09 9039813 ? 3% 11235261 ? 7% 4.577e+10 1.381e+13 20.96 ? 7% -9.2 15.02 ? 20% -4.7 5.46 ? 28% -2.9 4.64 ? 32% -2.4 4.55 ? 28% -2.4 3.70 ? 37% -1.9 3.55 ? 39% -1.8 2.78 ? 33% -1.5 2.63 ? 39% -1.4 2.60 ? 32% -1.3 2.27 ? 20% -1.2 2.29 ? 35% -1.1 2.08 ? 32% -1.1 1.74 ? 34% -0.9 1.25 ? 45% -0.9 1.18 ? 29% -0.9 1.07 ? 34% -0.8 1.06 ? 38% -0.8 1.03 ? 21% -0.7 1.38 ? 23% -0.7 0.00 +0.8 40.47 ? 4% +2.9 34.36 ? 8% +6.8 34.34 ? 8% +6.8 34.08 ? 8% +7.0 35.35 ? 8% +7.0 32.51 ? 14% +7.7 30.86 ? 16% +8.5 27.80 ? 22% +10.0 27.05 ? 23% +10.4 25.85 ? 26% +11.0 21.15 ? 38% +13.5 20.97 ? 39% +13.6 9.89 ? 37% +16.4 9.57 ? 39% +16.5 0.00 +18.8 0.00 +19.3 0.29 ?100% +19.4 0.00 +19.8 0.00 +20.3 0.29 ?100% +20.4 22.18 ? 7% -9.1 15.50 ? 20% -4.9 5.65 ? 29% -3.0 5.48 ? 35% -2.8 5.08 ? 26% -2.5 4.74 ? 28% -2.5 4.94 ? 33% -2.4 4.06 ? 30% -1.9 3.89 ? 32% -1.8 3.16 ? 21% -1.6 3.19 ? 34% -1.4 2.62 ? 18% -1.4 2.41 ? 35% -1.3 2.17 ? 32% -1.2 1.76 ? 42% -1.1 1.56 ? 48% -1.1 1.97 ? 34% -1.0 1.84 ? 34% -1.0 1.54 ? 23% -0.8 0.98 ? 51% -0.7 1.26 ? 30% -0.7 1.15 ? 33% -0.6 1.40 ? 10% -0.6 1.17 ? 34% -0.6 1.06 ? 32% -0.5 0.93 ? 26% -0.5 1.04 ? 21% -0.5 1.18 ? 12% -0.5 0.92 ? 34% -0.5 0.88 ? 36% -0.4 0.85 ? 36% -0.4 0.88 ? 34% -0.4 0.78 ? 23% -0.4 0.75 ? 38% -0.4 0.76 ? 31% -0.4 0.65 ? 39% -0.4 0.71 ? 42% -0.4 0.75 ? 35% -0.4 0.80 ? 22% -0.4 0.64 ? 34% -0.4 0.71 ? 18% -0.4 0.67 ? 38% -0.3 0.75 ? 29% -0.3 0.64 ? 40% -0.3 0.66 ? 22% -0.3 0.65 ? 20% -0.3 0.44 ? 42% -0.3 0.52 ? 31% -0.3 0.47 ? 18% -0.3 0.55 ? 24% -0.3 0.47 ? 29% -0.3 0.54 ? 5% -0.3 0.47 ? 10% -0.3 0.45 ? 35% -0.2 0.43 ? 23% -0.2 0.36 ? 37% -0.2 0.37 ? 42% -0.2 0.35 ? 26% -0.2 0.43 ? 6% -0.2 0.35 ? 45% -0.2 0.36 ? 27% -0.2 0.53 ? 16% -0.2 0.39 ? 35% -0.2 0.38 ? 37% -0.2 0.28 ? 52% -0.2 0.32 ? 30% -0.2 0.34 ? 15% -0.2 0.34 ? 8% -0.2 0.33 ? 33% -0.2 0.31 ? 38% -0.2 0.28 ? 30% -0.2 0.29 ? 31% -0.2 0.30 ? 26% -0.2 0.27 ? 34% -0.1 0.25 ? 39% -0.1 0.28 ? 30% -0.1 0.28 ? 9% -0.1 0.24 ? 32% -0.1 0.22 ? 29% -0.1 0.15 ? 17% -0.1 0.22 ? 27% -0.1 0.23 ? 28% -0.1 0.21 ? 26% -0.1 0.20 ? 38% -0.1 0.25 ? 28% -0.1 0.18 ? 18% -0.1 0.22 ? 13% -0.1 0.19 ? 36% -0.1 0.19 ? 29% -0.1 0.16 ? 21% -0.1 0.20 ? 14% -0.1 0.19 ? 31% -0.1 0.19 ? 21% -0.1 0.20 ? 8% -0.1 0.16 ? 9% -0.1 0.14 ? 9% -0.1 0.20 ? 25% -0.1 0.16 ? 25% -0.1 0.17 ? 22% -0.1 0.18 ? 34% -0.1 0.18 ? 9% -0.1 0.19 ? 20% -0.1 0.17 ? 39% -0.1 0.13 ? 11% -0.1 0.18 ? 15% -0.1 0.12 ? 44% -0.1 0.11 ? 9% -0.1 0.16 ? 29% -0.1 0.14 ? 14% -0.1 0.18 ? 26% -0.1 0.10 ? 23% -0.1 0.14 ? 24% -0.1 0.13 ? 23% -0.1 0.13 ? 38% -0.1 0.08 ? 13% -0.1 0.12 ? 30% -0.1 0.08 ? 10% -0.0 0.10 ? 19% -0.0 0.00 +0.1 0.28 ? 23% +0.6 40.66 ? 4% +2.8 34.36 ? 8% +6.8 35.35 ? 8% +7.0 35.35 ? 8% +7.0 35.24 ? 8% +7.0 27.86 ? 22% +10.0 27.09 ? 23% +10.4 25.98 ? 26% +10.9 21.16 ? 38% +13.5 10.22 ? 36% +16.9 31.01 ? 38% +30.5 0.00 +38.8 0.66 ? 29% +39.5 1.00 ? 18% +39.6 5.48 ? 35% -2.8 4.56 ? 33% -2.2 3.36 ? 34% -1.6 2.10 ? 36% -1.1 1.75 ? 42% -1.1 1.94 ? 28% -1.0 1.42 ? 47% -0.9 2.45 ? 29% -0.8 0.97 ? 51% -0.7 1.21 ? 33% -0.6 1.14 ? 33% -0.6 1.11 ? 15% -0.6 1.01 ? 31% -0.5 1.15 ? 12% -0.5 0.95 ? 36% -0.5 0.81 ? 34% -0.4 0.73 ? 17% -0.4 0.84 ? 24% -0.4 0.80 ? 41% -0.4 0.75 ? 22% -0.4 0.70 ? 27% -0.4 0.64 ? 13% -0.4 0.67 ? 34% -0.3 0.65 ? 18% -0.3 0.66 ? 38% -0.3 0.63 ? 40% -0.3 0.71 ? 27% -0.3 0.55 ? 20% -0.3 0.53 ? 25% -0.3 0.44 ? 42% -0.3 0.52 ? 31% -0.3 0.45 ? 32% -0.3 0.48 ? 37% -0.2 0.47 ? 29% -0.2 0.44 ? 35% -0.2 0.42 ? 18% -0.2 0.47 ? 30% -0.2 0.45 ? 29% -0.2 0.51 ? 9% -0.2 0.48 ? 23% -0.2 0.42 ? 34% -0.2 0.43 ? 33% -0.2 0.36 ? 43% -0.2 0.34 ? 24% -0.2 0.28 ? 52% -0.2 0.30 ? 36% -0.2 0.30 ? 30% -0.2 0.34 ? 37% -0.2 0.28 ? 34% -0.2 0.30 ? 7% -0.2 0.29 ? 46% -0.2 0.28 ? 37% -0.2 0.28 ? 30% -0.1 0.24 ? 32% -0.1 0.27 ? 7% -0.1 0.29 ? 22% -0.1 0.30 ? 16% -0.1 0.22 ? 20% -0.1 0.28 ? 29% -0.1 0.24 ? 35% -0.1 0.24 ? 39% -0.1 0.26 ? 30% -0.1 0.27 ? 23% -0.1 0.23 ? 17% -0.1 0.40 ? 12% -0.1 0.21 ? 36% -0.1 0.15 ? 18% -0.1 0.17 ? 35% -0.1 0.20 ? 32% -0.1 0.18 ? 7% -0.1 0.20 ? 15% -0.1 0.18 ? 37% -0.1 0.19 ? 35% -0.1 0.16 ? 23% -0.1 0.18 ? 39% -0.1 0.22 ? 24% -0.1 0.16 ? 15% -0.1 0.16 ? 20% -0.1 0.19 ? 29% -0.1 0.14 ? 9% -0.1 0.18 ? 31% -0.1 0.18 ? 21% -0.1 0.18 ? 23% -0.1 0.22 ? 26% -0.1 0.12 ? 8% -0.1 0.18 ? 32% -0.1 0.18 ? 9% -0.1 0.19 ? 8% -0.1 0.19 ? 32% -0.1 0.17 ? 33% -0.1 0.15 ? 31% -0.1 0.20 ? 2% -0.1 0.16 ? 27% -0.1 0.17 ? 19% -0.1 0.15 ? 30% -0.1 0.16 ? 37% -0.1 0.14 ? 11% -0.1 0.17 ? 26% -0.1 0.17 ? 20% -0.1 0.16 ? 22% -0.1 0.17 ? 16% -0.1 0.12 ? 31% -0.1 0.18 ? 10% -0.1 0.14 ? 43% -0.1 0.12 ? 15% -0.1 0.11 ? 9% -0.1 0.16 ? 16% -0.1 0.16 ? 35% -0.1 0.14 ? 19% -0.1 0.15 ? 30% -0.1 0.10 ? 23% -0.1 0.15 ? 8% -0.1 0.14 ? 34% -0.1 0.12 ? 21% -0.1 0.12 ? 38% -0.1 0.08 ? 26% -0.1 0.14 ? 8% -0.1 0.12 ? 28% -0.1 0.10 ? 29% -0.1 0.10 ? 21% -0.0 0.08 ? 10% -0.0 0.00 +0.1 0.28 ? 23% +0.6 0.65 ? 30% +0.8 0.00 +38.8

************************** lkp-skl-fpga01: 104 ========================== compiler/cpufreq_governor/ gcc-11/performance/x86_6
commit:
99babd04b2 ("selftests/rseq: af7f588d8f ("sched:
99babd04b2505471 ---------------- %stddev %change \ | 8413415 ? 2% 22.36 -83.3% 161796 ? 2% -85.0% 8413415 ? 2% 10216 ? 2% -40.9% 2.233e+09 2.46e+09 ? 2% 329714 ? 11% 374570 ? 9% 4491204 ? 2% 4540680 ? 2% 22.83 -18.4 1.44 +0.2 0.12 ? 2% -0.0 59.94 +30.2 15.67 -12.0 22.83 -78.1% 60.33 +50.6% 15.00 -80.0% 14294773 ? 2% 59.50 -12.3% 16450432 ? 2% 276754 1431257 ? 8% 0.02 ? 17% +0.3 0.77 ? 6% +31.1% 53.67 ? 2% -14.6% 0.09 ? 4% -78.2% 84431937 6.83 ? 3% -6.8 2.458e+09 ? 2% 6.16 -5.5 53.50 ? 2% -13.1% 365.46 92948 +1761.8% 92857 +1763.6% 475097 14131358 ? 2% 12167680 ? 2% 11089942 ? 2% 11089749 ? 2% 146766 5019103 ? 3% 16059430 ? 2% 14911 ? 2% -47.7% 146766 10707913 ? 2% 16146332 ? 2% 382.33 ? 13% +30777.3% 44471 ? 32% +315.1% 44053 +211.7% 1598 ? 2% +15438.7% 382.33 ? 13% +30777.3% 44471 ? 32% +315.1% 374056 ? 9% 329200 ? 11% 22812 +1275.5% 75507 ? 19% -55.4% 2702384 ? 2% 2727949 ? 2% 1210795 ? 3% 3083 ? 3% -61.4% 2675300 ? 2% 15383 ? 3% -31.4% 22812 +1275.5% 2727947 ? 2% 4540436 ? 2% 4490961 ? 3% 1577 ? 15% +29859.5% 1532 ? 13% +30744.5% 177949 ? 32% +315.0% 177854 ? 32% +315.2% 176243 +211.6% 6399 ? 2% +15435.9% 94498 ? 6% +1225.9% 94453 ? 6% +1226.4% 155852 ? 34% -68.9% 301933 ? 19% 376400 ? 15% 10818908 ? 2% 10917966 ? 2% 10917868 ? 2% 61560 ? 3% -31.5% 4856202 ? 3% 11816207 ? 2% 12364 ? 2% -61.6% 61560 ? 3% -31.5% 10710570 ? 2% 147398 23418 +1746.6% 118759 -26.0% 4488037 8987049 3533819 ? 2% 45164003 2773197 ? 2% 1256312 ? 3% 3731 ? 2% -48.1% 2677956 ? 2% 36695 -11.5% 23418 +1746.6% 2773197 ? 2% 624422 ? 17% 539128 ? 21% 4916730 ? 2% 2036 ? 18% 4822398 ? 2% 1802325 ? 4% 83499 ?162% +1435.7% 5055756 ? 2% 1857025 ? 5% 742457 ? 9% 39235 308870 ? 19% -99.3% 4280681 1073798 ? 9% -97.9% 0.68 ? 2% +13.8% 0.42 ? 2% -21.5% 0.67 ? 45% +183.3% 308870 ? 19% -99.3% 4280681 1073798 ? 9% -97.9% 4201184 4494117 4124946 0.68 +12.2% 0.39 ? 3% -20.5% 451.34 1319 -11.5% 154.04 447.52 135.46 230.77 ? 3% 940.25 ? 3% 225.80 148237 ? 2% 4546 ? 13% +579.3% 5.75 +33.2% 2742 ? 3% -10.0% 6610 ? 23% -31.1% 0.00 ? 5% +42.1% 0.52 ? 4% -11.0% 23781980 ? 2% 26139865 ? 3% 20830845 ? 5% 1902082 ? 27% 1.76 ? 17% +206.4% 1.904e+10 1.64 -0.5 3.056e+08 ? 2% 20.15 ? 13% +6.3 16636101 ? 2% 3.12 ? 2% +296.6% 1861 -33.6% 0.08 ? 5% -0.0 20981469 ? 5% 2.678e+10 0.00 ? 18% +0.0 233103 ? 18% -67.8% 1.604e+10 11175689 ? 3% 23053320 ? 6% 9.415e+10 8923 ? 2% +47.9% 0.33 -74.4% 596.29 5512 ? 6% -29.0% 56.02 +41.6 1970708 ? 4% 2639467 ? 5% 85.83 +13.0 1884736 ? 3% 357304 ? 3% -74.0% 5512 ? 6% -29.0% 1.74 ? 16% +210.5% 1.60 -0.5 19.62 ? 13% +6.0 3.06 +290.5% 0.08 ? 5% -0.0 0.00 ? 18% +0.0 8429 ? 2% +50.4% 0.33 -74.4% 42.42 ? 2% +54.4 84.10 +14.0 3374614 1.898e+10 3.046e+08 ? 2% 16582299 ? 2% 1861 -33.8% 20900401 ? 5% 2.669e+10 231656 ? 18% -67.6% 1.599e+10 11141515 ? 3% 22975410 ? 6% 9.385e+10 5511 ? 6% -29.2% 1964291 ? 4% 2667620 ? 5% 1878586 ? 3% 354848 ? 3% -73.9% 5512 ? 6% -29.2% 2.839e+13 26.70 -21.8 26.52 -21.6 24.81 -20.3 19.22 -15.2 16.54 -13.1 15.23 -12.4 15.46 -12.2 13.72 -11.5 13.79 -10.7 13.58 -10.7 11.44 ? 2% -10.3 12.38 -9.7 12.12 -9.5 9.83 ? 2% -9.2 11.66 -9.2 9.47 ? 3% -9.0 10.60 -8.2 10.31 -8.1 10.31 -8.0 10.23 -7.9 10.04 -7.8 9.04 -7.6 9.54 -7.5 8.81 -7.4 9.34 -7.4 7.56 -7.2 8.54 -7.2 9.07 -7.1 9.00 -7.1 8.84 -7.0 8.29 -6.6 8.19 -6.5 7.98 -6.3 7.59 -6.0 7.54 -5.9 7.28 -5.7 7.08 -5.5 6.89 -5.5 6.85 -5.4 5.32 -4.0 5.26 -4.0 4.42 -3.7 52.88 -3.6 2.85 ? 2% -2.3 2.38 -2.1 0.75 ? 14% -0.5 0.75 ? 14% -0.5 0.75 ? 14% -0.5 0.75 ? 14% -0.4 0.76 ? 14% -0.4 0.76 ? 14% -0.4 0.73 ? 6% +0.7 0.00 +0.8 0.60 ? 6% +0.8 0.00 +0.9 0.00 +0.9 0.00 +1.1 45.74 +2.2 45.24 +2.6 44.49 +3.2 41.00 +6.1 40.89 +6.2 25.48 +19.7 25.46 +19.7 25.73 +19.8 25.30 +19.8 11.35 +31.1 11.18 +31.2 0.00 +39.4 0.00 +39.6 0.00 +39.7 0.00 +40.2 0.00 +40.4 0.00 +40.5 33.56 -26.7 32.38 -25.7 32.13 -25.5 29.34 -23.8 27.00 -21.9 26.82 -21.7 25.26 -20.5 20.14 -15.5 18.18 -14.7 66.41 -13.7 16.42 -13.4 64.30 -12.1 14.42 -11.0 13.86 -10.5 12.41 -9.7 10.68 ? 2% -9.7 11.71 ? 2% -9.6 12.13 -9.6 11.72 -9.3 10.62 -8.2 10.68 -8.2 10.32 -8.0 10.25 -7.9 10.06 -7.8 9.79 -7.5 8.99 -7.2 8.19 -6.5 5.37 -4.1 5.33 -4.1 4.52 -3.8 52.99 -3.6 3.85 -3.1 3.84 -3.1 3.53 -2.8 3.18 -2.5 3.05 -2.4 2.92 -2.4 2.89 ? 3% -2.3 2.66 ? 2% -2.2 2.39 -1.9 2.20 ? 5% -1.9 2.62 -1.8 2.19 -1.8 2.50 ? 2% -1.7 2.04 ? 2% -1.4 1.58 ? 3% -1.1 1.42 -1.1 1.48 ? 2% -1.1 1.30 ? 3% -1.1 1.10 ? 3% -0.9 0.99 -0.9 1.02 ? 4% -0.9 0.93 ? 2% -0.8 0.93 ? 3% -0.8 1.02 ? 2% -0.8 0.91 ? 2% -0.7 0.82 ? 6% -0.7 0.86 -0.7 0.80 ? 4% -0.7 0.78 ? 2% -0.7 0.79 ? 4% -0.6 0.74 ? 3% -0.6 0.71 ? 6% -0.6 0.70 ? 3% -0.6 1.44 ? 4% -0.6 2.24 ? 4% -0.5 0.72 ? 2% -0.5 0.58 ? 4% -0.5 0.62 ? 2% -0.5 0.59 ? 11% -0.5 1.17 ? 3% -0.5 0.57 ? 4% -0.4 0.54 ? 4% -0.4 0.47 ? 8% -0.4 0.94 ? 4% -0.4 0.52 ? 6% -0.4 0.47 ? 6% -0.4 0.47 ? 14% -0.4 0.45 ? 4% -0.4 0.52 ? 3% -0.4 0.47 -0.4 0.46 ? 10% -0.4 0.44 -0.4 0.45 ? 4% -0.4 0.45 ? 3% -0.3 0.41 ? 12% -0.3 0.41 ? 5% -0.3 0.81 ? 3% -0.3 0.41 ? 2% -0.3 0.38 ? 3% -0.3 0.38 ? 18% -0.3 0.80 ? 17% -0.3 0.38 ? 2% -0.3 0.80 ? 17% -0.3 0.42 -0.3 0.81 ? 17% -0.3 0.44 ? 9% -0.3 0.41 ? 3% -0.3 0.35 ? 6% -0.3 0.30 ? 3% -0.3 0.30 ? 22% -0.3 0.35 ? 7% -0.3 0.32 ? 5% -0.3 0.32 ? 24% -0.3 0.33 ? 3% -0.3 0.29 ? 8% -0.3 0.32 ? 7% -0.3 0.40 ? 11% -0.3 0.28 ? 9% -0.3 0.32 ? 3% -0.2 0.26 ? 2% -0.2 0.42 ? 3% -0.2 0.27 ? 2% -0.2 0.27 ? 4% -0.2 0.26 ? 3% -0.2 0.25 ? 5% -0.2 0.22 ? 7% -0.2 0.22 ? 3% -0.2 0.68 ? 3% -0.2 0.81 ? 16% -0.2 0.81 ? 16% -0.2 0.60 ? 4% -0.2 0.24 ? 5% -0.2 0.30 ? 17% -0.2 0.19 ? 7% -0.2 0.34 ? 18% -0.1 0.33 ? 18% -0.1 0.47 ? 4% -0.1 0.46 ? 4% -0.1 0.14 ? 6% -0.1 0.15 ? 5% -0.1 0.17 ? 4% -0.1 0.12 ? 12% -0.1 0.19 ? 12% -0.1 0.18 ? 11% -0.1 0.12 ? 16% -0.1 0.13 ? 20% -0.1 0.31 ? 3% -0.1 0.09 ? 21% -0.1 0.10 ? 19% -0.1 0.26 ? 4% -0.0 0.10 ? 5% -0.0 0.23 ? 5% -0.0 0.23 ? 5% -0.0 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.00 +0.2 0.09 ? 14% +0.3 0.09 ? 11% +0.3 0.00 +0.8 0.61 ? 5% +0.8 0.00 +0.9 0.00 +0.9 0.00 +1.1 45.78 +2.2 45.26 +2.6 44.56 +3.1 41.02 +6.2 25.48 +19.7 25.73 +19.8 25.73 +19.8 25.66 +19.9 11.49 +31.3 52.29 +37.6 0.63 ? 2% +79.9 0.00 +80.0 0.30 +80.3 9.60 ? 3% -9.2 8.30 ? 2% -7.8 9.18 ? 2% -7.5 4.01 -3.3 3.81 -3.1 3.14 ? 2% -2.5 3.18 -2.5 2.39 -2.0 2.24 -1.8 2.16 -1.7 1.91 -1.5 1.30 -1.1 1.23 -1.0 1.14 -0.9 0.99 ? 5% -0.8 0.96 ? 2% -0.8 0.89 -0.7 0.89 ? 2% -0.7 0.81 ? 6% -0.7 0.85 ? 5% -0.7 0.78 ? 2% -0.6 0.73 ? 6% -0.6 0.73 ? 2% -0.6 0.72 ? 4% -0.6 0.67 ? 7% -0.6 0.66 ? 4% -0.6 0.70 ? 4% -0.5 0.55 ? 3% -0.5 0.58 ? 6% -0.5 0.54 ? 4% -0.4 0.52 ? 7% -0.4 0.45 ? 11% -0.4 0.46 -0.4 0.45 ? 5% -0.4 0.46 ? 2% -0.4 0.42 ? 7% -0.4 0.40 ? 7% -0.3 0.37 ? 4% -0.3 0.44 ? 5% -0.3 0.40 ? 11% -0.3 0.44 ? 4% -0.3 0.79 ? 3% -0.3 0.40 ? 3% -0.3 0.40 ? 3% -0.3 0.37 ? 4% -0.3 0.37 ? 3% -0.3 0.43 ? 4% -0.3 0.39 ? 4% -0.3 0.35 ? 2% -0.3 0.34 ? 6% -0.3 0.34 ? 2% -0.3 0.32 ? 6% -0.3 0.32 ? 2% -0.3 0.31 ? 6% -0.3 0.33 ? 8% -0.3 0.31 ? 4% -0.3 0.31 ? 6% -0.3 0.28 ? 26% -0.2 0.31 ? 2% -0.2 0.38 ? 11% -0.2 0.29 ? 2% -0.2 0.28 ? 6% -0.2 0.27 ? 2% -0.2 0.26 ? 6% -0.2 0.38 ? 2% -0.2 0.24 ? 3% -0.2 0.24 ? 6% -0.2 0.22 ? 4% -0.2 0.22 ? 5% -0.2 0.24 ? 9% -0.2 0.29 ? 5% -0.2 0.27 ? 5% -0.2 0.33 ? 6% -0.1 0.18 ? 4% -0.1 0.14 ? 6% -0.1 0.23 ? 9% -0.1 0.17 ? 10% -0.1 0.00 +0.1 0.00 +0.1 0.00 +0.1 0.30 +0.3 0.09 ? 14% +0.3 0.00 +0.6 0.00 +80.0

************************** lkp-csl-2sp3: 96 ========================== compiler/cpufreq_governor/ gcc-11/performance/x86_6
commit:
99babd04b2 ("selftests/rseq: af7f588d8f ("sched:
99babd04b2505471 ---------------- %stddev %change \ | 210861 20242720 55019520 1.9e+08 2259 +7.5% 4300 +19.2% 2642 ? 2% -11.8% 2.705e+08 6.075e+09 14.58 +3.3 11802313 ? 2% 11868695 ? 2% 31341412 1739822 198273 29152003 10192230 ? 8% 27542178 27541034 26690232 57049 27457190 910.33 +15.6% 29.46 +4.6 1.992e+08 ? 2% 44.78 -5.7 56047 ? 92% +1006.4% 0.24 -20.8% 62645660 225.94 39.08 248641 ? 19% 17517837 114473 ? 2% -35.4% 18308067 114473 ? 2% -35.4% 224848 ? 2% 3738197 13816698 ? 3% 13816181 ? 3% 13327964 ? 3% 28651 ? 2% -13.3% 13689339 ? 2% 26826 ?122% +9309.0% 4378477 28614 ? 2% -35.4% 934549 934549 3452954 ? 3% 3330866 ? 3% 7166 ? 2% -13.3% 3421209 ? 2% 6706 ?122% +9309.8% 3452953 ? 3% 6706 ?122% +9309.8% 11868617 ? 2% 11802234 ? 2% 61039 2454405 4914813 7805675 24733767 6885305 6672510 14262 6864084 44375 61039 6885305 24681264 24592528 24808648 1.912e+08 24814673 6448 ? 45% +115.5% 531865 ? 56% 57529 ? 51% 0.22 ? 6% +11.1% 10123 ? 6% +15.6% 365.17 ? 36% 6448 ? 45% +115.5% 531865 ? 56% 57529 ? 51% 782070 821150 753697 0.22 ? 5% +11.4% -5981 -259.2% 1179 +11.2% 0.22 +13.5% 18886 ? 16% +41.6% 4.18 +8.6% 2.29e+10 0.19 +0.0 40178596 21.94 +1.8 66452612 1760358 1.20 +24.5% 8.839e+10 6205 +7.3% 1330 +5.9% 4572848 2.122e+10 0.02 -0.0 702307 3.917e+09 4714448 10445194 7.353e+10 0.84 -19.4% 0.92 +15.2% 473.64 503.52 620975 19518092 1039427 10095749 620979 4.22 +7.5% 0.18 +0.0 21.43 +2.1 1.20 +24.6% 1330 +5.8% 0.02 -0.0 0.02 -0.0 15600 0.83 -19.7% 3724 +1.6% 2.282e+10 40052157 66240006 1754492 8.81e+10 6191 +7.3% 4557705 2.115e+10 699965 3.904e+09 4698565 10410202 7.329e+10 618900 19458339 1036377 10063664 618903 2.263e+13 23.77 -3.8 17.03 -3.4 27.40 -2.9 47.87 -2.6 46.90 -2.5 47.06 -2.4 12.10 -1.7 12.01 -1.7 11.80 -1.6 10.69 -0.8 0.94 -0.6 8.46 -0.5 8.37 -0.5 4.22 ? 5% -0.4 1.20 -0.3 1.57 -0.3 0.97 -0.2 0.76 -0.2 1.03 -0.2 0.86 -0.2 0.76 ? 2% -0.2 0.80 -0.1 0.61 -0.1 0.97 -0.0 0.58 ? 2% -0.0 0.57 ? 2% -0.0 14.92 +0.9 14.92 +0.9 30.86 +1.4 29.74 +1.5 29.59 +1.5 28.77 +1.5 28.76 +1.5 30.41 +1.5 30.42 +1.5 7.72 ? 4% +1.9 50.46 +2.1 50.47 +2.1 50.45 +2.1 50.93 +2.2 4.41 +4.3 4.39 +4.4 0.00 +4.4 4.28 +4.4 3.86 +4.5 3.39 +4.6 0.00 +4.7 0.00 +4.7 0.00 +4.9 0.00 +5.1 0.66 +5.1 0.62 +5.1 1.77 +5.1 1.78 +5.1 1.74 +5.1 0.00 +5.2 14.97 -3.1 27.65 -2.8 48.31 -2.6 47.33 -2.4 47.32 -2.4 42.69 -1.8 12.10 -1.7 10.69 -0.8 4.27 ? 4% -0.4 1.02 -0.4 2.18 -0.3 1.59 -0.3 39.78 -0.2 1.73 -0.2 1.35 ? 2% -0.2 1.53 -0.2 0.76 ? 2% -0.2 1.05 -0.2 0.70 ? 2% -0.2 0.82 ? 2% -0.1 0.89 ? 5% -0.1 0.56 -0.1 0.64 ? 8% -0.1 0.50 -0.1 0.65 ? 8% -0.1 0.49 -0.1 0.52 -0.1 0.32 ? 2% -0.1 0.42 ? 2% -0.1 0.44 -0.1 0.26 -0.1 0.27 ? 3% -0.1 0.27 -0.1 0.34 -0.1 0.32 ? 6% -0.1 0.61 -0.1 0.21 ? 2% -0.1 0.26 ? 8% -0.0 0.31 ? 6% -0.0 0.34 ? 8% -0.0 0.22 ? 2% -0.0 0.19 ? 2% -0.0 0.20 ? 6% -0.0 0.33 ? 2% -0.0 0.28 -0.0 0.18 -0.0 0.20 ? 2% -0.0 0.19 ? 8% -0.0 0.13 ? 6% -0.0 0.12 ? 6% -0.0 0.20 ? 2% -0.0 0.06 ? 7% -0.0 0.21 ? 9% -0.0 0.15 ? 3% -0.0 0.35 -0.0 0.17 ? 2% -0.0 0.18 ? 2% -0.0 0.21 -0.0 0.08 ? 10% -0.0 0.09 ? 5% -0.0 0.12 -0.0 0.22 ? 2% -0.0 0.12 ? 3% -0.0 0.10 ? 9% -0.0 0.07 ? 6% -0.0 0.07 -0.0 0.15 ? 3% -0.0 0.12 -0.0 0.12 -0.0 0.07 ? 7% -0.0 0.11 ? 4% -0.0 0.10 ? 4% -0.0 0.07 -0.0 0.14 ? 5% +0.0 0.16 ? 5% +0.0 0.10 ? 4% +0.0 0.12 ? 8% +0.0 0.21 ? 6% +0.0 0.15 ? 5% +0.1 0.55 ? 5% +0.1 0.00 +0.1 0.09 +0.3 4.61 ? 4% +0.9 15.05 +1.0 30.89 +1.4 29.75 +1.5 29.60 +1.5 28.77 +1.5 28.77 +1.5 30.42 +1.5 30.43 +1.5 50.47 +2.1 50.93 +2.2 50.93 +2.2 50.91 +2.2 4.41 +4.3 4.39 +4.4 4.29 +4.4 3.86 +4.5 3.40 +4.6 0.67 +5.1 1.78 +5.1 1.78 +5.1 21.60 +6.6 21.40 +7.5 0.11 ? 4% +10.0 2.40 +10.3 10.91 -2.4 42.19 -1.8 6.82 -1.2 4.18 ? 4% -0.4 0.75 ? 2% -0.2 0.32 ? 4% -0.1 0.57 ? 3% -0.1 0.57 -0.1 0.29 -0.1 0.44 -0.1 0.09 ? 5% -0.1 0.26 ? 3% -0.1 0.21 -0.0 0.34 ? 2% -0.0 0.18 ? 2% -0.0 0.20 ? 2% -0.0 0.20 ? 2% -0.0 0.17 ? 2% -0.0 0.15 ? 5% -0.0 0.15 ? 3% -0.0 0.17 ? 2% -0.0 0.21 -0.0 0.20 ? 4% -0.0 0.10 ? 9% -0.0 0.08 ? 5% -0.0 0.07 ? 6% -0.0 0.13 -0.0 0.10 ? 4% -0.0 0.07 -0.0 0.14 ? 3% -0.0 0.12 -0.0 0.12 ? 4% -0.0 0.20 ? 4% -0.0 0.16 ? 3% -0.0 0.12 ? 4% -0.0 0.10 -0.0 0.07 -0.0 0.06 ? 7% -0.0 0.09 ? 5% -0.0 0.07 -0.0 0.08 ? 6% +0.0 0.14 ? 5% +0.0 0.21 ? 6% +0.0 0.21 +0.1 0.15 ? 5% +0.1 0.42 ? 2% +0.1 0.00 +0.1 0.00 +0.1 0.09 +0.3 0.65 +0.6 21.50 +6.6

Disclaimer:
Results have been for informational design or configuration

--
0-DAY CI Kernel Test Service
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
threads 2 sockets (Sapphire Rapids) with 256G memory
Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
//github.com/antonblanchard/will-it-scale">https://github.com/antonblanchard/will-it-scale
the commit also has significant impact on the following tests:
----------------------------------------------------------------------------------------------+
| will-it-scale: will-it-scale.per_thread_ops -73.8% regression |
| 224 threads 2 sockets (Sapphire Rapids) with 256G memory |
| cpufreq_governor=performance |
|
|
| test=context_switch1 |
----------------------------------------------------------------------------------------------+
| will-it-scale: will-it-scale.per_thread_ops -57.9% regression |
| 104 threads 2 sockets (Skylake) with 192G memory |
| cpufreq_governor=performance |
|
|
| test=context_switch1 |
----------------------------------------------------------------------------------------------+
| will-it-scale: will-it-scale.per_thread_ops -85.0% regression |
| 104 threads 2 sockets (Skylake) with 192G memory |
| cpufreq_governor=performance |
|
|
| test=context_switch1 |
----------------------------------------------------------------------------------------------+
| vm-scalability: vm-scalability.throughput -9.0% regression |
| 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory |
| cpufreq_governor=performance |
|
|
| test=shm-xread-seq-mt |
----------------------------------------------------------------------------------------------+
commit 223baf9d17f2 (sched: Fix performance
by mm_cid) fixed a sysbench regression, but
benchmark still saw a regression on this
the code diff in below link [1] on mainline, and
score was restored to the original level before this
.kernel.org/lkml/d96164a6-c522-1bfc-8b37-333726cdc573@efficios.com/">https://lore.kernel.org/lkml/[email protected]/
kindly add following tag
test robot <[email protected]>
lore.kernel.org/oe-lkp/202305151017.27581d75-yujie.liu@intel.com">https://lore.kernel.org/oe-lkp/[email protected]
===============================================================
kconfig/rootfs/tbox_group/test/testcase:
4-rhel-8.3/debian-11.1-x86_64-20220510.cgz/lkp-spr-r02/context_switch1/will-it-scale
Implement rseq numa node id field selftest")
Introduce per-memory-map concurrency ID")
af7f588d8f7355bc4298dd1962d
---------------------------
%stddev
\
-1.2% 621753 will-it-scale.1.linear
-1.2% 621753 will-it-scale.1.processes
-1.5% 604163 will-it-scale.1.threads
-1.2% 69636432 will-it-scale.112.linear
-72.0% 1571768 ? 3% will-it-scale.112.threads
2.50 ? 4% will-it-scale.112.threads_idle
-1.2% 1.045e+08 will-it-scale.168.linear
-20.6% 45250154 ? 2% will-it-scale.168.threads
1.96 ? 28% will-it-scale.168.threads_idle
-1.2% 1.393e+08 will-it-scale.224.linear
+1.9% 97207305 will-it-scale.224.processes
+1.4% 1.097e+08 will-it-scale.224.threads
-1.2% 34818216 will-it-scale.56.linear
-76.2% 2092567 will-it-scale.56.threads
-3.4% 66.22 will-it-scale.56.threads_idle
-13.9% 282968 will-it-scale.per_thread_ops
0.70 will-it-scale.scalability
-37.1% 2.564e+08 ? 8% will-it-scale.time.involuntary_context_switches
+1.4% 17713 will-it-scale.time.system_time
1369 will-it-scale.time.user_time
-8.6% 5.607e+09 will-it-scale.time.voluntary_context_switches
-4.2% 4.39e+08 will-it-scale.workload
-13.3% 4.166e+09 cpuidle..usage
0.09 ? 3% mpstat.cpu.all.soft%
-62.5% 8384327 ? 4% vmstat.memory.cache
-6.3% 55960083 vmstat.system.cs
+511.7% 2648925 ? 42% numa-numastat.node0.local_node
+391.1% 2814064 ? 39% numa-numastat.node0.numa_hit
-69.0% 2496648 ? 43% numa-numastat.node1.local_node
-68.6% 2550541 ? 42% numa-numastat.node1.numa_hit
-82.0% 39691191 ? 11% turbostat.C1
0.09 ? 14% turbostat.C1%
-56.5% 47858373 turbostat.C1E
2.73 turbostat.POLL%
4607008 ? 4% meminfo.Active
4606928 ? 4% meminfo.Active(anon)
-22.6% 170030 ? 15% meminfo.AnonHugePages
-24.2% 434172 meminfo.AnonPages
-62.4% 8263088 ? 4% meminfo.Cached
-55.0% 11347952 ? 3% meminfo.Committed_AS
-93.9% 1192582 ? 23% meminfo.Inactive
-93.9% 1192439 ? 23% meminfo.Inactive(anon)
-14.7% 183232 meminfo.KReclaimable
-56.0% 11175036 ? 3% meminfo.Memused
-14.7% 183232 meminfo.SReclaimable
-71.9% 5366191 ? 6% meminfo.Shmem
-34.2% 16986009 meminfo.max_used_kB
12679 ? 98% sched_debug.cfs_rq:/.MIN_vruntime.avg
-69.8% 846737 ? 85% sched_debug.cfs_rq:/.MIN_vruntime.max
88033 ? 76% sched_debug.cfs_rq:/.MIN_vruntime.stddev
12679 ? 98% sched_debug.cfs_rq:/.max_vruntime.avg
-69.8% 846738 ? 85% sched_debug.cfs_rq:/.max_vruntime.max
88033 ? 76% sched_debug.cfs_rq:/.max_vruntime.stddev
+12.8% 964472 ? 2% sched_debug.cfs_rq:/.min_vruntime.stddev
609692 ? 44% sched_debug.cfs_rq:/.spread0.max
+12.1% 966253 ? 2% sched_debug.cfs_rq:/.spread0.stddev
-14.0% 277.54 ? 6% sched_debug.cfs_rq:/.util_est_enqueued.avg
+11.5% 525288 ? 4% sched_debug.cpu.avg_idle.avg
30184 ? 27% sched_debug.cpu.avg_idle.min
28.54 ? 6% sched_debug.cpu.clock.stddev
0.00 ? 16% sched_debug.cpu.next_balance.stddev
-9.5% 26550823 sched_debug.cpu.nr_switches.avg
-12.1% 8262067 sched_debug.cpu.nr_switches.stddev
2458393 ? 56% numa-meminfo.node0.Active
2458381 ? 56% numa-meminfo.node0.Active(anon)
803087 ? 36% numa-meminfo.node0.Inactive
803069 ? 36% numa-meminfo.node0.Inactive(anon)
14936 ? 4% numa-meminfo.node0.PageTables
3020604 ? 55% numa-meminfo.node0.Shmem
+841.3% 2156586 ? 64% numa-meminfo.node1.Active
+841.5% 2156518 ? 64% numa-meminfo.node1.Active(anon)
79245 ? 34% numa-meminfo.node1.AnonHugePages
-49.7% 192976 ? 22% numa-meminfo.node1.AnonPages
-87.0% 2485880 ? 58% numa-meminfo.node1.FilePages
-98.0% 390419 ? 19% numa-meminfo.node1.Inactive
-98.0% 390295 ? 19% numa-meminfo.node1.Inactive(anon)
67069 ? 15% numa-meminfo.node1.KReclaimable
-49.4% 1410452 ? 52% numa-meminfo.node1.Mapped
+15.3% 1.283e+08 numa-meminfo.node1.MemFree
-81.8% 3782627 ? 38% numa-meminfo.node1.MemUsed
10254 ? 7% numa-meminfo.node1.PageTables
67069 ? 15% numa-meminfo.node1.SReclaimable
-87.6% 2354397 ? 60% numa-meminfo.node1.Shmem
-14.1% 242658 ? 7% numa-meminfo.node1.Slab
615291 ? 56% numa-vmstat.node0.nr_active_anon
201111 ? 35% numa-vmstat.node0.nr_inactive_anon
3738 ? 4% numa-vmstat.node0.nr_page_table_pages
756126 ? 55% numa-vmstat.node0.nr_shmem
615292 ? 56% numa-vmstat.node0.nr_zone_active_anon
201111 ? 35% numa-vmstat.node0.nr_zone_inactive_anon
+390.9% 2813153 ? 39% numa-vmstat.node0.numa_hit
+511.4% 2648015 ? 42% numa-vmstat.node0.numa_local
540005 ? 64% numa-vmstat.node1.nr_active_anon
48299 ? 22% numa-vmstat.node1.nr_anon_pages
-87.0% 622469 ? 58% numa-vmstat.node1.nr_file_pages
+15.3% 32063992 numa-vmstat.node1.nr_free_pages
97769 ? 19% numa-vmstat.node1.nr_inactive_anon
-49.3% 353752 ? 52% numa-vmstat.node1.nr_mapped
2567 ? 7% numa-vmstat.node1.nr_page_table_pages
-87.6% 589599 ? 60% numa-vmstat.node1.nr_shmem
16769 ? 15% numa-vmstat.node1.nr_slab_reclaimable
540006 ? 64% numa-vmstat.node1.nr_zone_active_anon
97769 ? 19% numa-vmstat.node1.nr_zone_inactive_anon
-68.6% 2548812 ? 42% numa-vmstat.node1.numa_hit
-69.0% 2494919 ? 43% numa-vmstat.node1.numa_local
1153633 ? 4% proc-vmstat.nr_active_anon
-24.2% 108550 proc-vmstat.nr_anon_pages
+6.0% 6274221 proc-vmstat.nr_dirty_background_threshold
+6.0% 12563784 proc-vmstat.nr_dirty_threshold
-62.4% 2067724 ? 4% proc-vmstat.nr_file_pages
+6.0% 63118840 proc-vmstat.nr_free_pages
-93.9% 298169 ? 23% proc-vmstat.nr_inactive_anon
6328 proc-vmstat.nr_page_table_pages
-71.8% 1343500 ? 6% proc-vmstat.nr_shmem
45811 proc-vmstat.nr_slab_reclaimable
1153633 ? 4% proc-vmstat.nr_zone_active_anon
-93.9% 298169 ? 23% proc-vmstat.nr_zone_inactive_anon
-50.0% 341609 ? 4% proc-vmstat.numa_hint_faults
-60.2% 206845 ? 27% proc-vmstat.numa_hint_faults_local
-38.3% 5371241 proc-vmstat.numa_hit
-55.8% 414.71 ? 8% proc-vmstat.numa_huge_pte_updates
-39.4% 5152209 proc-vmstat.numa_local
-46.7% 700042 ? 3% proc-vmstat.numa_pte_updates
830307 ? 21% proc-vmstat.pgactivate
-37.8% 5550603 proc-vmstat.pgalloc_normal
-11.1% 2006618 proc-vmstat.pgfault
+268.9% 5607989 proc-vmstat.pgfree
0.82 ? 11% perf-stat.i.MPKI
-4.8% 7.269e+10 perf-stat.i.branch-instructions
0.61 perf-stat.i.branch-miss-rate%
-5.5% 4.715e+08 perf-stat.i.branch-misses
-8.1% 5897662 ? 5% perf-stat.i.cache-misses
+51.6% 76473775 ? 10% perf-stat.i.cache-references
-6.7% 56856550 perf-stat.i.context-switches
2.42 perf-stat.i.cpi
+1.7% 4.424e+11 perf-stat.i.cpu-cycles
-21.6% 830.16 ? 4% perf-stat.i.cpu-migrations
+63.0% 627819 ? 4% perf-stat.i.cycles-between-cache-misses
0.00 ? 12% perf-stat.i.dTLB-load-miss-rate%
-65.0% 920193 ? 5% perf-stat.i.dTLB-load-misses
-6.0% 1.038e+11 perf-stat.i.dTLB-loads
0.00 ? 12% perf-stat.i.dTLB-store-miss-rate%
-19.0% 196450 ? 11% perf-stat.i.dTLB-store-misses
-8.4% 6.071e+10 perf-stat.i.dTLB-stores
-5.8% 3.595e+11 perf-stat.i.instructions
0.75 perf-stat.i.ipc
1.97 perf-stat.i.metric.GHz
-4.4% 482.60 perf-stat.i.metric.K/sec
1058 perf-stat.i.metric.M/sec
5560 perf-stat.i.minor-faults
88.22 ? 6% perf-stat.i.node-load-miss-rate%
+34.5% 1502746 ? 10% perf-stat.i.node-load-misses
-70.6% 134710 ? 30% perf-stat.i.node-loads
5562 perf-stat.i.page-faults
0.22 ? 10% perf-stat.overall.MPKI
7.76 ? 14% perf-stat.overall.cache-miss-rate%
1.24 perf-stat.overall.cpi
74790 ? 4% perf-stat.overall.cycles-between-cache-misses
0.00 ? 5% perf-stat.overall.dTLB-load-miss-rate%
0.81 perf-stat.overall.ipc
91.45 ? 3% perf-stat.overall.node-load-miss-rate%
-1.1% 257287 perf-stat.overall.path-length
-4.2% 7.195e+10 perf-stat.ps.branch-instructions
-5.1% 4.664e+08 perf-stat.ps.branch-misses
+52.5% 77201196 ? 10% perf-stat.ps.cache-references
-6.4% 56232111 perf-stat.ps.context-switches
+1.8% 4.408e+11 perf-stat.ps.cpu-cycles
-21.2% 826.67 ? 4% perf-stat.ps.cpu-migrations
-65.6% 930018 ? 5% perf-stat.ps.dTLB-load-misses
-5.5% 1.027e+11 perf-stat.ps.dTLB-loads
-19.3% 195733 ? 11% perf-stat.ps.dTLB-store-misses
-7.9% 6.005e+10 perf-stat.ps.dTLB-stores
-5.3% 3.558e+11 perf-stat.ps.instructions
5547 perf-stat.ps.minor-faults
+36.5% 1507093 ? 10% perf-stat.ps.node-load-misses
-71.3% 137270 ? 29% perf-stat.ps.node-loads
5549 perf-stat.ps.page-faults
-5.3% 1.129e+14 perf-stat.total.instructions
0.00 perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
4.38 ? 3% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
4.35 ? 3% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
3.97 ? 3% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
48.81 perf-profile.calltrace.cycles-pp.__libc_read
2.53 ? 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
2.48 ? 2% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
1.18 ? 3% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
2.45 ? 6% perf-profile.calltrace.cycles-pp.__libc_write
2.08 ? 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
2.33 ? 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
2.26 ? 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
2.31 ? 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
2.24 ? 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
2.19 ? 3% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
2.48 ? 13% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
0.66 ? 54% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
48.46 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
48.43 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
2.08 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
2.02 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
2.10 ? 5% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.95 ? 3% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
48.35 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
48.28 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
1.84 ? 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
1.90 ? 5% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
1.80 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
1.86 ? 5% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
48.19 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.85 ? 5% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
1.73 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
1.72 ? 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
1.83 ? 5% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
0.00 perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
1.47 ? 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
1.50 ? 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
1.46 ? 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.43 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.46 ? 4% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
1.39 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
1.37 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
1.36 ? 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
0.84 ? 5% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
1.14 ? 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
1.25 ? 4% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
1.21 ? 4% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
47.75 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
1.16 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
47.71 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
0.58 ? 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.58 ? 2% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
47.12 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
47.11 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
47.09 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
47.57 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
43.94 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
43.91 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
41.26 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
41.49 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
41.57 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
41.80 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
42.01 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
42.08 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
6.09 ? 3% perf-profile.children.cycles-pp.perf_tp_event
5.84 ? 3% perf-profile.children.cycles-pp.__perf_event_overflow
5.79 ? 3% perf-profile.children.cycles-pp.perf_event_output_forward
0.79 ? 8% perf-profile.children.cycles-pp.finish_task_switch
4.85 ? 3% perf-profile.children.cycles-pp.perf_prepare_sample
4.53 ? 3% perf-profile.children.cycles-pp.perf_callchain
4.51 ? 3% perf-profile.children.cycles-pp.get_perf_callchain
51.52 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
51.48 perf-profile.children.cycles-pp.do_syscall_64
4.15 ? 3% perf-profile.children.cycles-pp.perf_callchain_kernel
3.30 ? 3% perf-profile.children.cycles-pp.unwind_next_frame
2.80 ? 4% perf-profile.children.cycles-pp.perf_trace_sched_switch
48.86 perf-profile.children.cycles-pp.__libc_read
2.50 ? 3% perf-profile.children.cycles-pp.cpuidle_idle_call
3.21 ? 4% perf-profile.children.cycles-pp.__libc_write
2.14 ? 3% perf-profile.children.cycles-pp.__unwind_start
2.94 ? 3% perf-profile.children.cycles-pp.ksys_write
2.26 ? 3% perf-profile.children.cycles-pp.dequeue_task_fair
2.20 ? 3% perf-profile.children.cycles-pp.dequeue_entity
2.84 ? 3% perf-profile.children.cycles-pp.vfs_write
2.09 ? 3% perf-profile.children.cycles-pp.cpuidle_enter
2.08 ? 3% perf-profile.children.cycles-pp.cpuidle_enter_state
2.11 ? 5% perf-profile.children.cycles-pp.pipe_write
48.37 perf-profile.children.cycles-pp.ksys_read
2.12 ? 3% perf-profile.children.cycles-pp.update_curr
48.30 perf-profile.children.cycles-pp.vfs_read
1.98 ? 3% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
1.91 ? 5% perf-profile.children.cycles-pp.__wake_up_common_lock
48.21 perf-profile.children.cycles-pp.pipe_read
1.86 ? 5% perf-profile.children.cycles-pp.__wake_up_common
1.85 ? 5% perf-profile.children.cycles-pp.autoremove_wake_function
1.84 ? 5% perf-profile.children.cycles-pp.try_to_wake_up
1.46 ? 3% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.87 ? 5% perf-profile.children.cycles-pp.poll_idle
47.81 perf-profile.children.cycles-pp.schedule
0.15 ? 6% perf-profile.children.cycles-pp.__switch_to
0.91 ? 3% perf-profile.children.cycles-pp.__orc_find
0.58 ? 2% perf-profile.children.cycles-pp.mwait_idle_with_hints
0.58 ? 2% perf-profile.children.cycles-pp.intel_idle
0.58 ? 3% perf-profile.children.cycles-pp.stack_access_ok
0.48 ? 3% perf-profile.children.cycles-pp.unwind_get_return_address
0.41 ? 3% perf-profile.children.cycles-pp.__kernel_text_address
0.44 ? 3% perf-profile.children.cycles-pp.orc_find
0.49 ? 10% perf-profile.children.cycles-pp.sched_ttwu_pending
0.39 ? 6% perf-profile.children.cycles-pp.flush_smp_call_function_queue
0.30 ? 3% perf-profile.children.cycles-pp.kernel_text_address
0.33 ? 14% perf-profile.children.cycles-pp.menu_select
0.46 ? 17% perf-profile.children.cycles-pp.ttwu_do_activate
0.45 ? 17% perf-profile.children.cycles-pp.enqueue_task_fair
0.34 ? 9% perf-profile.children.cycles-pp.update_load_avg
0.23 ? 4% perf-profile.children.cycles-pp.perf_callchain_user
0.35 ? 15% perf-profile.children.cycles-pp.enqueue_entity
0.75 ? 4% perf-profile.children.cycles-pp.perf_output_sample
0.18 ? 3% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.24 ? 4% perf-profile.children.cycles-pp.sched_clock_cpu
0.16 ? 2% perf-profile.children.cycles-pp.__get_user_nocheck_8
0.13 ? 8% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.13 ? 8% perf-profile.children.cycles-pp.perf_output_begin_forward
0.30 ? 15% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.11 ? 4% perf-profile.children.cycles-pp.prepare_to_wait_event
0.58 ? 4% perf-profile.children.cycles-pp.__output_copy
0.14 ? 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.19 ? 5% perf-profile.children.cycles-pp.native_sched_clock
0.14 ? 3% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.12 ? 6% perf-profile.children.cycles-pp.__switch_to_asm
0.49 ? 4% perf-profile.children.cycles-pp.perf_output_copy
0.19 ? 22% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.11 ? 3% perf-profile.children.cycles-pp.set_next_entity
0.10 ? 6% perf-profile.children.cycles-pp.mutex_lock
0.23 ? 12% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.11 ? 4% perf-profile.children.cycles-pp.__entry_text_start
0.10 ? 4% perf-profile.children.cycles-pp.core_kernel_text
0.20 ? 17% perf-profile.children.cycles-pp.sysvec_call_function_single
0.13 ? 7% perf-profile.children.cycles-pp.update_rq_clock
0.08 ? 4% perf-profile.children.cycles-pp.copy_page_from_iter
0.11 ? 3% perf-profile.children.cycles-pp.__fdget_pos
0.11 ? 4% perf-profile.children.cycles-pp.__update_load_avg_se
0.20 ? 13% perf-profile.children.cycles-pp.copyin
0.10 ? 5% perf-profile.children.cycles-pp.__fget_light
0.12 ? 11% perf-profile.children.cycles-pp.ktime_get
0.17 ? 18% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.29 ? 6% perf-profile.children.cycles-pp.memcpy_erms
0.07 ? 7% perf-profile.children.cycles-pp.copy_page_to_iter
0.08 ? 8% perf-profile.children.cycles-pp.__fget_files
0.07 ? 4% perf-profile.children.cycles-pp._copy_from_iter
0.07 ? 7% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.16 ? 21% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.07 ? 6% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.06 perf-profile.children.cycles-pp.kvm_guest_state
0.13 ? 30% perf-profile.children.cycles-pp.tick_nohz_next_event
0.06 ? 5% perf-profile.children.cycles-pp._copy_to_iter
0.05 perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.06 ? 5% perf-profile.children.cycles-pp.os_xsave
0.05 ? 9% perf-profile.children.cycles-pp.get_stack_info
0.07 ? 7% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.06 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.06 ? 7% perf-profile.children.cycles-pp.__might_resched
0.06 ? 10% perf-profile.children.cycles-pp.perf_trace_buf_update
0.05 ? 6% perf-profile.children.cycles-pp.security_file_permission
0.05 ? 6% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.05 perf-profile.children.cycles-pp.get_next_timer_interrupt
0.04 ? 40% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.07 ? 10% perf-profile.children.cycles-pp.___perf_sw_event
0.17 ? 16% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.63 ? 7% perf-profile.children.cycles-pp.generic_perform_write
0.06 perf-profile.children.cycles-pp.read_tsc
0.10 ? 8% perf-profile.children.cycles-pp.select_task_rq
0.11 ? 29% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.64 ? 7% perf-profile.children.cycles-pp.__generic_file_write_iter
0.65 ? 7% perf-profile.children.cycles-pp.generic_file_write_iter
0.09 ? 9% perf-profile.children.cycles-pp.select_task_rq_fair
0.10 ? 9% perf-profile.children.cycles-pp.native_irq_return_iret
0.11 ? 10% perf-profile.children.cycles-pp.__irq_exit_rcu
0.03 ? 86% perf-profile.children.cycles-pp.pick_next_task_idle
0.08 ? 11% perf-profile.children.cycles-pp.__do_softirq
0.05 ? 43% perf-profile.children.cycles-pp.rebalance_domains
0.08 ? 47% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.05 ? 6% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.07 ? 44% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.05 ? 9% perf-profile.children.cycles-pp.memcpy
0.06 ? 18% perf-profile.children.cycles-pp.clockevents_program_event
0.16 ? 8% perf-profile.children.cycles-pp.scheduler_tick
0.21 ? 6% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.23 ? 18% perf-profile.children.cycles-pp.tick_sched_handle
0.23 ? 18% perf-profile.children.cycles-pp.update_process_times
0.13 ? 24% perf-profile.children.cycles-pp.update_blocked_averages
0.45 ? 12% perf-profile.children.cycles-pp.start_kernel
0.45 ? 12% perf-profile.children.cycles-pp.arch_call_rest_init
0.45 ? 12% perf-profile.children.cycles-pp.rest_init
0.85 ? 46% perf-profile.children.cycles-pp.update_sg_lb_stats
0.91 ? 45% perf-profile.children.cycles-pp.update_sd_lb_stats
0.93 ? 45% perf-profile.children.cycles-pp.find_busiest_group
1.01 ? 44% perf-profile.children.cycles-pp.load_balance
1.20 ? 41% perf-profile.children.cycles-pp.newidle_balance
47.12 perf-profile.children.cycles-pp.start_secondary
47.57 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
47.57 perf-profile.children.cycles-pp.cpu_startup_entry
47.55 perf-profile.children.cycles-pp.do_idle
92.13 perf-profile.children.cycles-pp.__schedule
44.38 perf-profile.children.cycles-pp.schedule_idle
83.49 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
84.07 perf-profile.children.cycles-pp.prepare_task_switch
84.12 perf-profile.children.cycles-pp._raw_spin_lock
0.37 ? 7% perf-profile.self.cycles-pp.finish_task_switch
0.39 ? 6% perf-profile.self.cycles-pp.__schedule
1.41 ? 3% perf-profile.self.cycles-pp.unwind_next_frame
0.79 ? 6% perf-profile.self.cycles-pp.poll_idle
0.14 ? 4% perf-profile.self.cycles-pp.__switch_to
0.87 ? 3% perf-profile.self.cycles-pp.__orc_find
0.58 ? 2% perf-profile.self.cycles-pp.mwait_idle_with_hints
0.47 ? 3% perf-profile.self.cycles-pp.stack_access_ok
0.40 ? 3% perf-profile.self.cycles-pp.orc_find
0.19 ? 3% perf-profile.self.cycles-pp.__unwind_start
0.19 ? 4% perf-profile.self.cycles-pp.perf_callchain_kernel
0.15 ? 3% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.13 ? 8% perf-profile.self.cycles-pp.perf_output_begin_forward
0.14 ? 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.11 ? 5% perf-profile.self.cycles-pp.__switch_to_asm
0.18 ? 5% perf-profile.self.cycles-pp.native_sched_clock
0.07 ? 7% perf-profile.self.cycles-pp.pipe_read
0.11 ? 4% perf-profile.self.cycles-pp.kernel_text_address
0.22 ? 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.09 ? 3% perf-profile.self.cycles-pp.perf_prepare_sample
0.09 ? 5% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.11 ? 6% perf-profile.self.cycles-pp.menu_select
0.09 ? 5% perf-profile.self.cycles-pp.unwind_get_return_address
0.09 ? 3% perf-profile.self.cycles-pp.__kernel_text_address
0.09 ? 3% perf-profile.self.cycles-pp.get_perf_callchain
0.09 ? 3% perf-profile.self.cycles-pp.update_load_avg
0.06 ? 5% perf-profile.self.cycles-pp.cpuidle_idle_call
0.11 ? 13% perf-profile.self.cycles-pp.enqueue_entity
0.07 ? 4% perf-profile.self.cycles-pp.core_kernel_text
0.07 ? 5% perf-profile.self.cycles-pp.do_idle
0.08 ? 7% perf-profile.self.cycles-pp.__fget_files
0.10 ? 5% perf-profile.self.cycles-pp.__update_load_avg_se
0.26 ? 6% perf-profile.self.cycles-pp.memcpy_erms
0.06 ? 8% perf-profile.self.cycles-pp.__libc_read
0.06 ? 8% perf-profile.self.cycles-pp.prepare_to_wait_event
0.06 ? 7% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.07 ? 6% perf-profile.self.cycles-pp.update_curr
0.06 perf-profile.self.cycles-pp.__task_pid_nr_ns
0.15 ? 21% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.17 ? 25% perf-profile.self.cycles-pp.perf_tp_event
0.06 ? 8% perf-profile.self.cycles-pp.os_xsave
0.07 ? 9% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.24 ? 3% perf-profile.self.cycles-pp.__output_copy
0.06 perf-profile.self.cycles-pp.perf_callchain_user
0.07 ? 7% perf-profile.self.cycles-pp.__libc_write
0.06 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.04 ? 40% perf-profile.self.cycles-pp.kvm_guest_state
0.05 perf-profile.self.cycles-pp.__wrgsbase_inactive
0.06 perf-profile.self.cycles-pp.__might_resched
0.03 ? 86% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.06 ? 7% perf-profile.self.cycles-pp.dequeue_entity
0.16 ? 2% perf-profile.self.cycles-pp.perf_output_sample
0.06 ? 8% perf-profile.self.cycles-pp.update_rq_clock
0.07 ? 17% perf-profile.self.cycles-pp.ktime_get
0.06 ? 5% perf-profile.self.cycles-pp.read_tsc
0.04 ? 65% perf-profile.self.cycles-pp.___perf_sw_event
0.06 ? 41% perf-profile.self.cycles-pp.dequeue_task_fair
0.08 ? 42% perf-profile.self.cycles-pp.enqueue_task_fair
0.08 ? 4% perf-profile.self.cycles-pp.cpuidle_enter_state
0.10 ? 9% perf-profile.self.cycles-pp.native_irq_return_iret
0.06 ? 43% perf-profile.self.cycles-pp.newidle_balance
0.08 ? 13% perf-profile.self.cycles-pp.prepare_task_switch
0.05 ? 6% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.06 ? 5% perf-profile.self.cycles-pp.perf_output_copy
0.06 ? 43% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.21 ? 7% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.62 ? 7% perf-profile.self.cycles-pp._raw_spin_lock
0.66 ? 45% perf-profile.self.cycles-pp.update_sg_lb_stats
83.49 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
*************************************************************************
threads 2 sockets (Sapphire Rapids) with 256G memory
===============================================================
kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
4-rhel-8.3/thread/16/debian-11.1-x86_64-20220510.cgz/lkp-spr-r02/context_switch1/will-it-scale
Implement rseq numa node id field selftest")
Introduce per-memory-map concurrency ID")
af7f588d8f7355bc4298dd1962d
---------------------------
%stddev
\
-73.8% 2400232 ? 10% will-it-scale.16.threads
-73.8% 150014 ? 10% will-it-scale.per_thread_ops
-73.8% 2400232 ? 10% will-it-scale.workload
-73.1% 7.715e+08 ? 11% cpuidle..usage
7.15 mpstat.cpu.all.sys%
0.27 ? 10% mpstat.cpu.all.usr%
-78.9% 4696291 ? 10% vmstat.memory.cache
16.20 ? 2% vmstat.procs.r
-73.8% 4753330 ? 9% vmstat.system.cs
+35.8% 656610 ? 5% vmstat.system.in
+60.7% 2499655 ? 18% turbostat.C1
+24.1% 1.086e+08 ? 14% turbostat.C1E
75.30 ? 24% turbostat.C1E%
0.20 ? 8% turbostat.IPC
+35.9% 1.995e+08 ? 5% turbostat.IRQ
-76.2% 6.523e+08 ? 14% turbostat.POLL
0.68 ? 4% turbostat.POLL%
-9.9% 30.80 turbostat.PkgTmp
-10.4% 388.15 turbostat.PkgWatt
305243 ? 32% numa-vmstat.node1.nr_active_anon
43246 ? 10% numa-vmstat.node1.nr_anon_pages
-88.5% 513735 ? 65% numa-vmstat.node1.nr_file_pages
+14.5% 32194305 numa-vmstat.node1.nr_free_pages
97706 ? 85% numa-vmstat.node1.nr_inactive_anon
-91.3% 153214 ? 43% numa-vmstat.node1.nr_mapped
1025 ? 21% numa-vmstat.node1.nr_page_table_pages
-91.7% 359713 ? 46% numa-vmstat.node1.nr_shmem
305243 ? 32% numa-vmstat.node1.nr_zone_active_anon
97706 ? 85% numa-vmstat.node1.nr_zone_inactive_anon
+807.2% 1221042 ? 32% numa-meminfo.node1.Active
+807.7% 1220946 ? 32% numa-meminfo.node1.Active(anon)
-22.7% 172976 ? 10% numa-meminfo.node1.AnonPages
-88.5% 2055168 ? 65% numa-meminfo.node1.FilePages
-97.7% 391247 ? 85% numa-meminfo.node1.Inactive
-97.7% 391073 ? 85% numa-meminfo.node1.Inactive(anon)
-91.3% 614131 ? 43% numa-meminfo.node1.Mapped
+14.5% 1.288e+08 numa-meminfo.node1.MemFree
-83.3% 3266862 ? 41% numa-meminfo.node1.MemUsed
4105 ? 21% numa-meminfo.node1.PageTables
-91.7% 1439083 ? 46% numa-meminfo.node1.Shmem
-16.6% 351355 meminfo.AnonPages
-79.5% 4530612 ? 11% meminfo.Cached
-87.8% 2443971 ? 21% meminfo.Committed_AS
-97.0% 574987 ? 60% meminfo.Inactive
-97.0% 574813 ? 60% meminfo.Inactive(anon)
-18.3% 173997 meminfo.KReclaimable
-90.5% 705190 ? 33% meminfo.Mapped
-71.9% 7127544 ? 7% meminfo.Memused
7906 ? 7% meminfo.PageTables
-18.3% 173997 meminfo.SReclaimable
-91.5% 1633768 ? 30% meminfo.Shmem
-39.4% 16170877 ? 7% meminfo.max_used_kB
1210 ?143% sched_debug.cfs_rq:/.MIN_vruntime.avg
87886 ?144% sched_debug.cfs_rq:/.MIN_vruntime.max
10166 ?144% sched_debug.cfs_rq:/.MIN_vruntime.stddev
1210 ?143% sched_debug.cfs_rq:/.max_vruntime.avg
87886 ?144% sched_debug.cfs_rq:/.max_vruntime.max
10166 ?144% sched_debug.cfs_rq:/.max_vruntime.stddev
47201 ? 13% sched_debug.cfs_rq:/.min_vruntime.min
-15.8% 824.68 ? 4% sched_debug.cfs_rq:/.runnable_avg.max
-17.1% 811.25 ? 3% sched_debug.cfs_rq:/.util_avg.max
-30.0% 601.91 ? 4% sched_debug.cfs_rq:/.util_est_enqueued.max
10522 ? 91% sched_debug.cpu.avg_idle.min
0.08 ? 5% sched_debug.cpu.nr_running.avg
-71.4% 3041094 ? 6% sched_debug.cpu.nr_switches.avg
-71.5% 21318321 ? 6% sched_debug.cpu.nr_switches.max
-71.4% 7440127 ? 6% sched_debug.cpu.nr_switches.stddev
87842 proc-vmstat.nr_anon_pages
+7.7% 6375494 proc-vmstat.nr_dirty_background_threshold
+7.7% 12766578 proc-vmstat.nr_dirty_threshold
-79.5% 1132643 ? 11% proc-vmstat.nr_file_pages
+7.6% 64133047 proc-vmstat.nr_free_pages
-97.0% 143617 ? 60% proc-vmstat.nr_inactive_anon
-90.6% 175442 ? 33% proc-vmstat.nr_mapped
1976 ? 7% proc-vmstat.nr_page_table_pages
-91.5% 408431 ? 30% proc-vmstat.nr_shmem
43499 proc-vmstat.nr_slab_reclaimable
-1.4% 102106 proc-vmstat.nr_slab_unreclaimable
-97.0% 143617 ? 60% proc-vmstat.nr_zone_inactive_anon
68247 ? 12% proc-vmstat.numa_hint_faults
17501 ? 29% proc-vmstat.numa_hint_faults_local
-46.3% 4809967 ? 10% proc-vmstat.numa_hit
-89.0% 151.80 ? 23% proc-vmstat.numa_huge_pte_updates
-46.7% 4607108 ? 10% proc-vmstat.numa_local
-80.9% 197728 ? 15% proc-vmstat.numa_pte_updates
-45.0% 4950106 ? 9% proc-vmstat.pgalloc_normal
-23.0% 1311857 ? 3% proc-vmstat.pgfault
1.19 ? 3% perf-stat.i.MPKI
-65.5% 7.138e+09 ? 7% perf-stat.i.branch-instructions
-66.0% 45319480 ? 8% perf-stat.i.branch-misses
4.68 ? 6% perf-stat.i.cache-miss-rate%
-48.6% 3883482 ? 15% perf-stat.i.cache-misses
-73.8% 4797849 ? 10% perf-stat.i.context-switches
3.00 ? 7% perf-stat.i.cpi
-1.4% 9.855e+10 perf-stat.i.cpu-cycles
302970 ? 5% perf-stat.i.cycles-between-cache-misses
0.00 ? 5% perf-stat.i.dTLB-load-miss-rate%
-97.0% 168530 ? 11% perf-stat.i.dTLB-load-misses
-67.7% 9.412e+09 ? 7% perf-stat.i.dTLB-loads
0.00 ? 8% perf-stat.i.dTLB-store-miss-rate%
46788 ? 18% perf-stat.i.dTLB-store-misses
-71.8% 4.938e+09 ? 9% perf-stat.i.dTLB-stores
-66.7% 3.386e+10 ? 7% perf-stat.i.instructions
0.34 ? 7% perf-stat.i.ipc
0.44 perf-stat.i.metric.GHz
-30.0% 209.39 ? 10% perf-stat.i.metric.K/sec
95.93 ? 8% perf-stat.i.metric.M/sec
3576 ? 4% perf-stat.i.minor-faults
94.61 perf-stat.i.node-load-miss-rate%
-47.4% 595531 ? 25% perf-stat.i.node-load-misses
35044 ? 23% perf-stat.i.node-loads
3576 ? 4% perf-stat.i.page-faults
1.23 ? 2% perf-stat.overall.MPKI
9.34 ? 6% perf-stat.overall.cache-miss-rate%
2.93 ? 7% perf-stat.overall.cpi
25864 ? 15% perf-stat.overall.cycles-between-cache-misses
0.00 ? 6% perf-stat.overall.dTLB-load-miss-rate%
0.00 ? 10% perf-stat.overall.dTLB-store-miss-rate%
0.34 ? 7% perf-stat.overall.ipc
93.95 perf-stat.overall.node-load-miss-rate%
+27.6% 4250478 ? 2% perf-stat.overall.path-length
-65.5% 7.117e+09 ? 7% perf-stat.ps.branch-instructions
-66.0% 45185662 ? 8% perf-stat.ps.branch-misses
-48.3% 3888889 ? 15% perf-stat.ps.cache-misses
-73.8% 4781904 ? 10% perf-stat.ps.context-switches
-1.4% 9.823e+10 perf-stat.ps.cpu-cycles
-97.0% 168873 ? 11% perf-stat.ps.dTLB-load-misses
-67.7% 9.384e+09 ? 7% perf-stat.ps.dTLB-loads
46843 ? 19% perf-stat.ps.dTLB-store-misses
-71.8% 4.923e+09 ? 9% perf-stat.ps.dTLB-stores
-66.7% 3.376e+10 ? 7% perf-stat.ps.instructions
3577 ? 4% perf-stat.ps.minor-faults
-47.3% 593739 ? 25% perf-stat.ps.node-load-misses
36005 ? 22% perf-stat.ps.node-loads
3577 ? 4% perf-stat.ps.page-faults
-66.7% 1.018e+13 ? 7% perf-stat.total.instructions
14.42 ? 13% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
14.34 ? 13% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
13.07 ? 13% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
8.15 ? 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
7.40 ? 13% perf-profile.calltrace.cycles-pp.__libc_write
7.01 ? 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
6.97 ? 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
6.90 ? 13% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
8.64 ? 10% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
6.65 ? 13% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
6.47 ? 13% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.84 ? 13% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
5.86 ? 13% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
7.19 ? 13% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
5.73 ? 13% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
5.69 ? 13% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
7.05 ? 13% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
5.64 ? 13% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
5.79 ? 13% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
6.43 ? 13% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
3.75 ? 12% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
3.64 ? 12% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
4.89 ? 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
3.55 ? 12% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
4.78 ? 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
6.08 ? 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
5.95 ? 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
4.62 ? 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
5.82 ? 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
4.58 ? 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
5.77 ? 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
3.85 ? 14% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
4.94 ? 13% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
4.95 ? 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
4.81 ? 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
3.91 ? 13% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
4.68 ? 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
10.31 ? 5% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
8.53 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
8.25 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
0.76 ? 42% perf-profile.calltrace.cycles-pp.perf_output_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
3.08 ? 11% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
1.23 ? 15% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
1.07 ? 15% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
1.44 ? 18% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.81 ? 15% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry
0.76 ? 15% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle
0.63 ? 15% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue
0.34 ? 82% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.86 ? 29% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.64 ? 13% perf-profile.calltrace.cycles-pp.perf_output_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
0.90 perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
1.05 ? 21% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init
1.05 ? 21% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.rest_init
0.67 ? 7% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry
43.24 perf-profile.calltrace.cycles-pp.__libc_read
42.12 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
42.01 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
41.80 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
41.56 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
41.23 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.84 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
39.70 ? 2% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
44.37 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
44.35 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
44.27 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
45.63 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
31.89 ? 3% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
31.76 ? 3% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
24.34 ? 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
25.13 ? 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
25.28 ? 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
25.54 ? 8% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
25.88 ? 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
26.10 ? 8% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
19.49 ? 13% perf-profile.children.cycles-pp.perf_tp_event
18.97 ? 13% perf-profile.children.cycles-pp.__perf_event_overflow
18.82 ? 13% perf-profile.children.cycles-pp.perf_event_output_forward
15.72 ? 13% perf-profile.children.cycles-pp.perf_prepare_sample
14.66 ? 13% perf-profile.children.cycles-pp.perf_callchain
14.59 ? 13% perf-profile.children.cycles-pp.get_perf_callchain
13.46 ? 13% perf-profile.children.cycles-pp.perf_callchain_kernel
10.65 ? 13% perf-profile.children.cycles-pp.unwind_next_frame
8.91 ? 13% perf-profile.children.cycles-pp.perf_trace_sched_switch
9.70 ? 10% perf-profile.children.cycles-pp.__libc_write
8.98 ? 10% perf-profile.children.cycles-pp.ksys_write
8.69 ? 10% perf-profile.children.cycles-pp.vfs_write
6.50 ? 13% perf-profile.children.cycles-pp.pipe_write
7.05 ? 13% perf-profile.children.cycles-pp.__unwind_start
5.87 ? 13% perf-profile.children.cycles-pp.__wake_up_common_lock
7.20 ? 13% perf-profile.children.cycles-pp.dequeue_task_fair
5.73 ? 13% perf-profile.children.cycles-pp.__wake_up_common
7.10 ? 13% perf-profile.children.cycles-pp.dequeue_entity
5.70 ? 13% perf-profile.children.cycles-pp.autoremove_wake_function
5.67 ? 13% perf-profile.children.cycles-pp.try_to_wake_up
6.58 ? 13% perf-profile.children.cycles-pp.update_curr
51.32 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
4.90 ? 13% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
51.19 perf-profile.children.cycles-pp.do_syscall_64
6.18 ? 12% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
10.47 ? 5% perf-profile.children.cycles-pp.cpuidle_idle_call
2.91 ? 13% perf-profile.children.cycles-pp.__orc_find
2.41 ? 13% perf-profile.children.cycles-pp.perf_output_sample
1.86 ? 14% perf-profile.children.cycles-pp.__output_copy
1.91 ? 13% perf-profile.children.cycles-pp.stack_access_ok
8.65 ? 3% perf-profile.children.cycles-pp.cpuidle_enter
8.62 ? 3% perf-profile.children.cycles-pp.cpuidle_enter_state
1.60 ? 13% perf-profile.children.cycles-pp.perf_output_copy
3.24 ? 10% perf-profile.children.cycles-pp.poll_idle
1.58 ? 12% perf-profile.children.cycles-pp.unwind_get_return_address
1.27 ? 14% perf-profile.children.cycles-pp.flush_smp_call_function_queue
1.31 ? 13% perf-profile.children.cycles-pp.sched_ttwu_pending
1.42 ? 13% perf-profile.children.cycles-pp.orc_find
1.34 ? 13% perf-profile.children.cycles-pp.__kernel_text_address
1.49 ? 18% perf-profile.children.cycles-pp.menu_select
0.98 ? 13% perf-profile.children.cycles-pp.ttwu_do_activate
0.93 ? 13% perf-profile.children.cycles-pp.enqueue_task_fair
0.90 ? 14% perf-profile.children.cycles-pp.memcpy_erms
0.97 ? 13% perf-profile.children.cycles-pp.kernel_text_address
0.81 ? 13% perf-profile.children.cycles-pp.enqueue_entity
0.89 ? 14% perf-profile.children.cycles-pp.update_load_avg
0.72 ? 13% perf-profile.children.cycles-pp.perf_callchain_user
0.83 ? 8% perf-profile.children.cycles-pp.pick_next_task_fair
0.79 ? 8% perf-profile.children.cycles-pp.sched_clock_cpu
0.89 ? 28% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.60 ? 13% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.49 ? 13% perf-profile.children.cycles-pp.__get_user_nocheck_8
0.46 ? 13% perf-profile.children.cycles-pp.__switch_to
0.61 ? 8% perf-profile.children.cycles-pp.native_sched_clock
0.48 ? 10% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.38 ? 12% perf-profile.children.cycles-pp.__switch_to_asm
0.83 ? 16% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.45 ? 12% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.40 ? 13% perf-profile.children.cycles-pp.set_next_entity
0.25 ? 16% perf-profile.children.cycles-pp.copy_page_from_iter
0.39 ? 13% perf-profile.children.cycles-pp.prepare_to_wait_event
0.51 ? 7% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.37 ? 12% perf-profile.children.cycles-pp.__entry_text_start
0.64 ? 36% perf-profile.children.cycles-pp.tick_nohz_next_event
0.22 ? 16% perf-profile.children.cycles-pp._copy_from_iter
0.73 ? 16% perf-profile.children.cycles-pp.copyin
0.33 ? 13% perf-profile.children.cycles-pp.core_kernel_text
0.32 ? 13% perf-profile.children.cycles-pp.mutex_lock
0.22 ? 12% perf-profile.children.cycles-pp.copy_page_to_iter
0.35 ? 10% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.33 ? 10% perf-profile.children.cycles-pp.__fdget_pos
0.33 ? 8% perf-profile.children.cycles-pp.update_rq_clock
0.29 ? 11% perf-profile.children.cycles-pp.__fget_light
0.29 ? 14% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.35 ? 13% perf-profile.children.cycles-pp.__update_load_avg_se
0.18 ? 14% perf-profile.children.cycles-pp._copy_to_iter
0.33 ? 7% perf-profile.children.cycles-pp.sysvec_call_function_single
1.41 ? 2% perf-profile.children.cycles-pp.finish_task_switch
0.20 ? 14% perf-profile.children.cycles-pp.kvm_guest_state
0.28 ? 13% perf-profile.children.cycles-pp.select_task_rq
0.26 ? 10% perf-profile.children.cycles-pp.__fget_files
0.22 ? 13% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.20 ? 12% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.20 ? 13% perf-profile.children.cycles-pp.os_xsave
0.21 ? 14% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.23 ? 14% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.63 ? 14% perf-profile.children.cycles-pp.ktime_get
0.17 ? 11% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.19 ? 13% perf-profile.children.cycles-pp.get_stack_info
0.24 ? 12% perf-profile.children.cycles-pp.select_task_rq_fair
0.64 ? 16% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.19 ? 15% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.27 ? 6% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.17 ? 11% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.20 ? 8% perf-profile.children.cycles-pp.__might_resched
0.19 ? 11% perf-profile.children.cycles-pp.perf_trace_buf_update
0.17 ? 15% perf-profile.children.cycles-pp.security_file_permission
0.09 ? 14% perf-profile.children.cycles-pp.__might_fault
0.17 ? 11% perf-profile.children.cycles-pp.memcpy
0.26 ? 11% perf-profile.children.cycles-pp.___perf_sw_event
0.16 ? 11% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.12 ? 16% perf-profile.children.cycles-pp.touch_atime
0.13 ? 15% perf-profile.children.cycles-pp.get_callchain_entry
0.23 ? 5% perf-profile.children.cycles-pp.read_tsc
0.15 ? 12% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.13 ? 12% perf-profile.children.cycles-pp.update_cfs_group
0.12 ? 13% perf-profile.children.cycles-pp.copyout
0.14 ? 14% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.13 ? 17% perf-profile.children.cycles-pp.apparmor_file_permission
0.12 ? 13% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.10 ? 14% perf-profile.children.cycles-pp.atime_needs_update
0.12 ? 8% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.17 ? 7% perf-profile.children.cycles-pp.ct_idle_exit
0.10 ? 14% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.10 ? 10% perf-profile.children.cycles-pp.fput
0.11 ? 12% perf-profile.children.cycles-pp.reweight_entity
0.14 ? 11% perf-profile.children.cycles-pp.mutex_unlock
0.10 ? 15% perf-profile.children.cycles-pp.__might_sleep
0.11 ? 15% perf-profile.children.cycles-pp.perf_misc_flags
0.09 ? 12% perf-profile.children.cycles-pp.perf_instruction_pointer
0.09 ? 9% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.11 ? 10% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test
0.10 ? 14% perf-profile.children.cycles-pp.finish_wait
0.10 ? 14% perf-profile.children.cycles-pp.current_time
0.21 ? 8% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.10 ? 11% perf-profile.children.cycles-pp.perf_swevent_event
0.50 ? 8% perf-profile.children.cycles-pp.perf_output_begin_forward
0.09 ? 9% perf-profile.children.cycles-pp.perf_output_end
0.10 ? 13% perf-profile.children.cycles-pp.get_stack_info_noinstr
0.10 ? 14% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.11 ? 15% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.10 ? 11% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit
0.09 ? 10% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.09 ? 14% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.11 ? 8% perf-profile.children.cycles-pp.__cond_resched
0.11 ? 13% perf-profile.children.cycles-pp.__is_insn_slot_addr
0.11 ? 9% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.10 ? 14% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.06 ? 10% perf-profile.children.cycles-pp.place_entity
0.12 ? 8% perf-profile.children.cycles-pp.ct_kernel_enter
0.48 ? 7% perf-profile.children.cycles-pp.shmem_get_folio_gfp
0.10 ? 12% perf-profile.children.cycles-pp.file_update_time
0.08 ? 21% perf-profile.children.cycles-pp.update_min_vruntime
0.06 ? 14% perf-profile.children.cycles-pp.put_prev_task_fair
0.06 ? 12% perf-profile.children.cycles-pp.switch_fpu_return
0.07 ? 15% perf-profile.children.cycles-pp.__calc_delta
0.07 ? 10% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context
0.06 ? 12% perf-profile.children.cycles-pp.testcase
0.07 ? 18% perf-profile.children.cycles-pp.pick_next_entity
0.11 ? 15% perf-profile.children.cycles-pp.select_idle_sibling
0.08 ? 11% perf-profile.children.cycles-pp.__smp_call_single_queue
0.06 ? 12% perf-profile.children.cycles-pp.__list_del_entry_valid
0.06 ? 14% perf-profile.children.cycles-pp.aa_file_perm
0.06 ? 14% perf-profile.children.cycles-pp.__perf_event_account_interrupt
0.07 ? 13% perf-profile.children.cycles-pp.is_bpf_text_address
0.06 ? 18% perf-profile.children.cycles-pp.pick_next_task_idle
0.04 ? 50% perf-profile.children.cycles-pp.in_task_stack
0.06 ? 12% perf-profile.children.cycles-pp.__list_add_valid
0.04 ? 50% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.08 ? 13% perf-profile.children.cycles-pp.__pthread_enable_asynccancel
0.08 ? 13% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.05 ? 52% perf-profile.children.cycles-pp.check_preempt_curr
0.03 ? 82% perf-profile.children.cycles-pp.rb_next
0.04 ? 50% perf-profile.children.cycles-pp.cpuacct_charge
0.25 ? 4% perf-profile.children.cycles-pp.native_irq_return_iret
0.03 ? 84% perf-profile.children.cycles-pp.__update_idle_core
0.06 ? 12% perf-profile.children.cycles-pp.rcu_note_context_switch
0.11 ? 10% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.03 ? 82% perf-profile.children.cycles-pp.is_module_text_address
0.12 ? 15% perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.04 ? 50% perf-profile.children.cycles-pp.__get_task_ioprio
0.07 ? 7% perf-profile.children.cycles-pp.llist_add_batch
0.10 ? 32% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.06 ? 12% perf-profile.children.cycles-pp.sched_clock
0.06 ? 12% perf-profile.children.cycles-pp.tick_nohz_tick_stopped
0.05 ? 9% perf-profile.children.cycles-pp.send_call_function_single_ipi
0.04 ? 50% perf-profile.children.cycles-pp.is_ftrace_trampoline
0.04 ? 50% perf-profile.children.cycles-pp.schedule_debug
0.13 ? 27% perf-profile.children.cycles-pp.ordered_events__queue
0.09 ? 33% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.06 ? 6% perf-profile.children.cycles-pp.rcu_all_qs
0.09 ? 14% perf-profile.children.cycles-pp.shmem_alloc_folio
0.14 ? 5% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.09 ? 13% perf-profile.children.cycles-pp.vma_alloc_folio
0.04 ? 51% perf-profile.children.cycles-pp.get_page_from_freelist
0.05 ? 7% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.06 ? 12% perf-profile.children.cycles-pp.folio_add_lru
0.07 ? 10% perf-profile.children.cycles-pp.__folio_alloc
0.07 ? 10% perf-profile.children.cycles-pp.__alloc_pages
0.06 ? 12% perf-profile.children.cycles-pp.folio_batch_move_lru
0.10 ? 4% perf-profile.children.cycles-pp.irqtime_account_irq
0.09 ? 17% perf-profile.children.cycles-pp.__filemap_get_folio
0.14 ? 17% perf-profile.children.cycles-pp.perf_session__deliver_event
0.14 ? 18% perf-profile.children.cycles-pp.perf_session__process_user_event
0.14 ? 18% perf-profile.children.cycles-pp.__ordered_events__flush
0.09 ? 14% perf-profile.children.cycles-pp.update_sg_lb_stats
0.11 ? 15% perf-profile.children.cycles-pp.update_sd_lb_stats
0.12 ? 13% perf-profile.children.cycles-pp.find_busiest_group
0.07 ? 12% perf-profile.children.cycles-pp.folio_mark_accessed
0.15 ? 11% perf-profile.children.cycles-pp.load_balance
0.08 ? 27% perf-profile.children.cycles-pp.__perf_mmap__read_init
0.08 ? 20% perf-profile.children.cycles-pp.update_blocked_averages
0.13 ? 27% perf-profile.children.cycles-pp.perf_mmap__read_init
0.23 ? 18% perf-profile.children.cycles-pp.ring_buffer_read_head
0.24 ? 17% perf-profile.children.cycles-pp.perf_mmap__read_head
0.69 ? 7% perf-profile.children.cycles-pp.switch_mm_irqs_off
43.40 perf-profile.children.cycles-pp.__libc_read
41.84 perf-profile.children.cycles-pp.ksys_read
41.60 perf-profile.children.cycles-pp.vfs_read
41.30 perf-profile.children.cycles-pp.pipe_read
39.86 perf-profile.children.cycles-pp.schedule
44.37 perf-profile.children.cycles-pp.start_secondary
45.63 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
45.63 perf-profile.children.cycles-pp.cpu_startup_entry
45.57 perf-profile.children.cycles-pp.do_idle
32.96 ? 4% perf-profile.children.cycles-pp.schedule_idle
72.65 ? 2% perf-profile.children.cycles-pp.__schedule
49.57 ? 8% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
51.66 ? 8% perf-profile.children.cycles-pp.prepare_task_switch
51.44 ? 8% perf-profile.children.cycles-pp._raw_spin_lock
4.59 ? 13% perf-profile.self.cycles-pp.unwind_next_frame
2.77 ? 13% perf-profile.self.cycles-pp.__orc_find
1.54 ? 13% perf-profile.self.cycles-pp.stack_access_ok
1.27 ? 13% perf-profile.self.cycles-pp.orc_find
3.00 ? 10% perf-profile.self.cycles-pp.poll_idle
0.82 ? 15% perf-profile.self.cycles-pp.memcpy_erms
0.80 ? 14% perf-profile.self.cycles-pp.__output_copy
1.32 perf-profile.self.cycles-pp.__schedule
0.64 ? 12% perf-profile.self.cycles-pp.__unwind_start
0.61 ? 14% perf-profile.self.cycles-pp.perf_callchain_kernel
0.47 ? 13% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.50 ? 13% perf-profile.self.cycles-pp.perf_output_sample
0.58 ? 8% perf-profile.self.cycles-pp.native_sched_clock
0.44 ? 13% perf-profile.self.cycles-pp.__switch_to
0.37 ? 12% perf-profile.self.cycles-pp.__switch_to_asm
0.46 ? 10% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.81 ? 15% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.37 ? 14% perf-profile.self.cycles-pp.kernel_text_address
0.93 perf-profile.self.cycles-pp.finish_task_switch
0.46 ? 8% perf-profile.self.cycles-pp.menu_select
0.28 ? 12% perf-profile.self.cycles-pp.perf_tp_event
0.29 ? 14% perf-profile.self.cycles-pp.get_perf_callchain
0.31 ? 13% perf-profile.self.cycles-pp.unwind_get_return_address
0.21 ? 15% perf-profile.self.cycles-pp.pipe_read
0.29 ? 11% perf-profile.self.cycles-pp.perf_prepare_sample
0.27 ? 13% perf-profile.self.cycles-pp.enqueue_entity
0.31 ? 12% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.28 ? 14% perf-profile.self.cycles-pp.update_load_avg
0.30 ? 12% perf-profile.self.cycles-pp.__kernel_text_address
0.26 ? 14% perf-profile.self.cycles-pp.do_idle
0.26 ? 9% perf-profile.self.cycles-pp.cpuidle_idle_call
0.27 ? 14% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.24 ? 13% perf-profile.self.cycles-pp.core_kernel_text
0.25 ? 12% perf-profile.self.cycles-pp.__fget_files
0.31 ? 13% perf-profile.self.cycles-pp.__update_load_avg_se
0.20 ? 14% perf-profile.self.cycles-pp.prepare_to_wait_event
0.19 ? 12% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.21 ? 7% perf-profile.self.cycles-pp.__libc_write
0.20 ? 13% perf-profile.self.cycles-pp.os_xsave
0.19 ? 14% perf-profile.self.cycles-pp.perf_callchain_user
0.20 ? 15% perf-profile.self.cycles-pp.perf_output_copy
0.19 ? 14% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.20 ? 13% perf-profile.self.cycles-pp.__libc_read
0.20 ? 20% perf-profile.self.cycles-pp.update_curr
0.16 ? 14% perf-profile.self.cycles-pp.kvm_guest_state
0.17 ? 10% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.20 ? 13% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.13 ? 13% perf-profile.self.cycles-pp.pipe_write
0.18 ? 8% perf-profile.self.cycles-pp.__might_resched
0.14 ? 14% perf-profile.self.cycles-pp.vfs_read
0.14 ? 13% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.22 ? 5% perf-profile.self.cycles-pp.read_tsc
0.19 ? 15% perf-profile.self.cycles-pp.dequeue_entity
0.12 ? 16% perf-profile.self.cycles-pp.get_callchain_entry
0.14 ? 14% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.12 ? 15% perf-profile.self.cycles-pp.update_cfs_group
0.12 ? 13% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.09 ? 10% perf-profile.self.cycles-pp.try_to_wake_up
0.13 ? 12% perf-profile.self.cycles-pp.enqueue_task_fair
0.08 ? 14% perf-profile.self.cycles-pp.ksys_read
0.11 ? 11% perf-profile.self.cycles-pp.pick_next_task_fair
0.11 ? 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.13 ? 15% perf-profile.self.cycles-pp.perf_event_output_forward
0.14 ? 15% perf-profile.self.cycles-pp.mutex_lock
0.09 ? 15% perf-profile.self.cycles-pp.do_syscall_64
0.10 ? 14% perf-profile.self.cycles-pp.__perf_event_overflow
0.10 ? 13% perf-profile.self.cycles-pp.reweight_entity
0.12 ? 8% perf-profile.self.cycles-pp.update_rq_clock
0.13 ? 10% perf-profile.self.cycles-pp.mutex_unlock
0.08 ? 14% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.10 ? 14% perf-profile.self.cycles-pp.dequeue_task_fair
0.10 ? 11% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test
0.21 ? 13% perf-profile.self.cycles-pp.___perf_sw_event
0.09 ? 12% perf-profile.self.cycles-pp.fput
0.19 ? 9% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.08 ? 13% perf-profile.self.cycles-pp.__might_sleep
0.10 ? 13% perf-profile.self.cycles-pp.set_next_entity
0.14 ? 12% perf-profile.self.cycles-pp.sched_clock_cpu
0.09 ? 15% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.49 ? 7% perf-profile.self.cycles-pp.perf_output_begin_forward
0.08 ? 11% perf-profile.self.cycles-pp.newidle_balance
0.05 ? 52% perf-profile.self.cycles-pp._copy_from_iter
0.08 ? 9% perf-profile.self.cycles-pp.perf_output_end
0.15 ? 38% perf-profile.self.cycles-pp.tick_nohz_next_event
0.09 ? 11% perf-profile.self.cycles-pp.schedule_idle
0.10 ? 7% perf-profile.self.cycles-pp.vfs_write
0.41 ? 4% perf-profile.self.cycles-pp.cpuidle_enter_state
0.08 ? 13% perf-profile.self.cycles-pp.perf_callchain
0.08 ? 15% perf-profile.self.cycles-pp.perf_swevent_event
0.09 ? 14% perf-profile.self.cycles-pp.get_stack_info
0.08 ? 15% perf-profile.self.cycles-pp.schedule
0.08 ? 9% perf-profile.self.cycles-pp.memcpy
0.08 ? 12% perf-profile.self.cycles-pp.perf_trace_buf_alloc
0.10 ? 6% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.09 ? 8% perf-profile.self.cycles-pp.perf_trace_run_bpf_submit
0.08 ? 14% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.07 ? 18% perf-profile.self.cycles-pp.apparmor_file_permission
0.07 ? 15% perf-profile.self.cycles-pp.sched_ttwu_pending
0.08 ? 14% perf-profile.self.cycles-pp.__entry_text_start
0.07 ? 16% perf-profile.self.cycles-pp.perf_trace_buf_update
0.07 ? 16% perf-profile.self.cycles-pp.cpu_startup_entry
0.06 ? 14% perf-profile.self.cycles-pp.switch_fpu_return
0.13 ? 15% perf-profile.self.cycles-pp.select_task_rq_fair
0.20 ? 17% perf-profile.self.cycles-pp.prepare_task_switch
0.09 ? 13% perf-profile.self.cycles-pp.__is_insn_slot_addr
0.03 ? 82% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.06 ? 12% perf-profile.self.cycles-pp.current_time
0.06 ? 12% perf-profile.self.cycles-pp.testcase
0.07 ? 15% perf-profile.self.cycles-pp.__calc_delta
0.06 ? 15% perf-profile.self.cycles-pp.perf_misc_flags
0.08 ? 9% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.08 ? 18% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.06 ? 12% perf-profile.self.cycles-pp.pick_next_entity
0.06 ? 14% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.05 ? 52% perf-profile.self.cycles-pp.aa_file_perm
0.04 ? 51% perf-profile.self.cycles-pp.__perf_event_account_interrupt
0.06 ? 21% perf-profile.self.cycles-pp.update_min_vruntime
0.06 ? 12% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.07 ? 13% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.07 ? 15% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.25 ? 4% perf-profile.self.cycles-pp.native_irq_return_iret
0.06 ? 12% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.04 ? 51% perf-profile.self.cycles-pp.send_call_function_single_ipi
0.03 ? 82% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.06 ? 14% perf-profile.self.cycles-pp.llist_add_batch
0.04 ? 50% perf-profile.self.cycles-pp.__list_add_valid
0.04 ? 50% perf-profile.self.cycles-pp.cpuacct_charge
0.06 ? 10% perf-profile.self.cycles-pp.ct_idle_exit
0.05 ? 9% perf-profile.self.cycles-pp.get_stack_info_noinstr
0.07 ? 9% perf-profile.self.cycles-pp.__pthread_enable_asynccancel
0.06 ? 14% perf-profile.self.cycles-pp.rcu_note_context_switch
0.04 ? 51% perf-profile.self.cycles-pp.ct_kernel_enter
0.06 ? 6% perf-profile.self.cycles-pp.__cond_resched
0.04 ? 50% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.12 ? 27% perf-profile.self.cycles-pp.queue_event
0.07 ? 63% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.13 ? 3% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.05 ? 7% perf-profile.self.cycles-pp.__x2apic_send_IPI_dest
0.07 ? 13% perf-profile.self.cycles-pp.update_sg_lb_stats
0.21 ? 18% perf-profile.self.cycles-pp.ring_buffer_read_head
0.69 ? 6% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.84 perf-profile.self.cycles-pp._raw_spin_lock
49.57 ? 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
*************************************************************************
threads 2 sockets (Skylake) with 192G memory
===============================================================
kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
4-rhel-8.3/thread/16/debian-11.1-x86_64-20220510.cgz/lkp-skl-fpga01/context_switch1/will-it-scale
Implement rseq numa node id field selftest")
Introduce per-memory-map concurrency ID")
af7f588d8f7355bc4298dd1962d
---------------------------
fail:runs
|
:6 kmsg.EDAC_skx:Can't_find_handle_for_NVDIMM_ADR=
:6 kmsg.microcode:Attempting_late_microcode_loading-it_is_dangerous_and_taints_the_kernel
:6 kmsg.microcode:You_should_switch_to_early_loading,if_possible
%stddev
\
-57.9% 1818800 will-it-scale.16.threads
-57.9% 113674 will-it-scale.per_thread_ops
-57.9% 1818800 will-it-scale.workload
-58.0% 5.581e+08 cpuidle..usage
15.22 mpstat.cpu.all.sys%
2.57 ? 4% mpstat.cpu.all.usr%
17.00 vmstat.procs.r
-58.0% 3593440 vmstat.system.cs
+16.0% 270468 ? 2% vmstat.system.in
448837 ? 18% turbostat.C1
0.07 turbostat.IPC
+16.1% 82393754 ? 2% turbostat.IRQ
-60.0% 5.137e+08 turbostat.POLL
1.00 ? 2% turbostat.POLL%
-8.1% 232.09 turbostat.PkgWatt
7417 ? 59% sched_debug.cfs_rq:/.MIN_vruntime.avg
-76.4% 359932 ? 59% sched_debug.cfs_rq:/.MIN_vruntime.max
49952 ? 56% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.29 ? 2% sched_debug.cfs_rq:/.h_nr_running.avg
25920 ? 3% sched_debug.cfs_rq:/.load.avg
92526 ? 68% sched_debug.cfs_rq:/.load.max
7417 ? 59% sched_debug.cfs_rq:/.max_vruntime.avg
-76.4% 359932 ? 59% sched_debug.cfs_rq:/.max_vruntime.max
49952 ? 56% sched_debug.cfs_rq:/.max_vruntime.stddev
-16.0% 564476 ? 2% sched_debug.cfs_rq:/.min_vruntime.avg
-17.8% 1267561 sched_debug.cfs_rq:/.min_vruntime.max
-18.9% 575136 sched_debug.cfs_rq:/.min_vruntime.stddev
0.28 ? 2% sched_debug.cfs_rq:/.nr_running.avg
-19.6% -682164 sched_debug.cfs_rq:/.spread0.avg
-18.5% -1211535 sched_debug.cfs_rq:/.spread0.min
-18.9% 575149 sched_debug.cfs_rq:/.spread0.stddev
-10.2% 192.45 ? 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
6526 ? 6% sched_debug.cpu.avg_idle.min
-14.8% 906.25 ? 5% sched_debug.cpu.curr->pid.avg
0.17 ? 2% sched_debug.cpu.nr_running.avg
-57.7% 5209198 sched_debug.cpu.nr_switches.avg
-57.8% 16957686 sched_debug.cpu.nr_switches.max
-57.7% 7805110 sched_debug.cpu.nr_switches.stddev
7.31 ? 14% perf-stat.i.MPKI
-51.2% 4.543e+09 ? 5% perf-stat.i.branch-instructions
-49.6% 79141069 ? 4% perf-stat.i.branch-misses
+129.9% 1.563e+08 ? 13% perf-stat.i.cache-references
-58.0% 3628862 perf-stat.i.context-switches
4.24 ? 2% perf-stat.i.cpi
+10.5% 168.99 ? 2% perf-stat.i.cpu-migrations
-54.4% 4473079 ? 12% perf-stat.i.dTLB-load-misses
-52.5% 6.241e+09 ? 4% perf-stat.i.dTLB-loads
-54.9% 3.549e+09 ? 4% perf-stat.i.dTLB-stores
-58.2% 3793161 ? 49% perf-stat.i.iTLB-load-misses
-50.6% 5566942 ? 8% perf-stat.i.iTLB-loads
-51.8% 2.214e+10 ? 5% perf-stat.i.instructions
0.24 ? 4% perf-stat.i.ipc
93.67 ? 4% perf-stat.i.metric.K/sec
-52.2% 139.32 ? 4% perf-stat.i.metric.M/sec
7.06 ? 12% perf-stat.overall.MPKI
4.14 ? 4% perf-stat.overall.cpi
0.24 ? 4% perf-stat.overall.ipc
+14.7% 3668175 ? 4% perf-stat.overall.path-length
-51.2% 4.528e+09 ? 5% perf-stat.ps.branch-instructions
-49.6% 78882571 ? 4% perf-stat.ps.branch-misses
+129.8% 1.557e+08 ? 13% perf-stat.ps.cache-references
-58.0% 3616809 perf-stat.ps.context-switches
+10.4% 168.54 ? 2% perf-stat.ps.cpu-migrations
-54.4% 4458181 ? 12% perf-stat.ps.dTLB-load-misses
-52.5% 6.221e+09 ? 4% perf-stat.ps.dTLB-loads
-54.9% 3.538e+09 ? 4% perf-stat.ps.dTLB-stores
-58.2% 3780389 ? 49% perf-stat.ps.iTLB-load-misses
-50.6% 5548664 ? 8% perf-stat.ps.iTLB-loads
-51.8% 2.207e+10 ? 5% perf-stat.ps.instructions
-51.7% 6.671e+12 ? 4% perf-stat.total.instructions
11.78 ? 32% perf-profile.calltrace.cycles-pp.__libc_write
10.34 ? 8% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.58 ? 7% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.25 ? 11% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
2.18 ? 8% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
1.84 ? 12% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry
1.78 ? 12% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle
1.31 ? 6% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read
1.28 ? 7% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write
1.33 ? 6% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
1.05 ? 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
1.15 ? 4% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
0.94 ? 12% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.83 ? 3% perf-profile.calltrace.cycles-pp.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.36 ? 71% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary
0.30 ?100% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
0.26 ?100% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_read
0.27 ?100% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_write
0.28 ?100% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.vfs_write.ksys_write.do_syscall_64
0.67 ? 4% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule_idle.do_idle
0.83 ? 11% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry
43.39 ? 2% perf-profile.calltrace.cycles-pp.__libc_read
41.18 ? 10% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
41.17 ? 10% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
41.04 ? 10% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
42.35 ? 10% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
40.21 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
39.36 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
37.80 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
37.42 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
36.84 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
34.64 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
34.56 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
26.25 ? 18% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
26.10 ? 18% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
18.81 ? 49% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
19.34 ? 47% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
19.72 ? 46% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
19.76 ? 45% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
20.31 ? 44% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
20.67 ? 43% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
13.08 ? 38% perf-profile.children.cycles-pp.__libc_write
10.61 ? 7% perf-profile.children.cycles-pp.cpuidle_idle_call
2.66 ? 7% perf-profile.children.cycles-pp.flush_smp_call_function_queue
2.67 ? 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
2.61 ? 4% perf-profile.children.cycles-pp.sched_ttwu_pending
2.27 ? 8% perf-profile.children.cycles-pp.menu_select
2.55 ? 4% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
2.13 ? 4% perf-profile.children.cycles-pp.ttwu_do_activate
2.06 ? 5% perf-profile.children.cycles-pp.enqueue_task_fair
1.59 ? 8% perf-profile.children.cycles-pp.pick_next_task_fair
1.75 ? 6% perf-profile.children.cycles-pp.enqueue_entity
1.25 ? 3% perf-profile.children.cycles-pp.update_load_avg
1.15 ? 4% perf-profile.children.cycles-pp.__entry_text_start
0.98 ? 12% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.66 ? 13% perf-profile.children.cycles-pp.__switch_to_asm
0.51 ? 9% perf-profile.children.cycles-pp.__switch_to
0.95 ? 5% perf-profile.children.cycles-pp.ktime_get
0.88 ? 3% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.74 ? 2% perf-profile.children.cycles-pp.set_next_entity
0.26 ? 20% perf-profile.children.cycles-pp.os_xsave
0.55 ? 16% perf-profile.children.cycles-pp.tick_nohz_next_event
0.52 ? 7% perf-profile.children.cycles-pp.mutex_lock
0.79 ? 12% perf-profile.children.cycles-pp.sched_clock_cpu
0.58 ? 8% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.52 ? 2% perf-profile.children.cycles-pp.read_tsc
0.41 ? 5% perf-profile.children.cycles-pp.select_task_rq
0.52 ? 10% perf-profile.children.cycles-pp.copy_page_from_iter
0.66 ? 9% perf-profile.children.cycles-pp.native_sched_clock
0.47 ? 3% perf-profile.children.cycles-pp.__fdget_pos
0.45 ? 3% perf-profile.children.cycles-pp.__fget_light
0.42 ? 6% perf-profile.children.cycles-pp.security_file_permission
0.45 ? 5% perf-profile.children.cycles-pp.prepare_to_wait_event
0.35 ? 5% perf-profile.children.cycles-pp.__update_load_avg_se
0.32 ? 10% perf-profile.children.cycles-pp.copy_page_to_iter
0.34 ? 7% perf-profile.children.cycles-pp.select_task_rq_fair
0.26 ? 21% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.32 ? 11% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.37 ? 4% perf-profile.children.cycles-pp.apparmor_file_permission
0.43 ? 12% perf-profile.children.cycles-pp._copy_from_iter
0.26 ? 6% perf-profile.children.cycles-pp._copy_to_iter
0.34 ? 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.34 ? 3% perf-profile.children.cycles-pp.__fget_files
0.42 ? 9% perf-profile.children.cycles-pp.update_cfs_group
0.32 ? 16% perf-profile.children.cycles-pp.reweight_entity
0.34 ? 7% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.36 ? 5% perf-profile.children.cycles-pp.update_rq_clock
0.16 ? 6% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.25 ? 5% perf-profile.children.cycles-pp.__might_resched
0.20 ? 36% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.29 ? 8% perf-profile.children.cycles-pp.___perf_sw_event
0.22 ? 5% perf-profile.children.cycles-pp.put_prev_task_fair
0.29 ? 20% perf-profile.children.cycles-pp.touch_atime
0.22 ? 12% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.22 ? 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.21 ? 9% perf-profile.children.cycles-pp.__might_fault
0.15 ? 6% perf-profile.children.cycles-pp.__might_sleep
0.15 ? 4% perf-profile.children.cycles-pp.mutex_unlock
0.15 ? 23% perf-profile.children.cycles-pp.__calc_delta
0.22 ? 23% perf-profile.children.cycles-pp.atime_needs_update
0.14 ? 6% perf-profile.children.cycles-pp.file_update_time
0.16 ? 8% perf-profile.children.cycles-pp.pick_next_entity
0.34 ? 6% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.19 ? 7% perf-profile.children.cycles-pp.__cond_resched
0.19 ? 6% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.09 ? 9% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.13 ? 12% perf-profile.children.cycles-pp.copyout
0.16 ? 24% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.18 ? 23% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.16 ? 15% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.15 ? 10% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.12 ? 8% perf-profile.children.cycles-pp.select_idle_sibling
0.14 ? 5% perf-profile.children.cycles-pp.copy_user_short_string
0.16 ? 9% perf-profile.children.cycles-pp.pick_next_task_idle
0.13 ? 5% perf-profile.children.cycles-pp.fput
0.11 ? 5% perf-profile.children.cycles-pp.cpuacct_charge
0.14 ? 6% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.14 ? 17% perf-profile.children.cycles-pp.current_time
0.12 ? 6% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.10 ? 8% perf-profile.children.cycles-pp.finish_wait
0.03 ?100% perf-profile.children.cycles-pp.llist_reverse_order
0.10 ? 7% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.12 ? 6% perf-profile.children.cycles-pp.put_prev_entity
0.09 ? 7% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.09 ? 19% perf-profile.children.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.14 ? 13% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.08 ? 10% perf-profile.children.cycles-pp.update_min_vruntime
0.11 ? 6% perf-profile.children.cycles-pp.tick_nohz_tick_stopped
0.08 ? 11% perf-profile.children.cycles-pp.switch_fpu_return
0.09 ? 13% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.06 ? 9% perf-profile.children.cycles-pp.__get_task_ioprio
0.10 ? 19% perf-profile.children.cycles-pp.check_preempt_curr
0.10 ? 16% perf-profile.children.cycles-pp.rcu_note_context_switch
0.09 ? 12% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.10 ? 23% perf-profile.children.cycles-pp.get_cpu_device
0.07 ? 20% perf-profile.children.cycles-pp.rb_erase
0.05 ? 45% perf-profile.children.cycles-pp.clear_buddies
0.12 ? 10% perf-profile.children.cycles-pp.__update_idle_core
0.08 ? 16% perf-profile.children.cycles-pp.llist_add_batch
0.09 ? 7% perf-profile.children.cycles-pp.testcase
0.09 ? 13% perf-profile.children.cycles-pp.cpuidle_not_available
0.10 ? 22% perf-profile.children.cycles-pp.__list_del_entry_valid
0.11 ? 6% perf-profile.children.cycles-pp.__smp_call_single_queue
0.09 ? 17% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.05 ? 7% perf-profile.children.cycles-pp.put_prev_task_idle
0.10 ? 33% perf-profile.children.cycles-pp.nr_iowait_cpu
0.04 ? 73% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.03 ?100% perf-profile.children.cycles-pp.schedule_debug
0.08 ? 7% perf-profile.children.cycles-pp.__pthread_enable_asynccancel
0.06 ? 11% perf-profile.children.cycles-pp.tick_nohz_idle_retain_tick
0.11 ? 6% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.03 ?100% perf-profile.children.cycles-pp.send_call_function_single_ipi
0.07 perf-profile.children.cycles-pp.ct_kernel_exit
0.06 ? 19% perf-profile.children.cycles-pp.__list_add_valid
0.06 ? 14% perf-profile.children.cycles-pp.__x64_sys_write
0.03 ?100% perf-profile.children.cycles-pp.init_wait_entry
0.06 ? 6% perf-profile.children.cycles-pp.__x64_sys_read
0.04 ? 71% perf-profile.children.cycles-pp.child
0.05 ? 8% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.07 ? 25% perf-profile.children.cycles-pp._find_first_zero_bit
0.85 ? 11% perf-profile.children.cycles-pp.switch_mm_irqs_off
43.50 ? 2% perf-profile.children.cycles-pp.__libc_read
41.18 ? 10% perf-profile.children.cycles-pp.start_secondary
42.35 ? 10% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
42.35 ? 10% perf-profile.children.cycles-pp.cpu_startup_entry
42.28 ? 10% perf-profile.children.cycles-pp.do_idle
37.84 perf-profile.children.cycles-pp.ksys_read
37.45 perf-profile.children.cycles-pp.vfs_read
36.90 perf-profile.children.cycles-pp.pipe_read
34.65 perf-profile.children.cycles-pp.schedule
27.08 ? 18% perf-profile.children.cycles-pp.schedule_idle
61.54 ? 8% perf-profile.children.cycles-pp.__schedule
38.77 ? 47% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
40.18 ? 45% perf-profile.children.cycles-pp._raw_spin_lock
40.61 ? 44% perf-profile.children.cycles-pp.prepare_task_switch
2.67 ? 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
2.36 ? 5% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
1.72 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.99 ? 4% perf-profile.self.cycles-pp.__entry_text_start
0.65 ? 12% perf-profile.self.cycles-pp.__switch_to_asm
0.98 ? 5% perf-profile.self.cycles-pp.menu_select
0.48 ? 9% perf-profile.self.cycles-pp.__switch_to
1.63 perf-profile.self.cycles-pp.__schedule
0.26 ? 20% perf-profile.self.cycles-pp.os_xsave
0.62 ? 4% perf-profile.self.cycles-pp.enqueue_entity
0.57 ? 8% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.55 ? 4% perf-profile.self.cycles-pp.update_load_avg
0.50 perf-profile.self.cycles-pp.read_tsc
0.65 ? 9% perf-profile.self.cycles-pp.native_sched_clock
0.45 ? 12% perf-profile.self.cycles-pp.ktime_get
0.36 ? 6% perf-profile.self.cycles-pp.pipe_read
0.29 ? 5% perf-profile.self.cycles-pp.dequeue_entity
0.40 ? 7% perf-profile.self.cycles-pp.pipe_write
0.37 ? 17% perf-profile.self.cycles-pp.do_idle
0.34 ? 5% perf-profile.self.cycles-pp.__update_load_avg_se
0.31 ? 6% perf-profile.self.cycles-pp.enqueue_task_fair
0.28 ? 5% perf-profile.self.cycles-pp.pick_next_task_fair
0.33 ? 4% perf-profile.self.cycles-pp.apparmor_file_permission
0.32 ? 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.33 ? 4% perf-profile.self.cycles-pp.__fget_files
0.31 ? 16% perf-profile.self.cycles-pp.reweight_entity
0.40 ? 8% perf-profile.self.cycles-pp.update_cfs_group
0.25 ? 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.26 ? 2% perf-profile.self.cycles-pp.set_next_entity
0.16 ? 6% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.25 ? 6% perf-profile.self.cycles-pp.__might_resched
0.20 ? 11% perf-profile.self.cycles-pp.vfs_write
0.23 ? 5% perf-profile.self.cycles-pp.prepare_to_wait_event
0.23 ? 6% perf-profile.self.cycles-pp.mutex_lock
0.21 ? 3% perf-profile.self.cycles-pp.vfs_read
0.18 ? 39% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.23 ? 6% perf-profile.self.cycles-pp.newidle_balance
0.22 ? 4% perf-profile.self.cycles-pp.__libc_write
0.28 ? 25% perf-profile.self.cycles-pp.update_curr
0.25 ? 11% perf-profile.self.cycles-pp.___perf_sw_event
0.19 ? 5% perf-profile.self.cycles-pp.select_task_rq_fair
0.22 ? 9% perf-profile.self.cycles-pp.__libc_read
0.14 ? 6% perf-profile.self.cycles-pp.mutex_unlock
0.14 ? 23% perf-profile.self.cycles-pp.__calc_delta
0.09 ? 9% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.12 ? 6% perf-profile.self.cycles-pp.__might_sleep
0.12 ? 7% perf-profile.self.cycles-pp.pick_next_entity
0.16 ? 8% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.11 ? 13% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.14 ? 38% perf-profile.self.cycles-pp.try_to_wake_up
0.14 ? 4% perf-profile.self.cycles-pp.schedule_idle
0.13 ? 4% perf-profile.self.cycles-pp.__cond_resched
0.14 ? 8% perf-profile.self.cycles-pp.copy_user_short_string
0.10 ? 8% perf-profile.self.cycles-pp.put_prev_task_fair
0.13 ? 14% perf-profile.self.cycles-pp.sched_ttwu_pending
0.15 ? 10% perf-profile.self.cycles-pp._copy_from_iter
0.16 ? 8% perf-profile.self.cycles-pp.do_syscall_64
0.08 ? 5% perf-profile.self.cycles-pp.copy_page_from_iter
0.14 ? 8% perf-profile.self.cycles-pp.tick_nohz_next_event
0.11 ? 8% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.11 ? 8% perf-profile.self.cycles-pp.cpuacct_charge
0.14 ? 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.15 ? 11% perf-profile.self.cycles-pp.update_rq_clock
0.11 ? 8% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.28 ? 17% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.10 ? 9% perf-profile.self.cycles-pp.fput
0.04 ? 44% perf-profile.self.cycles-pp.__get_task_ioprio
0.06 ? 11% perf-profile.self.cycles-pp._copy_to_iter
0.10 ? 3% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.07 ? 5% perf-profile.self.cycles-pp.select_task_rq
0.09 ? 21% perf-profile.self.cycles-pp.current_time
0.08 ? 7% perf-profile.self.cycles-pp.switch_fpu_return
0.09 ? 13% perf-profile.self.cycles-pp.autoremove_wake_function
0.06 ? 15% perf-profile.self.cycles-pp.ttwu_do_activate
0.08 ? 20% perf-profile.self.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.12 ? 9% perf-profile.self.cycles-pp.ksys_read
0.06 ? 9% perf-profile.self.cycles-pp.update_min_vruntime
0.07 ? 23% perf-profile.self.cycles-pp.check_preempt_curr
0.09 ? 7% perf-profile.self.cycles-pp.put_prev_entity
0.05 ? 46% perf-profile.self.cycles-pp.cpuidle_governor_latency_req
0.09 ? 13% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.09 ? 7% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.08 ? 8% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.13 ? 12% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.03 ? 70% perf-profile.self.cycles-pp.put_prev_task_idle
0.09 ? 14% perf-profile.self.cycles-pp.rcu_note_context_switch
0.09 ? 26% perf-profile.self.cycles-pp.ksys_write
0.10 ? 23% perf-profile.self.cycles-pp.get_cpu_device
0.10 ? 13% perf-profile.self.cycles-pp.__fget_light
0.08 ? 10% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.07 ? 10% perf-profile.self.cycles-pp.select_idle_sibling
0.12 ? 27% perf-profile.self.cycles-pp.sched_clock_cpu
0.08 ? 15% perf-profile.self.cycles-pp.llist_add_batch
0.09 ? 5% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.06 ? 21% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.08 ? 13% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.06 ? 47% perf-profile.self.cycles-pp.rb_erase
0.09 ? 13% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.09 ? 13% perf-profile.self.cycles-pp.__update_idle_core
0.08 ? 8% perf-profile.self.cycles-pp.testcase
0.09 ? 31% perf-profile.self.cycles-pp.nr_iowait_cpu
0.04 ? 45% perf-profile.self.cycles-pp.__wake_up_common
0.10 ? 36% perf-profile.self.cycles-pp.atime_needs_update
0.06 ? 13% perf-profile.self.cycles-pp.file_update_time
0.04 ? 70% perf-profile.self.cycles-pp.tick_nohz_idle_retain_tick
0.03 ?100% perf-profile.self.cycles-pp.schedule_debug
0.08 ? 12% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.09 ? 12% perf-profile.self.cycles-pp.cpuidle_not_available
0.07 ? 27% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.08 ? 7% perf-profile.self.cycles-pp.__pthread_enable_asynccancel
0.03 ?100% perf-profile.self.cycles-pp.send_call_function_single_ipi
0.08 ? 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.08 ? 16% perf-profile.self.cycles-pp.cpu_startup_entry
0.06 ? 19% perf-profile.self.cycles-pp.__list_add_valid
0.06 ? 14% perf-profile.self.cycles-pp.__x64_sys_write
0.03 ?100% perf-profile.self.cycles-pp.rb_insert_color
0.08 ? 8% perf-profile.self.cycles-pp.__might_fault
0.06 ? 6% perf-profile.self.cycles-pp.__x64_sys_read
0.05 ? 7% perf-profile.self.cycles-pp.ct_kernel_exit
0.06 ? 8% perf-profile.self.cycles-pp.touch_atime
0.04 ? 71% perf-profile.self.cycles-pp.child
0.07 ? 25% perf-profile.self.cycles-pp._find_first_zero_bit
0.85 ? 11% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.41 ? 2% perf-profile.self.cycles-pp._raw_spin_lock
38.76 ? 47% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
*************************************************************************
threads 2 sockets (Skylake) with 192G memory
===============================================================
kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
4-rhel-8.3/thread/50%/debian-11.1-x86_64-20220510.cgz/lkp-skl-fpga01/context_switch1/will-it-scale
Implement rseq numa node id field selftest")
Introduce per-memory-map concurrency ID")
af7f588d8f7355bc4298dd1962d
---------------------------
%stddev
\
-85.0% 1259200 will-it-scale.52.threads
3.74 will-it-scale.52.threads_idle
24214 will-it-scale.per_thread_ops
-85.0% 1259200 will-it-scale.workload
6040 ? 2% uptime.idle
-73.2% 5.989e+08 cpuidle..time
-89.5% 2.577e+08 ? 3% cpuidle..usage
+232.0% 1094566 ? 72% numa-numastat.node0.local_node
+205.7% 1145060 ? 69% numa-numastat.node0.numa_hit
-53.1% 2106039 ? 36% numa-numastat.node1.local_node
-52.7% 2149877 ? 35% numa-numastat.node1.numa_hit
4.43 ? 2% mpstat.cpu.all.idle%
1.68 mpstat.cpu.all.irq%
0.10 mpstat.cpu.all.soft%
90.11 mpstat.cpu.all.sys%
3.68 mpstat.cpu.all.usr%
5.00 vmstat.cpu.id
90.83 vmstat.cpu.sy
3.00 vmstat.cpu.us
-53.0% 6714843 ? 9% vmstat.memory.cache
52.17 vmstat.procs.r
-85.4% 2405190 vmstat.system.cs
+52.0% 420615 ? 2% vmstat.system.in
+1030.1% 16174000 ? 6% turbostat.C1
0.29 ? 7% turbostat.C1%
1.01 ? 4% turbostat.CPU%c1
45.83 ? 2% turbostat.CoreTmp
0.02 turbostat.IPC
+52.1% 1.284e+08 ? 2% turbostat.IRQ
0.00 turbostat.PKG_%
-90.2% 2.41e+08 ? 3% turbostat.POLL
0.69 ? 5% turbostat.POLL%
46.50 ? 3% turbostat.PkgTmp
-17.4% 301.75 turbostat.PkgWatt
1730551 ? 4% meminfo.Active
1730458 ? 4% meminfo.Active(anon)
-26.0% 351530 meminfo.AnonPages
-53.3% 6596409 ? 9% meminfo.Cached
-62.9% 4516022 ? 5% meminfo.Committed_AS
-83.8% 1794273 ? 16% meminfo.Inactive
-83.8% 1794082 ? 16% meminfo.Inactive(anon)
-11.5% 129954 meminfo.KReclaimable
-69.4% 1533664 ? 7% meminfo.Mapped
-48.9% 8206619 ? 7% meminfo.Memused
7796 ? 2% meminfo.PageTables
-11.5% 129954 meminfo.SReclaimable
-70.4% 3172409 ? 6% meminfo.Shmem
-25.6% 12012181 ? 4% meminfo.max_used_kB
118054 ? 98% numa-vmstat.node0.nr_active_anon
184580 ? 82% numa-vmstat.node0.nr_inactive_anon
137333 ? 75% numa-vmstat.node0.nr_mapped
248438 ?106% numa-vmstat.node0.nr_shmem
118054 ? 98% numa-vmstat.node0.nr_zone_active_anon
184580 ? 82% numa-vmstat.node0.nr_zone_inactive_anon
+206.0% 1144611 ? 69% numa-vmstat.node0.numa_hit
+232.4% 1094117 ? 72% numa-vmstat.node0.numa_local
313777 ? 42% numa-vmstat.node1.nr_active_anon
33711 ? 22% numa-vmstat.node1.nr_anon_pages
-78.4% 583457 ? 37% numa-vmstat.node1.nr_file_pages
-90.3% 264439 ? 35% numa-vmstat.node1.nr_inactive_anon
-79.6% 247430 ? 32% numa-vmstat.node1.nr_mapped
1188 ? 24% numa-vmstat.node1.nr_page_table_pages
-79.7% 544388 ? 41% numa-vmstat.node1.nr_shmem
10547 ? 10% numa-vmstat.node1.nr_slab_reclaimable
313776 ? 42% numa-vmstat.node1.nr_zone_active_anon
-90.3% 264439 ? 35% numa-vmstat.node1.nr_zone_inactive_anon
-52.7% 2149540 ? 35% numa-vmstat.node1.numa_hit
-53.1% 2105702 ? 36% numa-vmstat.node1.numa_local
472661 ? 98% numa-meminfo.node0.Active
472639 ? 98% numa-meminfo.node0.Active(anon)
738476 ? 82% numa-meminfo.node0.Inactive
738443 ? 82% numa-meminfo.node0.Inactive(anon)
549130 ? 76% numa-meminfo.node0.Mapped
994269 ?107% numa-meminfo.node0.Shmem
1252939 ? 42% numa-meminfo.node1.Active
1252867 ? 42% numa-meminfo.node1.Active(anon)
48404 ? 41% numa-meminfo.node1.AnonHugePages
-55.4% 134805 ? 22% numa-meminfo.node1.AnonPages
-43.1% 214145 ? 19% numa-meminfo.node1.AnonPages.max
-78.5% 2330479 ? 37% numa-meminfo.node1.FilePages
-90.3% 1056761 ? 35% numa-meminfo.node1.Inactive
-90.3% 1056603 ? 35% numa-meminfo.node1.Inactive(anon)
42189 ? 10% numa-meminfo.node1.KReclaimable
-79.7% 987910 ? 32% numa-meminfo.node1.Mapped
-74.2% 3046680 ? 28% numa-meminfo.node1.MemUsed
4742 ? 24% numa-meminfo.node1.PageTables
42189 ? 10% numa-meminfo.node1.SReclaimable
-79.7% 2174201 ? 41% numa-meminfo.node1.Shmem
-13.8% 127057 ? 6% numa-meminfo.node1.Slab
432430 ? 4% proc-vmstat.nr_active_anon
87887 proc-vmstat.nr_anon_pages
+4.4% 4684192 proc-vmstat.nr_dirty_background_threshold
+4.4% 9379838 proc-vmstat.nr_dirty_threshold
-53.3% 1648861 ? 9% proc-vmstat.nr_file_pages
+4.3% 47128423 proc-vmstat.nr_free_pages
-83.8% 448466 ? 16% proc-vmstat.nr_inactive_anon
-69.5% 383482 ? 6% proc-vmstat.nr_mapped
1936 ? 3% proc-vmstat.nr_page_table_pages
-70.4% 792860 ? 6% proc-vmstat.nr_shmem
32488 proc-vmstat.nr_slab_reclaimable
432430 ? 4% proc-vmstat.nr_zone_active_anon
-83.8% 448466 ? 16% proc-vmstat.nr_zone_inactive_anon
-60.7% 245678 ? 15% proc-vmstat.numa_hint_faults
-73.5% 142632 ? 19% proc-vmstat.numa_hint_faults_local
-33.0% 3296362 proc-vmstat.numa_hit
-83.4% 337.33 ? 17% proc-vmstat.numa_huge_pte_updates
-33.6% 3202031 proc-vmstat.numa_local
-69.5% 550361 ? 9% proc-vmstat.numa_pte_updates
1282307 ? 12% proc-vmstat.pgactivate
-32.8% 3395392 proc-vmstat.pgalloc_normal
-26.5% 1365139 ? 2% proc-vmstat.pgfault
+350.4% 3344225 proc-vmstat.pgfree
+8.1% 42423 ? 6% proc-vmstat.pgreuse
2179 ?115% sched_debug.cfs_rq:/.MIN_vruntime.avg
-94.7% 226660 ?115% sched_debug.cfs_rq:/.MIN_vruntime.max
22118 ?115% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.77 sched_debug.cfs_rq:/.h_nr_running.avg
0.33 ? 3% sched_debug.cfs_rq:/.h_nr_running.stddev
1.89 ? 8% sched_debug.cfs_rq:/.load_avg.min
2179 ?115% sched_debug.cfs_rq:/.max_vruntime.avg
-94.7% 226660 ?115% sched_debug.cfs_rq:/.max_vruntime.max
22118 ?115% sched_debug.cfs_rq:/.max_vruntime.stddev
-19.5% 3381693 sched_debug.cfs_rq:/.min_vruntime.avg
-13.4% 3892039 ? 2% sched_debug.cfs_rq:/.min_vruntime.max
-19.2% 3331483 sched_debug.cfs_rq:/.min_vruntime.min
0.76 sched_debug.cfs_rq:/.nr_running.avg
0.31 ? 4% sched_debug.cfs_rq:/.nr_running.stddev
-10.3% 404.65 sched_debug.cfs_rq:/.runnable_avg.avg
1167 ? 3% sched_debug.cfs_rq:/.runnable_avg.max
-13.1% 133.83 ? 5% sched_debug.cfs_rq:/.runnable_avg.stddev
-10.2% 401.95 sched_debug.cfs_rq:/.util_avg.avg
-11.1% 120.41 ? 5% sched_debug.cfs_rq:/.util_avg.stddev
-23.8% 175.94 ? 2% sched_debug.cfs_rq:/.util_est_enqueued.avg
-12.9% 818.86 ? 8% sched_debug.cfs_rq:/.util_est_enqueued.max
-10.3% 202.54 sched_debug.cfs_rq:/.util_est_enqueued.stddev
+20.6% 178708 sched_debug.cpu.avg_idle.avg
30886 ? 2% sched_debug.cpu.avg_idle.min
7.66 sched_debug.cpu.clock.stddev
2468 ? 3% sched_debug.cpu.curr->pid.avg
4557 ? 23% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ? 29% sched_debug.cpu.next_balance.stddev
0.47 ? 3% sched_debug.cpu.nr_running.avg
-85.3% 3489556 sched_debug.cpu.nr_switches.avg
-70.4% 7732937 ? 11% sched_debug.cpu.nr_switches.max
-85.8% 2960756 ? 6% sched_debug.cpu.nr_switches.min
-74.7% 480695 ? 16% sched_debug.cpu.nr_switches.stddev
5.38 ? 3% perf-stat.i.MPKI
-73.1% 5.12e+09 perf-stat.i.branch-instructions
1.15 perf-stat.i.branch-miss-rate%
-80.9% 58311136 ? 2% perf-stat.i.branch-misses
26.45 ? 2% perf-stat.i.cache-miss-rate%
-85.4% 2432393 perf-stat.i.context-switches
12.37 perf-stat.i.cpi
1236 perf-stat.i.cpu-migrations
0.04 ? 3% perf-stat.i.dTLB-load-miss-rate%
-85.8% 2981601 ? 5% perf-stat.i.dTLB-load-misses
-74.9% 6.71e+09 perf-stat.i.dTLB-loads
0.00 ? 23% perf-stat.i.dTLB-store-miss-rate%
74993 ? 22% perf-stat.i.dTLB-store-misses
-82.2% 2.863e+09 ? 2% perf-stat.i.dTLB-stores
-82.1% 2004383 ? 24% perf-stat.i.iTLB-load-misses
-81.3% 4309745 ? 13% perf-stat.i.iTLB-loads
-74.4% 2.406e+10 perf-stat.i.instructions
13200 ? 22% perf-stat.i.instructions-per-iTLB-miss
0.08 perf-stat.i.ipc
-76.2% 142.20 perf-stat.i.metric.M/sec
3912 ? 3% perf-stat.i.minor-faults
97.64 perf-stat.i.node-load-miss-rate%
+314.9% 8176399 ? 4% perf-stat.i.node-load-misses
-89.9% 266188 ? 29% perf-stat.i.node-loads
98.81 perf-stat.i.node-store-miss-rate%
+152.2% 4752825 ? 2% perf-stat.i.node-store-misses
93012 ? 6% perf-stat.i.node-stores
3913 ? 3% perf-stat.i.page-faults
5.41 ? 3% perf-stat.overall.MPKI
1.14 perf-stat.overall.branch-miss-rate%
25.62 ? 2% perf-stat.overall.cache-miss-rate%
11.94 perf-stat.overall.cpi
0.04 ? 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 ? 22% perf-stat.overall.dTLB-store-miss-rate%
12675 ? 22% perf-stat.overall.instructions-per-iTLB-miss
0.08 perf-stat.overall.ipc
96.80 perf-stat.overall.node-load-miss-rate%
98.08 perf-stat.overall.node-store-miss-rate%
+71.0% 5770768 perf-stat.overall.path-length
-73.1% 5.105e+09 perf-stat.ps.branch-instructions
-80.9% 58149017 ? 2% perf-stat.ps.branch-misses
-85.4% 2424954 perf-stat.ps.context-switches
1231 perf-stat.ps.cpu-migrations
-85.7% 2979011 ? 5% perf-stat.ps.dTLB-load-misses
-74.9% 6.691e+09 perf-stat.ps.dTLB-loads
74972 ? 22% perf-stat.ps.dTLB-store-misses
-82.1% 2.855e+09 ? 2% perf-stat.ps.dTLB-stores
-82.0% 2001300 ? 24% perf-stat.ps.iTLB-load-misses
-81.3% 4295158 ? 13% perf-stat.ps.iTLB-loads
-74.4% 2.4e+10 perf-stat.ps.instructions
3901 ? 3% perf-stat.ps.minor-faults
+315.4% 8159316 ? 4% perf-stat.ps.node-load-misses
-90.0% 268054 ? 29% perf-stat.ps.node-loads
+152.2% 4737805 ? 2% perf-stat.ps.node-store-misses
92786 ? 6% perf-stat.ps.node-stores
3902 ? 3% perf-stat.ps.page-faults
-74.4% 7.266e+12 perf-stat.total.instructions
4.92 ? 2% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
4.88 ? 2% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
4.54 ? 2% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
4.02 ? 3% perf-profile.calltrace.cycles-pp.__libc_write
3.48 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
2.79 perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
3.28 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
2.27 ? 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
3.09 ? 7% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
2.90 ? 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
1.17 perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
2.69 ? 2% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.58 ? 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
0.60 ? 76% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
2.44 ? 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
0.44 ? 44% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
2.37 ? 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
2.18 ? 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
2.32 ? 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
2.30 ? 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
2.26 ? 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
1.42 ? 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
2.04 ? 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
1.36 ? 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
1.98 ? 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
0.34 ? 70% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
1.32 ? 2% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
1.93 ? 2% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
1.92 ? 2% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
1.80 ? 5% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
1.66 ? 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
1.72 ? 4% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.69 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.62 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
1.60 ? 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
1.60 perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
1.54 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
1.34 ? 4% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
1.49 perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
1.27 ? 6% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
1.26 ? 6% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
0.74 ? 5% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
49.32 perf-profile.calltrace.cycles-pp.__libc_read
0.57 ? 2% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.26 ?100% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
0.26 ?100% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.vfs_write.ksys_write
0.27 ?100% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64
0.27 ?100% perf-profile.calltrace.cycles-pp.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.37 ? 71% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.writen
0.39 ? 71% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.writen.record__pushfn
0.40 ? 71% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.writen.record__pushfn.perf_mmap__push
1.44 ? 8% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_read.vfs_read
0.76 ? 10% perf-profile.calltrace.cycles-pp.update_sg_lb_stats.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance
1.41 ? 8% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.pipe_read
0.89 ? 9% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair
0.93 ? 9% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule
1.09 ? 10% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule
47.91 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
47.83 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
47.69 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
47.15 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
47.13 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
45.16 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
45.16 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
45.52 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
45.14 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
42.42 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
42.39 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
39.41 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
39.63 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
39.73 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
40.19 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
40.41 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
40.50 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
6.85 ? 2% perf-profile.children.cycles-pp.perf_tp_event
6.64 ? 2% perf-profile.children.cycles-pp.__perf_event_overflow
6.59 ? 2% perf-profile.children.cycles-pp.perf_event_output_forward
5.53 ? 2% perf-profile.children.cycles-pp.perf_prepare_sample
5.14 ? 2% perf-profile.children.cycles-pp.perf_callchain
5.10 ? 2% perf-profile.children.cycles-pp.get_perf_callchain
4.79 ? 2% perf-profile.children.cycles-pp.perf_callchain_kernel
4.66 perf-profile.children.cycles-pp.__libc_write
3.44 perf-profile.children.cycles-pp.unwind_next_frame
52.69 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
3.05 perf-profile.children.cycles-pp.perf_trace_sched_switch
52.24 perf-profile.children.cycles-pp.do_syscall_64
3.45 perf-profile.children.cycles-pp.ksys_write
3.34 perf-profile.children.cycles-pp.vfs_write
2.70 ? 3% perf-profile.children.cycles-pp.pipe_write
1.02 ? 4% perf-profile.children.cycles-pp.finish_task_switch
2.14 perf-profile.children.cycles-pp.__unwind_start
2.58 ? 2% perf-profile.children.cycles-pp.dequeue_task_fair
2.45 ? 2% perf-profile.children.cycles-pp.dequeue_entity
2.38 ? 3% perf-profile.children.cycles-pp.__wake_up_common_lock
2.48 ? 2% perf-profile.children.cycles-pp.update_curr
2.32 ? 3% perf-profile.children.cycles-pp.__wake_up_common
2.31 ? 3% perf-profile.children.cycles-pp.autoremove_wake_function
2.27 ? 3% perf-profile.children.cycles-pp.try_to_wake_up
2.29 ? 2% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
1.82 ? 6% perf-profile.children.cycles-pp.cpuidle_idle_call
1.73 ? 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
1.28 ? 6% perf-profile.children.cycles-pp.cpuidle_enter
1.27 ? 6% perf-profile.children.cycles-pp.cpuidle_enter_state
0.76 ? 4% perf-profile.children.cycles-pp.poll_idle
49.35 perf-profile.children.cycles-pp.__libc_read
0.75 ? 3% perf-profile.children.cycles-pp.unwind_get_return_address
0.75 ? 3% perf-profile.children.cycles-pp.__orc_find
0.71 ? 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.68 ? 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.69 ? 4% perf-profile.children.cycles-pp.sched_ttwu_pending
0.56 ? 3% perf-profile.children.cycles-pp.__kernel_text_address
0.58 perf-profile.children.cycles-pp.flush_smp_call_function_queue
0.46 perf-profile.children.cycles-pp.stack_access_ok
0.48 ? 2% perf-profile.children.cycles-pp.orc_find
0.34 ? 7% perf-profile.children.cycles-pp.menu_select
0.78 ? 3% perf-profile.children.cycles-pp.ttwu_do_activate
0.44 ? 4% perf-profile.children.cycles-pp.kernel_text_address
0.76 ? 3% perf-profile.children.cycles-pp.enqueue_task_fair
0.61 ? 2% perf-profile.children.cycles-pp.enqueue_entity
0.44 ? 3% perf-profile.children.cycles-pp.update_load_avg
0.29 ? 4% perf-profile.children.cycles-pp.__entry_text_start
0.35 ? 7% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.21 ? 4% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.22 ? 2% perf-profile.children.cycles-pp.sched_clock_cpu
0.12 ? 4% perf-profile.children.cycles-pp.perf_output_begin_forward
0.15 ? 9% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.12 ? 6% perf-profile.children.cycles-pp.ktime_get
0.17 ? 5% perf-profile.children.cycles-pp.set_next_entity
0.26 ? 8% perf-profile.children.cycles-pp.sysvec_call_function_single
0.18 ? 4% perf-profile.children.cycles-pp.native_sched_clock
0.12 ? 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.18 ? 3% perf-profile.children.cycles-pp.perf_callchain_user
0.13 ? 7% perf-profile.children.cycles-pp.mutex_lock
0.13 ? 10% perf-profile.children.cycles-pp.__switch_to_asm
0.15 ? 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.10 ? 11% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.11 ? 5% perf-profile.children.cycles-pp.prepare_to_wait_event
0.11 ? 6% perf-profile.children.cycles-pp.__switch_to
0.86 perf-profile.children.cycles-pp.perf_output_sample
1.70 ? 7% perf-profile.children.cycles-pp.pick_next_task_fair
0.20 ? 9% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.08 ? 10% perf-profile.children.cycles-pp.read_tsc
0.14 ? 5% perf-profile.children.cycles-pp.core_kernel_text
0.11 ? 8% perf-profile.children.cycles-pp.copy_page_from_iter
0.69 perf-profile.children.cycles-pp.__output_copy
0.12 ? 4% perf-profile.children.cycles-pp.__fdget_pos
0.11 ? 4% perf-profile.children.cycles-pp.__fget_light
0.05 ? 8% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.53 perf-profile.children.cycles-pp.perf_output_copy
0.11 ? 10% perf-profile.children.cycles-pp.security_file_permission
0.08 ? 4% perf-profile.children.cycles-pp.update_cfs_group
0.08 ? 20% perf-profile.children.cycles-pp.copy_page_to_iter
0.06 ? 7% perf-profile.children.cycles-pp.pick_next_task_idle
0.15 ? 2% perf-profile.children.cycles-pp.select_task_rq
0.10 ? 3% perf-profile.children.cycles-pp.__get_user_nocheck_8
0.09 ? 9% perf-profile.children.cycles-pp._copy_from_iter
0.08 ? 8% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.10 ? 9% perf-profile.children.cycles-pp.apparmor_file_permission
0.11 ? 4% perf-profile.children.cycles-pp.__update_load_avg_se
0.07 ? 19% perf-profile.children.cycles-pp._copy_to_iter
0.08 ? 11% perf-profile.children.cycles-pp.tick_nohz_next_event
0.48 perf-profile.children.cycles-pp.memcpy_erms
0.08 ? 5% perf-profile.children.cycles-pp.__fget_files
0.06 ? 9% perf-profile.children.cycles-pp.__update_idle_core
0.07 ? 18% perf-profile.children.cycles-pp.touch_atime
0.50 ? 16% perf-profile.children.cycles-pp.generic_perform_write
0.08 ? 6% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.51 ? 16% perf-profile.children.cycles-pp.__generic_file_write_iter
0.12 ? 3% perf-profile.children.cycles-pp.update_rq_clock
0.52 ? 16% perf-profile.children.cycles-pp.generic_file_write_iter
0.16 ? 10% perf-profile.children.cycles-pp.copyin
0.13 ? 5% perf-profile.children.cycles-pp.select_task_rq_fair
0.07 ? 6% perf-profile.children.cycles-pp.reweight_entity
0.02 ? 99% perf-profile.children.cycles-pp.put_prev_task_fair
0.03 ?100% perf-profile.children.cycles-pp.atime_needs_update
0.08 ? 24% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.06 ? 11% perf-profile.children.cycles-pp.os_xsave
0.05 ? 47% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.07 ? 7% perf-profile.children.cycles-pp.___perf_sw_event
0.02 ? 99% perf-profile.children.cycles-pp.perf_trace_buf_update
0.06 ? 8% perf-profile.children.cycles-pp.get_stack_info
0.15 ? 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.03 ?100% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.06 ? 7% perf-profile.children.cycles-pp.__might_resched
0.02 ? 99% perf-profile.children.cycles-pp.perf_instruction_pointer
0.19 ? 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.05 ? 7% perf-profile.children.cycles-pp.__might_fault
0.06 ? 9% perf-profile.children.cycles-pp.kvm_guest_state
0.06 ? 8% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.05 ? 7% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.02 ? 99% perf-profile.children.cycles-pp.get_callchain_entry
0.02 ? 99% perf-profile.children.cycles-pp.__cond_resched
0.49 ? 4% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.63 ? 17% perf-profile.children.cycles-pp.writen
0.63 ? 17% perf-profile.children.cycles-pp.record__pushfn
0.42 ? 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.07 ? 7% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.14 ? 13% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.02 ? 99% perf-profile.children.cycles-pp.file_update_time
0.20 ? 17% perf-profile.children.cycles-pp.shmem_write_begin
0.19 ? 18% perf-profile.children.cycles-pp.shmem_get_folio_gfp
0.33 ? 5% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.33 ? 4% perf-profile.children.cycles-pp.hrtimer_interrupt
0.05 perf-profile.children.cycles-pp.native_irq_return_iret
0.06 ? 7% perf-profile.children.cycles-pp.select_idle_sibling
0.08 ? 5% perf-profile.children.cycles-pp.__irq_exit_rcu
0.04 ? 44% perf-profile.children.cycles-pp.place_entity
0.12 ? 29% perf-profile.children.cycles-pp.process_simple
0.11 ? 27% perf-profile.children.cycles-pp.ordered_events__queue
0.06 ? 45% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.07 ? 15% perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.26 ? 5% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.04 ? 71% perf-profile.children.cycles-pp.vma_alloc_folio
0.05 ? 47% perf-profile.children.cycles-pp.shmem_alloc_folio
0.21 ? 5% perf-profile.children.cycles-pp.tick_sched_timer
0.05 ? 7% perf-profile.children.cycles-pp.__do_softirq
0.20 ? 4% perf-profile.children.cycles-pp.update_process_times
0.20 ? 5% perf-profile.children.cycles-pp.tick_sched_handle
0.06 ? 9% perf-profile.children.cycles-pp._find_next_and_bit
0.08 ? 25% perf-profile.children.cycles-pp.perf_session__deliver_event
0.08 ? 5% perf-profile.children.cycles-pp.__update_blocked_fair
0.10 ? 8% perf-profile.children.cycles-pp.idle_cpu
0.11 ? 22% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested
0.18 ? 2% perf-profile.children.cycles-pp.update_blocked_averages
0.40 ? 13% perf-profile.children.cycles-pp.intel_idle_irq
0.40 ? 13% perf-profile.children.cycles-pp.mwait_idle_with_hints
0.78 ? 10% perf-profile.children.cycles-pp.update_sg_lb_stats
1.41 ? 8% perf-profile.children.cycles-pp.newidle_balance
0.90 ? 9% perf-profile.children.cycles-pp.update_sd_lb_stats
0.94 ? 9% perf-profile.children.cycles-pp.find_busiest_group
1.11 ? 10% perf-profile.children.cycles-pp.load_balance
47.94 perf-profile.children.cycles-pp.ksys_read
47.85 perf-profile.children.cycles-pp.vfs_read
47.71 perf-profile.children.cycles-pp.pipe_read
47.19 perf-profile.children.cycles-pp.schedule
45.16 perf-profile.children.cycles-pp.start_secondary
45.52 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
45.52 perf-profile.children.cycles-pp.cpu_startup_entry
45.51 perf-profile.children.cycles-pp.do_idle
42.77 perf-profile.children.cycles-pp.schedule_idle
89.92 perf-profile.children.cycles-pp.__schedule
80.56 perf-profile.children.cycles-pp.prepare_task_switch
80.01 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
80.56 perf-profile.children.cycles-pp._raw_spin_lock
0.40 ? 3% perf-profile.self.cycles-pp.finish_task_switch
0.46 ? 3% perf-profile.self.cycles-pp.__schedule
1.72 perf-profile.self.cycles-pp.unwind_next_frame
0.68 ? 4% perf-profile.self.cycles-pp.poll_idle
0.75 ? 3% perf-profile.self.cycles-pp.__orc_find
0.62 ? 3% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.68 ? 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.41 ? 2% perf-profile.self.cycles-pp.stack_access_ok
0.45 ? 2% perf-profile.self.cycles-pp.orc_find
0.45 ? 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.37 ? 3% perf-profile.self.cycles-pp.perf_callchain_kernel
0.23 perf-profile.self.cycles-pp.__unwind_start
0.26 ? 3% perf-profile.self.cycles-pp.__entry_text_start
0.22 ? 3% perf-profile.self.cycles-pp.kernel_text_address
0.14 ? 9% perf-profile.self.cycles-pp.menu_select
0.11 ? 4% perf-profile.self.cycles-pp.perf_output_begin_forward
0.18 ? 5% perf-profile.self.cycles-pp.unwind_get_return_address
0.18 ? 2% perf-profile.self.cycles-pp.native_sched_clock
0.12 ? 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.16 ? 4% perf-profile.self.cycles-pp.perf_tp_event
0.13 ? 10% perf-profile.self.cycles-pp.__switch_to_asm
0.10 ? 14% perf-profile.self.cycles-pp.cpuidle_idle_call
0.12 ? 4% perf-profile.self.cycles-pp.__kernel_text_address
0.16 ? 6% perf-profile.self.cycles-pp.enqueue_entity
0.11 ? 9% perf-profile.self.cycles-pp.perf_prepare_sample
0.10 ? 7% perf-profile.self.cycles-pp.__switch_to
0.18 ? 4% perf-profile.self.cycles-pp.update_load_avg
0.08 ? 8% perf-profile.self.cycles-pp.read_tsc
0.11 ? 6% perf-profile.self.cycles-pp.newidle_balance
0.11 ? 4% perf-profile.self.cycles-pp.pipe_write
0.10 ? 5% perf-profile.self.cycles-pp.pipe_read
0.05 ? 8% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.09 ? 5% perf-profile.self.cycles-pp.get_perf_callchain
0.08 perf-profile.self.cycles-pp.update_cfs_group
0.10 ? 3% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.07 ? 7% perf-profile.self.cycles-pp.dequeue_entity
0.06 ? 8% perf-profile.self.cycles-pp.mutex_lock
0.02 ? 99% perf-profile.self.cycles-pp.ktime_get
0.09 ? 5% perf-profile.self.cycles-pp.update_curr
0.06 ? 7% perf-profile.self.cycles-pp.pick_next_task_fair
0.10 ? 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.47 ? 2% perf-profile.self.cycles-pp.memcpy_erms
0.08 ? 5% perf-profile.self.cycles-pp.__fget_files
0.09 ? 10% perf-profile.self.cycles-pp.apparmor_file_permission
0.06 ? 11% perf-profile.self.cycles-pp.do_idle
0.08 ? 6% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.14 ? 8% perf-profile.self.cycles-pp.enqueue_task_fair
0.10 perf-profile.self.cycles-pp.dequeue_task_fair
0.06 ? 7% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.06 ? 6% perf-profile.self.cycles-pp.reweight_entity
0.07 ? 9% perf-profile.self.cycles-pp.perf_event_output_forward
0.06 ? 11% perf-profile.self.cycles-pp.os_xsave
0.06 perf-profile.self.cycles-pp.perf_callchain_user
0.05 ? 7% perf-profile.self.cycles-pp.vfs_write
0.07 ? 9% perf-profile.self.cycles-pp.core_kernel_text
0.06 ? 6% perf-profile.self.cycles-pp.set_next_entity
0.06 perf-profile.self.cycles-pp.prepare_to_wait_event
0.03 ?100% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.06 ? 7% perf-profile.self.cycles-pp.__might_resched
0.15 ? 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.06 perf-profile.self.cycles-pp.___perf_sw_event
0.05 ? 7% perf-profile.self.cycles-pp.vfs_read
0.05 ? 7% perf-profile.self.cycles-pp.__libc_write
0.04 ? 45% perf-profile.self.cycles-pp.__libc_read
0.16 ? 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.04 ? 45% perf-profile.self.cycles-pp.kvm_guest_state
0.04 ? 44% perf-profile.self.cycles-pp.try_to_wake_up
0.02 ? 99% perf-profile.self.cycles-pp.do_syscall_64
0.03 ? 70% perf-profile.self.cycles-pp.select_task_rq_fair
0.06 ? 9% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.11 ? 7% perf-profile.self.cycles-pp.prepare_task_switch
0.09 ? 9% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.20 ? 2% perf-profile.self.cycles-pp.__output_copy
0.05 ? 7% perf-profile.self.cycles-pp.update_rq_clock
0.05 perf-profile.self.cycles-pp.native_irq_return_iret
0.14 ? 5% perf-profile.self.cycles-pp.perf_output_sample
0.10 ? 27% perf-profile.self.cycles-pp.queue_event
0.06 ? 11% perf-profile.self.cycles-pp.load_balance
0.10 ? 8% perf-profile.self.cycles-pp.idle_cpu
0.10 ? 9% perf-profile.self.cycles-pp.update_sd_lb_stats
0.55 ? 2% perf-profile.self.cycles-pp._raw_spin_lock
0.40 ? 13% perf-profile.self.cycles-pp.mwait_idle_with_hints
0.64 ? 10% perf-profile.self.cycles-pp.update_sg_lb_stats
80.01 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
*************************************************************************
threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory
===============================================================
kconfig/rootfs/runtime/size/tbox_group/test/testcase:
4-rhel-8.3/debian-11.1-x86_64-20220510.cgz/300s/2T/lkp-csl-2sp3/shm-xread-seq-mt/vm-scalability
Implement rseq numa node id field selftest")
Introduce per-memory-map concurrency ID")
af7f588d8f7355bc4298dd1962d
---------------------------
%stddev
\
-9.0% 191845 vm-scalability.median
-9.0% 18417158 vm-scalability.throughput
-10.2% 49390360 vm-scalability.time.maximum_resident_set_size
-12.8% 1.657e+08 vm-scalability.time.minor_page_faults
2428 vm-scalability.time.percent_of_cpu_this_job_got
5126 vm-scalability.time.system_time
2329 vm-scalability.time.user_time
-4.5% 2.583e+08 vm-scalability.time.voluntary_context_switches
-8.9% 5.535e+09 vm-scalability.workload
17.91 mpstat.cpu.all.sys%
-15.5% 9968423 ? 8% numa-numastat.node1.local_node
-15.5% 10031212 ? 7% numa-numastat.node1.numa_hit
-8.8% 28589584 vmstat.memory.cache
-4.6% 1659696 vmstat.system.cs
+12.4% 222847 vmstat.system.in
-9.6% 26360019 meminfo.Committed_AS
-23.6% 7787187 ? 10% meminfo.DirectMap2M
-9.8% 24843166 meminfo.Inactive
-9.8% 24842025 meminfo.Inactive(anon)
-9.7% 24102542 meminfo.Mapped
-8.9% 51976 meminfo.PageTables
-9.9% 24751049 meminfo.Shmem
1052 turbostat.Avg_MHz
34.05 turbostat.Busy%
-11.3% 1.767e+08 turbostat.C1
39.12 turbostat.C1%
620087 ? 67% turbostat.C6
0.19 turbostat.IPC
+12.3% 70378539 turbostat.IRQ
+2.1% 230.71 turbostat.PkgWatt
+1.0% 39.47 turbostat.RAMWatt
+35.2% 336049 ? 14% numa-meminfo.node0.AnonPages.max
-19.5% 14100517 ? 16% numa-meminfo.node0.FilePages
73942 ? 36% numa-meminfo.node0.KReclaimable
-18.7% 14882836 ? 15% numa-meminfo.node0.MemUsed
73942 ? 36% numa-meminfo.node0.SReclaimable
-17.6% 185243 ? 16% numa-meminfo.node0.Slab
-66.8% 1241460 ?140% numa-meminfo.node0.Unevictable
-13.3% 11974436 ? 9% numa-meminfo.node1.Inactive
-13.3% 11973743 ? 9% numa-meminfo.node1.Inactive(anon)
-12.8% 11622819 ? 8% numa-meminfo.node1.Mapped
24845 ? 9% numa-meminfo.node1.PageTables
-13.2% 11880742 ? 8% numa-meminfo.node1.Shmem
2524087 ? 69% numa-meminfo.node1.Unevictable
-19.5% 3524382 ? 16% numa-vmstat.node0.nr_file_pages
18480 ? 36% numa-vmstat.node0.nr_slab_reclaimable
-66.8% 310364 ?140% numa-vmstat.node0.nr_unevictable
-66.8% 310364 ?140% numa-vmstat.node0.nr_zone_unevictable
-13.3% 2992588 ? 9% numa-vmstat.node1.nr_inactive_anon
-12.8% 2904357 ? 8% numa-vmstat.node1.nr_mapped
6214 ? 9% numa-vmstat.node1.nr_page_table_pages
-13.2% 2969320 ? 8% numa-vmstat.node1.nr_shmem
631021 ? 69% numa-vmstat.node1.nr_unevictable
-13.3% 2992587 ? 9% numa-vmstat.node1.nr_zone_inactive_anon
631021 ? 69% numa-vmstat.node1.nr_zone_unevictable
-15.5% 10031142 ? 7% numa-vmstat.node1.numa_hit
-15.5% 9968353 ? 8% numa-vmstat.node1.numa_local
-4.3% 58390 ? 2% proc-vmstat.nr_active_anon
+2.8% 2522406 proc-vmstat.nr_dirty_background_threshold
+2.8% 5050981 proc-vmstat.nr_dirty_threshold
-8.7% 7129014 proc-vmstat.nr_file_pages
+2.8% 25414774 proc-vmstat.nr_free_pages
-9.8% 6210359 proc-vmstat.nr_inactive_anon
-9.7% 6025638 proc-vmstat.nr_mapped
-8.9% 12994 proc-vmstat.nr_page_table_pages
-9.9% 6187292 proc-vmstat.nr_shmem
-3.4% 42867 proc-vmstat.nr_slab_reclaimable
-4.3% 58390 ? 2% proc-vmstat.nr_zone_active_anon
-9.8% 6210359 proc-vmstat.nr_zone_inactive_anon
-12.8% 21531077 proc-vmstat.numa_hit
-12.8% 21443489 proc-vmstat.numa_local
-12.6% 21680796 proc-vmstat.pgalloc_normal
-12.7% 1.669e+08 proc-vmstat.pgfault
-12.6% 21686002 proc-vmstat.pgfree
13893 ? 22% sched_debug.cfs_rq:/.MIN_vruntime.avg
+59.3% 847386 ? 2% sched_debug.cfs_rq:/.MIN_vruntime.max
+81.1% 104202 ? 10% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.25 ? 9% sched_debug.cfs_rq:/.h_nr_running.avg
11702 ? 7% sched_debug.cfs_rq:/.load.avg
-41.9% 212.22 sched_debug.cfs_rq:/.load_avg.max
13893 ? 22% sched_debug.cfs_rq:/.max_vruntime.avg
+59.3% 847386 ? 2% sched_debug.cfs_rq:/.max_vruntime.max
+81.1% 104202 ? 10% sched_debug.cfs_rq:/.max_vruntime.stddev
+13.6% 888573 sched_debug.cfs_rq:/.min_vruntime.avg
+13.6% 932625 sched_debug.cfs_rq:/.min_vruntime.max
+12.3% 846465 sched_debug.cfs_rq:/.min_vruntime.min
0.25 ? 9% sched_debug.cfs_rq:/.nr_running.avg
9521 ? 80% sched_debug.cfs_rq:/.spread0.avg
1311 ? 3% sched_debug.cpu.curr->pid.avg
0.25 ? 2% sched_debug.cpu.nr_running.avg
26735 ? 12% sched_debug.cpu.nr_switches.stddev
4.54 perf-stat.i.MPKI
-8.2% 2.101e+10 perf-stat.i.branch-instructions
0.22 ? 2% perf-stat.i.branch-miss-rate%
+6.8% 42925617 perf-stat.i.branch-misses
23.76 perf-stat.i.cache-miss-rate%
+8.9% 72391872 perf-stat.i.cache-misses
-4.6% 1678776 perf-stat.i.context-switches
1.49 perf-stat.i.cpi
+15.2% 1.018e+11 perf-stat.i.cpu-cycles
6660 perf-stat.i.cpu-migrations
1408 perf-stat.i.cycles-between-cache-misses
-10.1% 4112764 perf-stat.i.dTLB-load-misses
-7.9% 1.954e+10 perf-stat.i.dTLB-loads
0.02 perf-stat.i.dTLB-store-miss-rate%
-13.0% 611099 perf-stat.i.dTLB-store-misses
-7.5% 3.622e+09 perf-stat.i.dTLB-stores
-5.7% 4444422 perf-stat.i.iTLB-load-misses
-7.0% 9716326 perf-stat.i.iTLB-loads
-7.6% 6.797e+10 perf-stat.i.instructions
0.67 perf-stat.i.ipc
1.06 perf-stat.i.metric.GHz
+4.2% 493.63 perf-stat.i.metric.K/sec
-8.0% 463.30 perf-stat.i.metric.M/sec
-12.9% 541054 perf-stat.i.minor-faults
+6.5% 20793294 perf-stat.i.node-load-misses
-8.9% 946875 ? 2% perf-stat.i.node-loads
+15.1% 11618998 perf-stat.i.node-store-misses
-12.9% 541058 perf-stat.i.page-faults
4.54 perf-stat.overall.MPKI
0.20 perf-stat.overall.branch-miss-rate%
23.49 perf-stat.overall.cache-miss-rate%
1.50 perf-stat.overall.cpi
1406 ? 2% perf-stat.overall.cycles-between-cache-misses
0.02 perf-stat.overall.dTLB-load-miss-rate%
0.02 perf-stat.overall.dTLB-store-miss-rate%
-2.0% 15295 perf-stat.overall.instructions-per-iTLB-miss
0.67 perf-stat.overall.ipc
3783 perf-stat.overall.path-length
-8.2% 2.094e+10 perf-stat.ps.branch-instructions
+6.9% 42799990 perf-stat.ps.branch-misses
+9.0% 72170753 perf-stat.ps.cache-misses
-4.6% 1673329 perf-stat.ps.context-switches
+15.2% 1.015e+11 perf-stat.ps.cpu-cycles
6646 perf-stat.ps.cpu-migrations
-10.1% 4099312 perf-stat.ps.dTLB-load-misses
-7.9% 1.947e+10 perf-stat.ps.dTLB-loads
-13.0% 609113 perf-stat.ps.dTLB-store-misses
-7.5% 3.611e+09 perf-stat.ps.dTLB-stores
-5.7% 4429361 perf-stat.ps.iTLB-load-misses
-7.0% 9684466 perf-stat.ps.iTLB-loads
-7.6% 6.775e+10 perf-stat.ps.instructions
-12.9% 539304 perf-stat.ps.minor-faults
+6.6% 20734296 perf-stat.ps.node-load-misses
-8.9% 944375 ? 2% perf-stat.ps.node-loads
+15.1% 11583746 perf-stat.ps.node-store-misses
-12.9% 539307 perf-stat.ps.page-faults
-7.4% 2.094e+13 perf-stat.total.instructions
19.94 ? 3% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
13.62 perf-profile.calltrace.cycles-pp.do_rw_once
24.55 perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
45.25 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
44.43 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
44.67 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
10.37 perf-profile.calltrace.cycles-pp.finish_fault.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
10.30 perf-profile.calltrace.cycles-pp._raw_spin_lock.finish_fault.do_read_fault.do_fault.__handle_mm_fault
10.15 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.finish_fault.do_read_fault.do_fault
9.89 perf-profile.calltrace.cycles-pp.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
0.35 ? 70% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault
7.98 perf-profile.calltrace.cycles-pp._raw_spin_lock.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
7.89 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.filemap_map_pages.do_read_fault.do_fault
3.79 ? 2% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.93 ? 3% perf-profile.calltrace.cycles-pp.folio_wake_bit.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
1.30 perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.77 ? 2% perf-profile.calltrace.cycles-pp.__wake_up_common.folio_wake_bit.do_read_fault.do_fault.__handle_mm_fault
0.56 ? 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_page_function.__wake_up_common.folio_wake_bit.do_read_fault
0.85 perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
0.68 ? 2% perf-profile.calltrace.cycles-pp.wake_page_function.__wake_up_common.folio_wake_bit.do_read_fault.do_fault
0.59 ? 4% perf-profile.calltrace.cycles-pp.next_uptodate_page.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
0.69 ? 3% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.55 perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.92 perf-profile.calltrace.cycles-pp.folio_wake_bit.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
0.54 perf-profile.calltrace.cycles-pp._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.53 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault
15.86 perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
15.86 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
32.30 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.do_access
31.20 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
31.05 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
30.23 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
30.23 perf-profile.calltrace.cycles-pp.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
31.93 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
31.94 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.do_access
9.65 ? 6% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter
52.56 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
52.57 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
52.55 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
53.16 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
8.75 ? 2% perf-profile.calltrace.cycles-pp.__do_fault.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
8.74 ? 2% perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.do_read_fault.do_fault.__handle_mm_fault
4.36 ? 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
8.68 ? 2% perf-profile.calltrace.cycles-pp.shmem_get_folio_gfp.shmem_fault.__do_fault.do_read_fault.do_fault
8.34 ? 2% perf-profile.calltrace.cycles-pp.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault.__do_fault.do_read_fault
8.01 ? 3% perf-profile.calltrace.cycles-pp.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault.__do_fault
4.71 ? 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
4.73 ? 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
4.87 ? 5% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
5.05 ? 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.io_schedule
5.73 ? 4% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
5.70 ? 4% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
6.90 ? 4% perf-profile.calltrace.cycles-pp.schedule.io_schedule.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp
6.91 ? 4% perf-profile.calltrace.cycles-pp.io_schedule.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault
6.88 ? 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.folio_wait_bit_common.__filemap_get_folio
5.17 ? 5% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.io_schedule.folio_wait_bit_common
11.87 perf-profile.children.cycles-pp.do_rw_once
24.82 perf-profile.children.cycles-pp.intel_idle_irq
45.76 perf-profile.children.cycles-pp.cpuidle_idle_call
44.92 perf-profile.children.cycles-pp.cpuidle_enter
44.92 perf-profile.children.cycles-pp.cpuidle_enter_state
40.85 perf-profile.children.cycles-pp.mwait_idle_with_hints
10.37 perf-profile.children.cycles-pp.finish_fault
9.89 perf-profile.children.cycles-pp.filemap_map_pages
3.83 ? 2% perf-profile.children.cycles-pp.poll_idle
0.58 ? 3% perf-profile.children.cycles-pp._raw_spin_lock_irq
1.85 perf-profile.children.cycles-pp.folio_wake_bit
1.32 perf-profile.children.cycles-pp.flush_smp_call_function_queue
39.54 perf-profile.children.cycles-pp.do_access
1.52 perf-profile.children.cycles-pp.__wake_up_common
1.14 perf-profile.children.cycles-pp.try_to_wake_up
1.35 perf-profile.children.cycles-pp.wake_page_function
0.59 ? 4% perf-profile.children.cycles-pp.next_uptodate_page
0.88 perf-profile.children.cycles-pp.sched_ttwu_pending
0.55 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.70 ? 3% perf-profile.children.cycles-pp.menu_select
0.78 perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.47 perf-profile.children.cycles-pp.ttwu_do_activate
0.55 perf-profile.children.cycles-pp.hrtimer_interrupt
0.41 perf-profile.children.cycles-pp.enqueue_task_fair
0.56 perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.42 perf-profile.children.cycles-pp.dequeue_task_fair
0.45 ? 2% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.25 ? 4% perf-profile.children.cycles-pp.select_task_rq
0.35 ? 3% perf-profile.children.cycles-pp.enqueue_entity
0.38 ? 2% perf-profile.children.cycles-pp.dequeue_entity
0.21 ? 6% perf-profile.children.cycles-pp.select_task_rq_fair
0.21 ? 2% perf-profile.children.cycles-pp.__smp_call_single_queue
0.21 ? 2% perf-profile.children.cycles-pp.llist_add_batch
0.28 perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.27 ? 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.55 perf-profile.children.cycles-pp.handle_pte_fault
0.16 ? 2% perf-profile.children.cycles-pp.folio_unlock
0.21 ? 2% perf-profile.children.cycles-pp.tick_sched_timer
0.26 ? 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.30 ? 3% perf-profile.children.cycles-pp.ktime_get
0.18 perf-profile.children.cycles-pp.xas_load
0.14 ? 6% perf-profile.children.cycles-pp.wake_affine
0.15 ? 6% perf-profile.children.cycles-pp.tick_sched_handle
0.28 perf-profile.children.cycles-pp.update_load_avg
0.24 perf-profile.children.cycles-pp.update_rq_clock
0.14 ? 5% perf-profile.children.cycles-pp.available_idle_cpu
0.17 ? 2% perf-profile.children.cycles-pp.llist_reverse_order
0.15 ? 6% perf-profile.children.cycles-pp.update_process_times
0.10 ? 4% perf-profile.children.cycles-pp.scheduler_tick
0.09 ? 5% perf-profile.children.cycles-pp.xas_find
0.17 ? 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.03 ? 70% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.18 ? 6% perf-profile.children.cycles-pp.tick_nohz_next_event
0.12 ? 3% perf-profile.children.cycles-pp.native_sched_clock
0.32 ? 2% perf-profile.children.cycles-pp.pick_next_task_fair
0.15 ? 3% perf-profile.children.cycles-pp.__switch_to_asm
0.15 ? 5% perf-profile.children.cycles-pp.sched_clock_cpu
0.18 ? 5% perf-profile.children.cycles-pp.update_curr
0.06 ? 8% perf-profile.children.cycles-pp.read_tsc
0.07 perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.10 ? 4% perf-profile.children.cycles-pp.set_next_entity
0.20 ? 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.10 ? 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.08 ? 5% perf-profile.children.cycles-pp.__list_add_valid
0.06 ? 8% perf-profile.children.cycles-pp.__perf_sw_event
0.05 ? 8% perf-profile.children.cycles-pp.reweight_entity
0.13 perf-profile.children.cycles-pp.__switch_to
0.10 ? 4% perf-profile.children.cycles-pp.___perf_sw_event
0.10 ? 4% perf-profile.children.cycles-pp.sync_regs
0.05 perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.10 ? 4% perf-profile.children.cycles-pp.perf_tp_event
0.08 ? 5% perf-profile.children.cycles-pp.put_page
0.06 ? 8% perf-profile.children.cycles-pp.__list_del_entry_valid
0.16 ? 3% perf-profile.children.cycles-pp.load_balance
0.18 ? 2% perf-profile.children.cycles-pp.newidle_balance
0.12 perf-profile.children.cycles-pp.update_sg_lb_stats
0.14 perf-profile.children.cycles-pp.find_busiest_group
0.25 ? 3% perf-profile.children.cycles-pp.up_read
0.21 ? 6% perf-profile.children.cycles-pp.down_read_trylock
0.61 ? 2% perf-profile.children.cycles-pp.finish_task_switch
0.10 ? 12% perf-profile.children.cycles-pp._find_first_zero_bit
0.38 ? 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
5.48 ? 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
16.04 perf-profile.children.cycles-pp.intel_idle
32.33 perf-profile.children.cycles-pp.asm_exc_page_fault
31.21 perf-profile.children.cycles-pp.handle_mm_fault
31.06 perf-profile.children.cycles-pp.__handle_mm_fault
30.23 perf-profile.children.cycles-pp.do_read_fault
30.24 perf-profile.children.cycles-pp.do_fault
31.94 perf-profile.children.cycles-pp.do_user_addr_fault
31.95 perf-profile.children.cycles-pp.exc_page_fault
52.57 perf-profile.children.cycles-pp.start_secondary
53.16 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
53.16 perf-profile.children.cycles-pp.cpu_startup_entry
53.14 perf-profile.children.cycles-pp.do_idle
8.75 ? 2% perf-profile.children.cycles-pp.__do_fault
8.74 ? 2% perf-profile.children.cycles-pp.shmem_fault
8.69 ? 2% perf-profile.children.cycles-pp.shmem_get_folio_gfp
8.34 ? 2% perf-profile.children.cycles-pp.__filemap_get_folio
8.01 ? 3% perf-profile.children.cycles-pp.folio_wait_bit_common
5.79 ? 4% perf-profile.children.cycles-pp.schedule_idle
6.91 ? 4% perf-profile.children.cycles-pp.schedule
6.91 ? 4% perf-profile.children.cycles-pp.io_schedule
28.16 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
28.92 perf-profile.children.cycles-pp._raw_spin_lock
10.10 ? 5% perf-profile.children.cycles-pp.prepare_task_switch
12.66 ? 4% perf-profile.children.cycles-pp.__schedule
8.54 perf-profile.self.cycles-pp.do_rw_once
40.35 perf-profile.self.cycles-pp.mwait_idle_with_hints
5.58 perf-profile.self.cycles-pp.do_access
3.76 ? 2% perf-profile.self.cycles-pp.poll_idle
0.58 ? 5% perf-profile.self.cycles-pp.next_uptodate_page
0.20 ? 6% perf-profile.self.cycles-pp.__filemap_get_folio
0.46 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.49 perf-profile.self.cycles-pp.folio_wait_bit_common
0.22 ? 3% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.38 ? 2% perf-profile.self.cycles-pp.menu_select
0.04 ? 71% perf-profile.self.cycles-pp.shmem_fault
0.21 ? 2% perf-profile.self.cycles-pp.llist_add_batch
0.16 ? 2% perf-profile.self.cycles-pp.folio_unlock
0.30 ? 4% perf-profile.self.cycles-pp.filemap_map_pages
0.14 ? 3% perf-profile.self.cycles-pp.available_idle_cpu
0.17 ? 2% perf-profile.self.cycles-pp.llist_reverse_order
0.17 ? 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.14 ? 3% perf-profile.self.cycles-pp.__switch_to_asm
0.12 ? 6% perf-profile.self.cycles-pp.native_sched_clock
0.12 ? 3% perf-profile.self.cycles-pp.enqueue_entity
0.15 ? 3% perf-profile.self.cycles-pp.xas_load
0.18 ? 2% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.17 ? 2% perf-profile.self.cycles-pp.__wake_up_common
0.08 ? 5% perf-profile.self.cycles-pp.__list_add_valid
0.06 ? 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.06 ? 8% perf-profile.self.cycles-pp.read_tsc
0.11 ? 4% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.08 ? 10% perf-profile.self.cycles-pp.cpuidle_idle_call
0.05 ? 8% perf-profile.self.cycles-pp.reweight_entity
0.13 ? 3% perf-profile.self.cycles-pp.__switch_to
0.10 ? 4% perf-profile.self.cycles-pp.sync_regs
0.10 ? 8% perf-profile.self.cycles-pp.update_load_avg
0.18 ? 2% perf-profile.self.cycles-pp.update_rq_clock
0.14 ? 5% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.10 perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.09 ? 5% perf-profile.self.cycles-pp.___perf_sw_event
0.06 ? 8% perf-profile.self.cycles-pp.__list_del_entry_valid
0.05 perf-profile.self.cycles-pp.select_task_rq_fair
0.07 ? 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.06 perf-profile.self.cycles-pp.sched_ttwu_pending
0.09 perf-profile.self.cycles-pp.update_sg_lb_stats
0.17 ? 5% perf-profile.self.cycles-pp.wake_page_function
0.25 ? 3% perf-profile.self.cycles-pp.up_read
0.26 perf-profile.self.cycles-pp.__handle_mm_fault
0.21 ? 6% perf-profile.self.cycles-pp.down_read_trylock
0.50 ? 3% perf-profile.self.cycles-pp.__schedule
0.09 ? 10% perf-profile.self.cycles-pp._find_first_zero_bit
0.12 ? 8% perf-profile.self.cycles-pp.prepare_task_switch
0.38 ? 3% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.21 ? 2% perf-profile.self.cycles-pp._raw_spin_lock
28.08 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
estimated based on internal Intel analysis and are provided
purposes only. Any difference in system hardware or software
may affect actual performance.
om/intel/lkp-tests">https://github.com/intel/lkp-tests

Attachments:

(No filename) (227.03 kB)
job-script (7.87 kB)
job.yaml (5.47 kB)
reproduce (380.00 B)
Download all attachments

2023-05-15 08:59:22

by Peter Zijlstra

[permalink] [raw]

Subject: Re: [linus:master] [sched] af7f588d8f: will-it-scale.per_thread_ops -13.9% regression

On Mon, May 15, 2023 at 03:00:44PM +0800, kernel test robot wrote:
> Hello,
>
> kernel test robot noticed a -13.9% regression of will-it-scale.per_thread_ops on:
>
> commit: af7f588d8f7355bc4298dd1962d7826358fc95f0 ("sched: Introduce per-memory-map concurrency ID")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
> testcase: will-it-scale
> test machine: 224 threads 2 sockets (Sapphire Rapids) with 256G memory
> parameters:
>
> test: context_switch1
> cpufreq_governor: performance
>
> test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
> test-url: https://github.com/antonblanchard/will-it-scale
>
> In addition to that, the commit also has significant impact on the following tests:
>
> +------------------+----------------------------------------------------------------------------------------------------+
> | testcase: change | will-it-scale: will-it-scale.per_thread_ops -73.8% regression |
> | test machine | 224 threads 2 sockets (Sapphire Rapids) with 256G memory |
> | test parameters | cpufreq_governor=performance |
> | | mode=thread |
> | | nr_task=16 |
> | | test=context_switch1 |
> +------------------+----------------------------------------------------------------------------------------------------+
> | testcase: change | will-it-scale: will-it-scale.per_thread_ops -57.9% regression |
> | test machine | 104 threads 2 sockets (Skylake) with 192G memory |
> | test parameters | cpufreq_governor=performance |
> | | mode=thread |
> | | nr_task=16 |
> | | test=context_switch1 |
> +------------------+----------------------------------------------------------------------------------------------------+
> | testcase: change | will-it-scale: will-it-scale.per_thread_ops -85.0% regression |
> | test machine | 104 threads 2 sockets (Skylake) with 192G memory |
> | test parameters | cpufreq_governor=performance |
> | | mode=thread |
> | | nr_task=50% |
> | | test=context_switch1 |
> +------------------+----------------------------------------------------------------------------------------------------+
> | testcase: change | vm-scalability: vm-scalability.throughput -9.0% regression |
> | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory |
> | test parameters | cpufreq_governor=performance |
> | | runtime=300s |
> | | size=2T |
> | | test=shm-xread-seq-mt |
> +------------------+----------------------------------------------------------------------------------------------------+
>
> FYI, we noticed that commit 223baf9d17f2 (sched: Fix performance
> regression introduced by mm_cid) fixed a sysbench regression, but
> will-it-scale context_switch1 benchmark still saw a regression on this
> fix commit.
>
> Furthermore, we applied the code diff in below link [1] on mainline, and
> the will-it-scale score was restored to the original level before this
> patch.
>
> [1] https://lore.kernel.org/lkml/[email protected]/
>

Right; so I'm thinking we can do that patch -- I'll try and get the
whole lazy TLB thing sorted, but I'm not sure I can find the piece and
quiet to think that over in a hurry :/