2021-05-11 08:16:22

by kernel test robot

[permalink] [raw]
Subject: [mm] 2d146aa3aa: vm-scalability.throughput -2.8% regression



Greeting,

FYI, we noticed a -2.8% regression of vm-scalability.throughput due to commit:


commit: 2d146aa3aa842d7f5065802556b4f9a2c6e8ef12 ("mm: memcontrol: switch to rstat")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: vm-scalability
on test machine: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:

runtime: 300s
test: lru-file-mmap-read-rand
cpufreq_governor: performance
ucode: 0x5003006

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/

In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------------------------+
| testcase: change | reaim: reaim.jobs_per_min -2.7% regression |
| test machine | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | nr_task=100% |
| | runtime=300s |
| | test=aim9 |
| | ucode=0x5003006 |
+------------------+-------------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
bin/lkp run generated-yaml-file

=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap4/lru-file-mmap-read-rand/vm-scalability/0x5003006

commit:
dc26532aed ("cgroup: rstat: punt root-level optimization to individual controllers")
2d146aa3aa ("mm: memcontrol: switch to rstat")

dc26532aed0ab25c 2d146aa3aa842d7f5065802556b
---------------- ---------------------------
%stddev %change %stddev
\ | \
8798 -3.0% 8533 vm-scalability.median
1696442 -2.8% 1648941 vm-scalability.throughput
378.62 +2.8% 389.16 vm-scalability.time.elapsed_time
378.62 +2.8% 389.16 vm-scalability.time.elapsed_time.max
48931042 -1.2% 48355398 vm-scalability.time.major_page_faults
3116506 ? 3% -19.3% 2514494 ? 10% vm-scalability.time.maximum_resident_set_size
854811 -2.2% 835712 vm-scalability.time.minor_page_faults
58258 +3.4% 60219 vm-scalability.time.system_time
36.55 ? 5% -25.4% 27.27 ? 5% vm-scalability.time.user_time
929.17 ? 53% +874.8% 9057 ? 99% softirqs.CPU12.NET_RX
6700 -1.9% 6573 vmstat.system.cs
362.61 ? 5% -10.3% 325.09 ? 5% perf-sched.wait_and_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
359.35 ? 3% -9.6% 324.98 ? 5% perf-sched.wait_time.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
1670 ? 48% +956.2% 17638 ?103% interrupts.40:PCI-MSI.524291-edge.eth0-TxRx-2
185.83 ? 25% -53.5% 86.33 ? 28% interrupts.CPU116.RES:Rescheduling_interrupts
1670 ? 48% +956.2% 17638 ?103% interrupts.CPU12.40:PCI-MSI.524291-edge.eth0-TxRx-2
97.00 ? 13% -27.5% 70.33 ? 20% interrupts.CPU176.RES:Rescheduling_interrupts
107.17 ? 21% -33.9% 70.83 ? 18% interrupts.CPU189.RES:Rescheduling_interrupts
361.67 ? 6% +11.1% 401.67 ? 3% interrupts.CPU2.RES:Rescheduling_interrupts
284.67 ? 5% +43.1% 407.50 ? 23% interrupts.CPU41.RES:Rescheduling_interrupts
268.33 ? 9% +15.7% 310.33 ? 12% interrupts.CPU46.RES:Rescheduling_interrupts
226.00 ? 7% +18.4% 267.67 ? 6% interrupts.CPU52.RES:Rescheduling_interrupts
138.83 ? 10% -17.5% 114.50 ? 14% interrupts.CPU71.RES:Rescheduling_interrupts
562317 ? 3% -8.4% 515046 ? 6% interrupts.CPU72.TLB:TLB_shootdowns
3617 ? 11% -29.9% 2535 ? 15% proc-vmstat.allocstall_normal
1016 ? 4% -13.4% 879.83 ? 8% proc-vmstat.kswapd_low_wmark_hit_quickly
1131 -6.2% 1060 ? 2% proc-vmstat.nr_isolated_file
1684103 -1.3% 1662379 proc-vmstat.nr_page_table_pages
1189118 -1.6% 1169561 proc-vmstat.nr_slab_reclaimable
1.54e+09 -1.2% 1.522e+09 proc-vmstat.pgalloc_normal
1.004e+08 -1.1% 99249683 proc-vmstat.pgfault
1.557e+09 -1.1% 1.539e+09 proc-vmstat.pgfree
48883247 -1.2% 48308956 proc-vmstat.pgmajfault
99332 +2.6% 101937 proc-vmstat.pgreuse
2.229e+09 -1.2% 2.201e+09 proc-vmstat.pgscan_file
1.496e+09 -1.2% 1.478e+09 proc-vmstat.pgsteal_file
17500257 -4.3% 16741696 proc-vmstat.workingset_nodereclaim
4747501 -2.4% 4631656 proc-vmstat.workingset_nodes
1.486e+08 -3.6% 1.433e+08 proc-vmstat.workingset_refault_file
80.71 ? 5% -25.4 55.30 ? 6% perf-profile.calltrace.cycles-pp.do_access
80.69 ? 5% -25.4 55.28 ? 6% perf-profile.calltrace.cycles-pp.asm_exc_page_fault.do_access
80.66 ? 5% -25.4 55.26 ? 6% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.do_access
80.66 ? 5% -25.4 55.26 ? 6% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
80.65 ? 5% -25.4 55.25 ? 6% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
0.67 ? 13% -0.4 0.26 ?100% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.shrink_page_list.shrink_inactive_list.shrink_lruvec
17.05 ? 28% +25.2 42.25 ? 8% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
17.06 ? 28% +25.2 42.26 ? 8% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
17.06 ? 28% +25.2 42.26 ? 8% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
17.06 ? 28% +25.2 42.26 ? 8% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
80.74 ? 5% -25.4 55.31 ? 6% perf-profile.children.cycles-pp.do_access
0.14 ? 6% -0.1 0.03 ?100% perf-profile.children.cycles-pp.__count_memcg_events
0.27 ? 16% -0.1 0.18 ? 6% perf-profile.children.cycles-pp.try_charge
0.09 ? 4% +0.0 0.11 ? 8% perf-profile.children.cycles-pp.workingset_refault
0.29 ? 3% +0.1 0.34 ? 10% perf-profile.children.cycles-pp.xfs_filemap_map_pages
0.28 ? 3% +0.1 0.34 ? 11% perf-profile.children.cycles-pp.filemap_map_pages
0.11 ? 6% +0.1 0.18 ? 22% perf-profile.children.cycles-pp.page_add_file_rmap
0.12 ? 6% +0.1 0.19 ? 19% perf-profile.children.cycles-pp.do_set_pte
0.54 ? 8% +0.2 0.76 ? 18% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.14 ? 6% -0.1 0.03 ?100% perf-profile.self.cycles-pp.__count_memcg_events
0.40 ? 38% -0.1 0.28 perf-stat.i.branch-miss-rate%
29502377 ? 5% -8.9% 26877024 perf-stat.i.branch-misses
1.102e+08 -8.6% 1.008e+08 perf-stat.i.cache-misses
2.578e+08 ? 4% -7.1% 2.395e+08 ? 2% perf-stat.i.cache-references
6587 -2.0% 6459 perf-stat.i.context-switches
3923 +9.0% 4277 perf-stat.i.cycles-between-cache-misses
0.12 ? 11% -0.0 0.10 ? 11% perf-stat.i.dTLB-load-miss-rate%
1.494e+10 +1.3% 1.515e+10 perf-stat.i.dTLB-loads
0.02 ? 14% -0.0 0.02 ? 8% perf-stat.i.dTLB-store-miss-rate%
3782013 ? 2% -8.2% 3470625 ? 2% perf-stat.i.iTLB-load-misses
14984 +7.6% 16129 ? 2% perf-stat.i.instructions-per-iTLB-miss
125639 -3.7% 120964 perf-stat.i.major-faults
385.69 ? 9% -12.7% 336.62 ? 2% perf-stat.i.metric.K/sec
24655193 -3.6% 23766292 perf-stat.i.node-load-misses
62.43 -4.3 58.12 perf-stat.i.node-store-miss-rate%
16903554 -18.7% 13745638 perf-stat.i.node-store-misses
132189 -3.6% 127372 perf-stat.i.page-faults
4.30 ? 3% -6.7% 4.01 ? 2% perf-stat.overall.MPKI
0.22 ? 4% -0.0 0.20 perf-stat.overall.branch-miss-rate%
4354 +10.2% 4797 perf-stat.overall.cycles-between-cache-misses
84.81 -2.2 82.63 perf-stat.overall.iTLB-load-miss-rate%
15931 +8.8% 17329 ? 2% perf-stat.overall.instructions-per-iTLB-miss
60.20 -4.1 56.14 perf-stat.overall.node-store-miss-rate%
43107 +2.6% 44240 perf-stat.overall.path-length
29668531 ? 4% -8.2% 27224284 perf-stat.ps.branch-misses
1.121e+08 -8.7% 1.024e+08 perf-stat.ps.cache-misses
2.623e+08 ? 3% -6.7% 2.446e+08 ? 2% perf-stat.ps.cache-references
6672 -2.0% 6538 perf-stat.ps.context-switches
1.536e+10 +1.2% 1.555e+10 perf-stat.ps.dTLB-loads
3828017 -8.0% 3520592 ? 2% perf-stat.ps.iTLB-load-misses
128812 -3.7% 124041 perf-stat.ps.major-faults
25194762 -3.7% 24262604 perf-stat.ps.node-load-misses
17325664 -18.8% 14075191 perf-stat.ps.node-store-misses
135306 -3.6% 130410 perf-stat.ps.page-faults
2.316e+13 +2.6% 2.377e+13 perf-stat.total.instructions



vm-scalability.median

9000 +--------------------------------------------------------------------+
| +. + + + +. +. + + + + + + .+ +|
8800 |++ +++ +. + +.+ + +.++ : + ++ ++ :.+ ++ .+ ++ + :+.+ :+ +++ .+ |
| + + ++ + + + +++ + O + |
8600 |-+ O O O O O O O O |
|O OO OOO O OO O O O OO O O O |
8400 |-+ OO OO O OO O OOOOO O O O O |
| O OO |
8200 |-+ O |
| |
8000 |-+ |
| O O O O |
7800 |-+ O |
| |
7600 +--------------------------------------------------------------------+


[*] bisect-good sample
[O] bisect-bad sample

***************************************************************************************************
lkp-csl-2ap3: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/aim9/reaim/0x5003006

commit:
dc26532aed ("cgroup: rstat: punt root-level optimization to individual controllers")
2d146aa3aa ("mm: memcontrol: switch to rstat")

dc26532aed0ab25c 2d146aa3aa842d7f5065802556b
---------------- ---------------------------
%stddev %change %stddev
\ | \
84.10 ? 2% +54.3% 129.73 ? 2% reaim.child_systime
98.64 -2.7% 96.02 reaim.child_utime
123355 -2.7% 120037 reaim.jobs_per_min
642.47 -2.7% 625.20 reaim.jobs_per_min_child
124335 -2.5% 121192 reaim.max_jobs_per_min
11.21 +2.8% 11.52 reaim.parent_time
1.01 ? 4% +87.3% 1.90 ? 2% reaim.std_dev_percent
0.11 ? 4% +91.0% 0.21 ? 2% reaim.std_dev_time
304.73 +2.3% 311.86 reaim.time.elapsed_time
304.73 +2.3% 311.86 reaim.time.elapsed_time.max
1380 +20.7% 1666 reaim.time.percent_of_cpu_this_job_got
1938 ? 2% +54.2% 2988 ? 2% reaim.time.system_time
2270 -2.6% 2210 reaim.time.user_time
3236827 -1.1% 3200863 reaim.time.voluntary_context_switches
0.33 ? 4% +0.1 0.39 ? 4% mpstat.cpu.all.soft%
3.55 +1.6 5.11 mpstat.cpu.all.sys%
122697 ? 2% -9.0% 111603 ? 3% meminfo.Active
122697 ? 2% -9.0% 111603 ? 3% meminfo.Active(anon)
141299 ? 2% -8.5% 129326 ? 3% meminfo.Shmem
139.33 ? 3% +17.6% 163.83 ? 3% turbostat.Avg_MHz
10.90 ? 9% +1.9 12.76 ? 8% turbostat.Busy%
91.46 +4.1% 95.19 ? 2% turbostat.PkgWatt
90.00 -1.7% 88.50 vmstat.cpu.id
13.17 ? 2% +25.3% 16.50 ? 4% vmstat.procs.r
30506 -1.8% 29962 vmstat.system.cs
30622 ? 2% -9.0% 27871 ? 3% proc-vmstat.nr_active_anon
279241 -1.1% 276304 proc-vmstat.nr_file_pages
11132 ? 2% -5.5% 10519 proc-vmstat.nr_mapped
35312 ? 2% -8.3% 32376 ? 2% proc-vmstat.nr_shmem
40363 -5.7% 38052 proc-vmstat.nr_slab_reclaimable
110643 -3.7% 106541 proc-vmstat.nr_slab_unreclaimable
30622 ? 2% -9.0% 27871 ? 3% proc-vmstat.nr_zone_active_anon
27867 ? 4% +13.4% 31609 ? 8% softirqs.CPU102.RCU
27291 ? 2% +13.1% 30868 ? 6% softirqs.CPU115.RCU
27063 ? 4% +10.6% 29942 ? 5% softirqs.CPU168.RCU
27214 ? 4% +12.6% 30637 ? 8% softirqs.CPU19.RCU
26223 ? 5% +12.4% 29479 ? 7% softirqs.CPU20.RCU
26918 ? 3% +11.2% 29939 ? 5% softirqs.CPU21.RCU
27358 ? 3% +12.8% 30859 ? 6% softirqs.CPU72.RCU
72679 ? 5% -14.8% 61952 slabinfo.cred_jar.active_objs
1732 ? 5% -14.8% 1475 slabinfo.cred_jar.active_slabs
72758 ? 5% -14.8% 61977 slabinfo.cred_jar.num_objs
1732 ? 5% -14.8% 1475 slabinfo.cred_jar.num_slabs
163390 ? 2% -9.4% 148066 slabinfo.kmalloc-32.active_objs
1276 ? 2% -9.3% 1157 slabinfo.kmalloc-32.active_slabs
163419 ? 2% -9.3% 148174 slabinfo.kmalloc-32.num_objs
1276 ? 2% -9.3% 1157 slabinfo.kmalloc-32.num_slabs
17365 ? 4% -12.4% 15219 slabinfo.kmalloc-rcl-512.active_objs
17851 ? 5% -13.8% 15380 slabinfo.kmalloc-rcl-512.num_objs
46066 ? 8% -32.6% 31064 slabinfo.pid.active_objs
719.33 ? 8% -32.6% 485.17 slabinfo.pid.active_slabs
46066 ? 8% -32.6% 31064 slabinfo.pid.num_objs
719.33 ? 8% -32.6% 485.17 slabinfo.pid.num_slabs
23819 ? 2% -8.2% 21874 ? 3% slabinfo.proc_inode_cache.num_objs
50922 ? 4% -18.8% 41342 slabinfo.radix_tree_node.active_objs
908.83 ? 4% -18.8% 737.83 slabinfo.radix_tree_node.active_slabs
50932 ? 4% -18.8% 41342 slabinfo.radix_tree_node.num_objs
908.83 ? 4% -18.8% 737.83 slabinfo.radix_tree_node.num_slabs
26831 ? 2% -13.0% 23354 slabinfo.shmem_inode_cache.active_objs
26831 ? 2% -13.0% 23354 slabinfo.shmem_inode_cache.num_objs
9939 ? 6% -14.2% 8531 slabinfo.sock_inode_cache.active_objs
9939 ? 6% -14.2% 8531 slabinfo.sock_inode_cache.num_objs
58065 ? 6% -21.1% 45815 slabinfo.task_delay_info.active_objs
1138 ? 6% -21.1% 898.00 slabinfo.task_delay_info.active_slabs
58067 ? 6% -21.1% 45815 slabinfo.task_delay_info.num_objs
1138 ? 6% -21.1% 898.00 slabinfo.task_delay_info.num_slabs
31451 ? 2% +12.7% 35438 ? 3% slabinfo.vmap_area.active_objs
4.963e+09 -4.1% 4.761e+09 perf-stat.i.branch-instructions
41054171 ? 2% +5.6% 43339523 ? 2% perf-stat.i.cache-misses
30475 -2.6% 29684 perf-stat.i.context-switches
5.083e+10 ? 3% +18.7% 6.036e+10 ? 3% perf-stat.i.cpu-cycles
8.482e+09 -3.5% 8.187e+09 perf-stat.i.dTLB-loads
5.232e+09 -3.1% 5.072e+09 perf-stat.i.dTLB-stores
8995832 +7.5% 9666624 perf-stat.i.iTLB-load-misses
3.179e+10 -3.9% 3.056e+10 perf-stat.i.instructions
0.26 ? 3% +19.0% 0.32 ? 3% perf-stat.i.metric.GHz
98.17 -3.3% 94.94 perf-stat.i.metric.M/sec
291539 -3.1% 282571 perf-stat.i.minor-faults
4098219 ? 2% -16.3% 3428560 perf-stat.i.node-store-misses
291652 -3.1% 282684 perf-stat.i.page-faults
6.08 ? 21% +24.5% 7.56 ? 21% perf-stat.overall.MPKI
1.60 ? 3% +23.8% 1.98 ? 3% perf-stat.overall.cpi
1236 ? 2% +12.4% 1389 perf-stat.overall.cycles-between-cache-misses
66.75 ? 4% +6.5 73.24 ? 8% perf-stat.overall.iTLB-load-miss-rate%
3537 -10.5% 3164 perf-stat.overall.instructions-per-iTLB-miss
0.63 ? 3% -19.3% 0.51 ? 3% perf-stat.overall.ipc
68.80 -3.4 65.35 perf-stat.overall.node-store-miss-rate%
4.991e+09 -3.3% 4.827e+09 perf-stat.ps.branch-instructions
41348616 ? 2% +6.8% 44155400 ? 2% perf-stat.ps.cache-misses
30650 -1.8% 30099 perf-stat.ps.context-switches
5.113e+10 ? 3% +20.0% 6.135e+10 ? 3% perf-stat.ps.cpu-cycles
2894 +2.2% 2958 perf-stat.ps.cpu-migrations
8.531e+09 -2.7% 8.3e+09 perf-stat.ps.dTLB-loads
5.261e+09 -2.3% 5.141e+09 perf-stat.ps.dTLB-stores
9039686 +8.3% 9792543 perf-stat.ps.iTLB-load-misses
3.198e+10 -3.1% 3.098e+10 perf-stat.ps.instructions
293396 -2.2% 286854 perf-stat.ps.minor-faults
4129671 -15.4% 3492596 perf-stat.ps.node-store-misses
293509 -2.2% 286969 perf-stat.ps.page-faults
0.15 ? 39% -93.5% 0.01 ? 69% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.01 ? 6% -33.3% 0.01 perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.02 ? 11% -69.6% 0.00 ? 10% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.02 ? 69% -80.0% 0.00 ? 20% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
0.32 ? 62% -98.8% 0.00 ? 27% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
0.01 ? 17% -61.7% 0.01 ? 20% perf-sched.sch_delay.avg.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault
0.00 ? 20% -54.5% 0.00 ? 74% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.down_write.__vma_adjust.shift_arg_pages
0.06 ?201% -99.2% 0.00 ?223% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.dput.do_unlinkat.do_syscall_64
0.05 ? 94% -88.8% 0.01 ? 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.02 ? 44% -81.6% 0.00 ? 27% perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.path_openat.do_filp_open.do_sys_openat2
0.30 ? 13% -99.0% 0.00 perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0
0.10 ? 56% -95.8% 0.00 ? 46% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.13 ? 84% -97.6% 0.00 ? 66% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link
0.19 ? 69% -98.1% 0.00 ? 21% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.filename_create.do_mknodat.do_syscall_64
0.09 ? 50% -96.1% 0.00 ? 64% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
0.03 ? 37% -48.4% 0.01 ? 30% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
7.03 ? 24% -99.6% 0.03 ? 40% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
5.94 ? 20% -46.3% 3.19 ? 32% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
9.44 ? 25% -70.2% 2.81 ? 22% perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.03 ? 64% +678.2% 0.23 ? 46% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
5.41 ? 53% -94.2% 0.31 ?109% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
4.67 ? 65% -99.7% 0.01 ?104% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
2.73 ? 59% -94.8% 0.14 ?208% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
1.39 ?124% -91.3% 0.12 ?113% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.27 ?215% -98.1% 0.01 ? 36% perf-sched.sch_delay.max.ms.io_schedule.__lock_page.pagecache_get_page.shmem_getpage_gfp
1.25 ? 35% -72.5% 0.34 ? 70% perf-sched.sch_delay.max.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault
3.42 ? 22% -45.5% 1.86 ? 44% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.12 ?208% -95.8% 0.01 ? 37% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages_nodemask.__pmd_alloc.copy_page_range
3.79 ? 36% -59.5% 1.53 ? 20% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.01 ? 81% -67.2% 0.00 ? 11% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.down_read.walk_component.link_path_walk
0.57 ?185% -97.3% 0.02 ?107% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.down_write.dup_mmap.dup_mm
0.07 ?184% -99.3% 0.00 ?223% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.dput.do_unlinkat.do_syscall_64
4.87 ? 57% -99.9% 0.01 ? 39% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.dput.scan_positives.dcache_readdir
2.55 ? 98% -93.8% 0.16 ?165% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.24 ?200% -96.6% 0.01 ? 16% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.pagecache_get_page.shmem_getpage_gfp.shmem_write_begin
0.04 ?105% -85.9% 0.01 ? 73% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.20 ?120% -94.9% 0.01 ?163% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.down_read_killable.iterate_dir.__x64_sys_getdents64
0.58 ? 62% -97.7% 0.01 ? 61% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.path_openat.do_filp_open.do_sys_openat2
7.71 ? 16% -99.8% 0.01 ? 25% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0
6.08 ? 30% -99.4% 0.04 ?170% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.84 ?102% -96.2% 0.03 ?205% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process
4.56 ? 87% -99.8% 0.01 ? 49% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link
6.50 ? 49% -99.8% 0.01 ? 5% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.filename_create.do_mknodat.do_syscall_64
8.19 ? 20% -99.2% 0.07 ?139% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
0.75 ? 61% -94.5% 0.04 ?192% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap
0.24 ?152% -99.4% 0.00 ?152% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap
0.01 ? 79% +359.3% 0.04 ?105% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
0.01 ? 3% -52.6% 0.01 ? 9% perf-sched.total_sch_delay.average.ms
12.56 ? 26% -35.1% 8.15 ? 35% perf-sched.total_sch_delay.max.ms
30.21 +109.9% 63.41 ? 3% perf-sched.total_wait_and_delay.average.ms
157346 -10.3% 141167 ? 2% perf-sched.total_wait_and_delay.count.ms
30.20 +109.9% 63.41 ? 3% perf-sched.total_wait_time.average.ms
1.83 ? 3% +4426.0% 82.86 ? 4% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
3.48 ? 10% +387.5% 16.97 ? 4% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
8.49 ? 75% +2944.8% 258.38 ? 11% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
132.38 ? 26% +371.3% 623.98 ? 45% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
94.13 ? 18% +122.5% 209.45 ? 29% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.04 ? 14% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
431.12 ? 5% +37.1% 590.85 ? 3% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
208.00 -90.7% 19.33 ?155% perf-sched.wait_and_delay.count.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
41990 -10.8% 37469 ? 2% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
42011 -12.7% 36693 ? 2% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
29.67 ? 19% -45.5% 16.17 ? 22% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
889.50 ? 7% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
155.33 ? 33% -65.6% 53.50 ? 21% perf-sched.wait_and_delay.count.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0
982.50 ? 6% +44.7% 1422 ? 34% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
5738 ? 5% -15.3% 4862 ? 4% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
1514 ? 41% +230.5% 5006 perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
1032 +384.9% 5007 perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
685.22 ? 70% +630.6% 5006 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
1023 +389.0% 5004 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
1014 +393.3% 5004 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
3.70 ? 59% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
1027 +387.0% 5004 perf-sched.wait_and_delay.max.ms.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0
362.48 ? 10% -37.3% 227.31 ? 30% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
3081 ? 9% +100.1% 6166 ? 14% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.02 ? 55% -79.6% 0.01 ? 98% perf-sched.wait_time.avg.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part
1.82 ? 3% +4445.6% 82.85 ? 4% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
3.47 ? 10% +389.5% 16.96 ? 4% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
5.18 ? 92% +1930.5% 105.14 ? 64% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
11.39 ? 43% +2168.2% 258.38 ? 11% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
132.06 ? 26% +372.5% 623.97 ? 45% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
94.06 ? 18% +122.7% 209.45 ? 29% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.02 ? 57% -90.4% 0.00 ?223% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork.[unknown]
0.47 ? 61% +118.1% 1.03 ? 39% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node.__vmalloc_node_range.dup_task_struct
1.51 ?100% +20748.9% 315.17 ?106% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_write.anon_vma_clone.__split_vma
0.05 ? 50% -98.7% 0.00 ?223% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.d_alloc_parallel.__lookup_slow
36.86 ? 85% +717.8% 301.47 ? 43% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
3.18 ? 14% +3169.3% 104.10 ? 82% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.shmem_undo_range.shmem_truncate_range.shmem_evict_inode
0.10 ? 43% -88.9% 0.01 ? 73% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.path_openat.do_filp_open.do_sys_openat2
0.11 ? 37% -90.7% 0.01 ? 34% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.walk_component.link_path_walk.part.0
431.09 ? 5% +37.1% 590.84 ? 3% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.95 ? 28% -81.9% 0.17 ?132% perf-sched.wait_time.max.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part
1514 ? 41% +230.5% 5006 perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
1024 +388.6% 5007 perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
506.35 ? 97% +888.4% 5004 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
852.80 ? 44% +487.0% 5006 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
1019 +390.9% 5004 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
1011 +394.6% 5004 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.59 ? 42% -92.5% 0.04 ?223% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork.[unknown]
2.98 ? 85% +83912.9% 2505 ? 99% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.anon_vma_clone.__split_vma
0.25 ? 64% -99.5% 0.00 ?223% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.d_alloc_parallel.__lookup_slow
4.39 ? 35% -64.9% 1.54 ? 85% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds
5.41 ? 48% +61566.5% 3337 ? 70% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
0.78 ?138% +1.1e+05% 836.18 ?222% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault
1.36 ? 78% -89.9% 0.14 ? 96% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.path_openat.do_filp_open.do_sys_openat2
3.77 ? 15% -84.0% 0.60 ? 34% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.walk_component.link_path_walk.part.0
1022 +389.7% 5004 perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0
178.25 ?210% -99.4% 1.10 ?124% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process
1.92 ? 85% -88.7% 0.22 ?223% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap
362.45 ? 10% -37.3% 227.27 ? 30% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
3081 ? 9% +100.1% 6166 ? 14% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
7.00 ? 64% -7.0 0.00 perf-profile.calltrace.cycles-pp.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64
7.19 ? 63% -6.9 0.29 ?100% perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.18 ? 63% -6.9 0.28 ?100% perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.83 ? 65% -6.8 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
7.19 ? 63% -6.8 0.37 ? 71% perf-profile.calltrace.cycles-pp.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.56 ? 67% -6.6 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir
2.90 ? 38% -2.5 0.38 ? 70% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork
2.90 ? 38% -2.5 0.38 ? 70% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
2.90 ? 38% -2.5 0.38 ? 70% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
2.90 ? 38% -2.5 0.38 ? 70% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
3.11 ? 36% -2.5 0.62 ? 13% perf-profile.calltrace.cycles-pp.__libc_fork
1.23 ? 63% -1.0 0.27 ?100% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.08 ? 18% -0.5 0.58 ? 52% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
1.53 ? 20% -0.5 1.04 ? 9% perf-profile.calltrace.cycles-pp.execve
1.53 ? 20% -0.5 1.04 ? 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
1.53 ? 20% -0.5 1.04 ? 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
1.53 ? 20% -0.5 1.04 ? 9% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
1.52 ? 20% -0.5 1.04 ? 9% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.89 ? 15% -0.3 0.62 ? 8% perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2
0.93 ? 15% -0.3 0.67 ? 7% perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open
1.17 ? 10% +0.3 1.51 ? 10% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
1.21 ? 10% +0.3 1.56 ? 10% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
1.26 ? 10% +0.4 1.61 ? 9% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
0.35 ? 70% +0.4 0.71 ? 9% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap
0.48 ? 45% +0.4 0.89 ? 11% perf-profile.calltrace.cycles-pp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
0.98 ? 9% +0.4 1.41 ? 10% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.66 ? 14% +0.4 1.08 ? 10% perf-profile.calltrace.cycles-pp.copy_page_range.dup_mmap.dup_mm.copy_process.kernel_clone
1.03 ? 8% +0.5 1.49 ? 10% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.61 ? 8% +0.5 1.11 ? 18% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.64 ? 14% +0.5 1.15 ? 6% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.73 ? 13% +0.5 2.24 ? 7% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.65 ? 8% +0.5 1.17 ? 17% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.55 ? 46% +0.6 1.13 ? 6% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.41 ? 71% +0.6 1.00 ? 16% perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm.copy_process
0.20 ?141% +0.6 0.82 ? 10% perf-profile.calltrace.cycles-pp.copy_pte_range.copy_page_range.dup_mmap.dup_mm.copy_process
1.85 ? 13% +0.7 2.54 ? 8% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +0.7 0.72 ? 22% perf-profile.calltrace.cycles-pp.__do_munmap.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.18 ?141% +0.7 0.92 ? 23% perf-profile.calltrace.cycles-pp.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.8 0.78 ? 27% perf-profile.calltrace.cycles-pp.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.58 ? 46% +0.8 1.36 ? 36% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
0.00 +0.9 0.85 ? 34% perf-profile.calltrace.cycles-pp.shmem_undo_range.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat
0.75 ? 11% +0.9 1.60 ? 7% perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.86 ? 34% perf-profile.calltrace.cycles-pp.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat.do_syscall_64
0.00 +0.9 0.86 ? 34% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.86 ? 34% perf-profile.calltrace.cycles-pp.shmem_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.87 ? 25% perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.00 +0.9 0.90 ? 32% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.90 ? 33% perf-profile.calltrace.cycles-pp.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.00 +1.0 0.96 ? 11% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
0.10 ?223% +1.0 1.06 ? 37% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.43 ? 72% +1.0 1.42 ? 34% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.73 ? 10% +1.0 1.76 ? 11% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
2.22 ? 11% +1.1 3.34 ? 11% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.75 ? 11% +1.1 1.87 ? 8% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
1.05 ? 11% +1.2 2.22 ? 25% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.80 ? 11% +1.2 2.01 ? 8% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
2.32 ? 11% +1.4 3.70 ? 13% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
2.55 ? 11% +1.4 3.94 ? 24% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
2.60 ? 11% +1.4 3.99 ? 24% perf-profile.calltrace.cycles-pp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
2.56 ? 11% +1.4 3.98 ? 12% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
2.55 ? 11% +1.4 3.97 ? 12% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
2.60 ? 11% +1.4 4.03 ? 11% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
3.33 ? 27% +1.6 4.96 ? 10% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.33 ? 27% +1.6 4.96 ? 10% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.09 ?223% +1.9 1.94 ? 41% perf-profile.calltrace.cycles-pp.shmem_add_to_page_cache.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter
1.09 ? 17% +1.9 3.03 ? 16% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
1.96 ? 12% +3.1 5.10 ? 10% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.98 ? 13% +3.2 5.19 ? 10% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
3.84 ? 10% +3.7 7.54 ? 10% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.94 ? 10% +3.8 7.72 ? 10% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.29 ? 13% +4.6 8.93 ? 8% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
4.59 ? 12% +5.5 10.06 ? 9% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
32.38 ? 10% +14.5 46.86 ? 8% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
37.20 ? 8% +20.0 57.16 ? 6% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
37.47 ? 8% +20.3 57.80 ? 6% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
39.04 ? 8% +21.9 60.91 ? 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
39.04 ? 8% +21.9 60.94 ? 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
39.04 ? 8% +21.9 60.94 ? 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
39.29 ? 8% +22.0 61.29 ? 5% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
38.93 ? 6% -7.9 31.04 ? 8% perf-profile.children.cycles-pp.do_syscall_64
39.07 ? 6% -7.9 31.19 ? 8% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
8.37 ? 53% -7.2 1.22 ? 18% perf-profile.children.cycles-pp._raw_spin_lock
7.19 ? 63% -6.7 0.48 ? 24% perf-profile.children.cycles-pp.iterate_dir
7.19 ? 63% -6.7 0.48 ? 23% perf-profile.children.cycles-pp.__x64_sys_getdents64
7.18 ? 63% -6.7 0.48 ? 23% perf-profile.children.cycles-pp.dcache_readdir
7.00 ? 64% -6.6 0.40 ? 25% perf-profile.children.cycles-pp.scan_positives
7.90 ? 56% -6.6 1.33 ? 43% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
4.42 ? 79% -3.8 0.63 ? 24% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
3.79 ? 90% -3.5 0.25 ? 29% perf-profile.children.cycles-pp.osq_lock
3.12 ? 36% -2.5 0.62 ? 12% perf-profile.children.cycles-pp.__libc_fork
1.92 ? 80% -1.6 0.36 ? 5% perf-profile.children.cycles-pp.unlink_file_vma
0.84 ? 13% -0.8 0.05 ? 46% perf-profile.children.cycles-pp.read
2.80 ? 13% -0.7 2.06 ? 7% perf-profile.children.cycles-pp.path_openat
2.81 ? 13% -0.7 2.06 ? 7% perf-profile.children.cycles-pp.do_filp_open
1.36 ? 57% -0.7 0.62 ? 8% perf-profile.children.cycles-pp.ksys_mmap_pgoff
2.87 ? 13% -0.7 2.15 ? 6% perf-profile.children.cycles-pp.do_sys_openat2
2.87 ? 12% -0.7 2.15 ? 6% perf-profile.children.cycles-pp.do_sys_open
1.31 ? 13% -0.7 0.60 ? 12% perf-profile.children.cycles-pp.common_file_perm
1.33 ? 13% -0.7 0.63 ? 12% perf-profile.children.cycles-pp.security_file_permission
2.61 ? 15% -0.7 1.95 ? 15% perf-profile.children.cycles-pp.rcu_do_batch
2.66 ? 15% -0.6 2.05 ? 14% perf-profile.children.cycles-pp.rcu_core
1.55 ? 19% -0.5 1.04 ? 9% perf-profile.children.cycles-pp.execve
1.12 ? 15% -0.5 0.61 ? 11% perf-profile.children.cycles-pp.refill_obj_stock
0.84 ? 22% -0.4 0.40 ? 49% perf-profile.children.cycles-pp.run_ksoftirqd
0.87 ? 21% -0.4 0.43 ? 46% perf-profile.children.cycles-pp.smpboot_thread_fn
1.08 ? 18% -0.4 0.66 ? 28% perf-profile.children.cycles-pp.kthread
1.10 ? 17% -0.4 0.68 ? 12% perf-profile.children.cycles-pp.page_counter_try_charge
0.89 ? 48% -0.4 0.48 ? 9% perf-profile.children.cycles-pp.__vma_adjust
1.23 ? 17% -0.4 0.84 ? 23% perf-profile.children.cycles-pp.ret_from_fork
0.91 ? 16% -0.4 0.53 ? 7% perf-profile.children.cycles-pp.page_counter_uncharge
0.90 ? 16% -0.4 0.53 ? 7% perf-profile.children.cycles-pp.page_counter_cancel
1.00 ? 17% -0.4 0.63 ? 14% perf-profile.children.cycles-pp.__memcg_kmem_charge
0.74 ? 15% -0.3 0.41 ? 11% perf-profile.children.cycles-pp.drain_obj_stock
0.54 ? 52% -0.3 0.23 ? 9% perf-profile.children.cycles-pp.vma_link
0.57 ? 21% -0.3 0.27 ? 12% perf-profile.children.cycles-pp.propagate_protected_usage
0.59 ? 13% -0.3 0.32 ? 16% perf-profile.children.cycles-pp.do_open
0.39 ? 18% -0.2 0.14 ? 8% perf-profile.children.cycles-pp.lockref_put_or_lock
0.47 ? 17% -0.2 0.26 ? 7% perf-profile.children.cycles-pp.__memcg_kmem_uncharge
0.22 ? 18% -0.2 0.06 ? 52% perf-profile.children.cycles-pp.apparmor_task_getsecid
0.22 ? 18% -0.2 0.06 ? 52% perf-profile.children.cycles-pp.security_task_getsecid_subj
0.31 ? 14% -0.2 0.16 ? 17% perf-profile.children.cycles-pp.apparmor_file_alloc_security
0.34 ? 14% -0.1 0.19 ? 16% perf-profile.children.cycles-pp.security_file_alloc
0.20 ? 18% -0.1 0.05 ? 51% perf-profile.children.cycles-pp.ima_file_check
0.43 ? 13% -0.1 0.29 ? 9% perf-profile.children.cycles-pp.workingset_age_nonresident
0.25 ? 15% -0.1 0.11 ? 8% perf-profile.children.cycles-pp.lockref_get_not_dead
0.26 ? 15% -0.1 0.13 ? 8% perf-profile.children.cycles-pp.__legitimize_path
0.48 ? 15% -0.1 0.35 ? 14% perf-profile.children.cycles-pp.uncharge_batch
0.27 ? 15% -0.1 0.14 ? 5% perf-profile.children.cycles-pp.try_to_unlazy
0.35 ? 13% -0.1 0.22 ? 12% perf-profile.children.cycles-pp.step_into
0.30 ? 10% -0.1 0.18 ? 12% perf-profile.children.cycles-pp.task_tick_fair
0.18 ? 18% -0.1 0.06 ? 47% perf-profile.children.cycles-pp.aa_get_task_label
0.26 ? 13% -0.1 0.14 ? 14% perf-profile.children.cycles-pp.do_dentry_open
0.20 ? 13% -0.1 0.09 ? 17% perf-profile.children.cycles-pp.security_file_open
0.20 ? 14% -0.1 0.08 ? 16% perf-profile.children.cycles-pp.apparmor_file_open
0.22 ? 13% -0.1 0.11 ? 16% perf-profile.children.cycles-pp.security_file_free
0.22 ? 13% -0.1 0.11 ? 17% perf-profile.children.cycles-pp.apparmor_file_free_security
0.22 ? 13% -0.1 0.11 ? 9% perf-profile.children.cycles-pp.terminate_walk
0.31 ? 13% -0.1 0.21 ? 5% perf-profile.children.cycles-pp.lookup_fast
0.18 ? 7% -0.1 0.09 ? 12% perf-profile.children.cycles-pp.update_cfs_group
0.18 ? 15% -0.1 0.10 ? 21% perf-profile.children.cycles-pp.__x64_sys_kill
0.17 ? 14% -0.1 0.09 ? 23% perf-profile.children.cycles-pp.kill_something_info
0.27 ? 7% -0.1 0.19 ? 10% perf-profile.children.cycles-pp.__entry_text_start
0.16 ? 17% -0.1 0.09 ? 19% perf-profile.children.cycles-pp.kill_pid_info
0.13 ? 16% -0.1 0.05 ? 48% perf-profile.children.cycles-pp.apparmor_task_kill
0.13 ? 16% -0.1 0.06 ? 18% perf-profile.children.cycles-pp.security_task_kill
0.13 ? 18% -0.1 0.06 ? 17% perf-profile.children.cycles-pp.group_send_sig_info
0.08 ? 8% -0.0 0.06 ? 13% perf-profile.children.cycles-pp.rcu_cblist_dequeue
0.08 ? 13% -0.0 0.06 ? 11% perf-profile.children.cycles-pp.lockref_put_return
0.06 ? 11% +0.0 0.08 ? 6% perf-profile.children.cycles-pp.create_elf_tables
0.08 ? 8% +0.0 0.10 ? 8% perf-profile.children.cycles-pp.flush_tlb_func
0.11 ? 8% +0.0 0.13 ? 8% perf-profile.children.cycles-pp.flush_tlb_mm_range
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.cpumask_next_and
0.00 +0.1 0.07 ? 19% perf-profile.children.cycles-pp.call_cpuidle
0.05 ? 47% +0.1 0.11 ? 41% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.01 ?223% +0.1 0.08 ? 30% perf-profile.children.cycles-pp.pagevec_lru_move_fn
0.01 ?223% +0.1 0.08 ? 19% perf-profile.children.cycles-pp.trigger_load_balance
0.24 ? 14% +0.1 0.31 ? 11% perf-profile.children.cycles-pp.update_sd_lb_stats
0.01 ?223% +0.1 0.09 ? 32% perf-profile.children.cycles-pp.__hrtimer_get_next_event
0.00 +0.1 0.08 ? 13% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.07 ? 14% +0.1 0.15 ? 20% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.04 ? 73% +0.1 0.12 ? 22% perf-profile.children.cycles-pp.update_irq_load_avg
0.24 ? 14% +0.1 0.33 ? 11% perf-profile.children.cycles-pp.find_busiest_group
0.02 ?141% +0.1 0.11 ? 29% perf-profile.children.cycles-pp.free_pgd_range
0.04 ? 73% +0.1 0.13 ? 20% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.04 ? 82% +0.1 0.14 ? 32% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.06 ? 11% +0.1 0.16 ? 10% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.09 ? 28% +0.1 0.20 ? 29% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.10 ? 23% +0.1 0.20 ? 27% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.01 ?223% +0.1 0.12 ? 43% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.05 ? 52% +0.1 0.16 ? 12% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.01 ?223% +0.1 0.12 ? 49% perf-profile.children.cycles-pp.timerqueue_add
0.11 ? 10% +0.1 0.23 ? 11% perf-profile.children.cycles-pp.read_tsc
0.01 ?223% +0.1 0.13 ? 44% perf-profile.children.cycles-pp.enqueue_hrtimer
0.06 ? 53% +0.1 0.18 ? 28% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.02 ?223% +0.1 0.15 ? 51% perf-profile.children.cycles-pp.timerqueue_del
0.01 ?223% +0.1 0.14 ? 37% perf-profile.children.cycles-pp.rcu_eqs_exit
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.asm_sysvec_irq_work
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.sysvec_irq_work
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.__sysvec_irq_work
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.irq_work_run
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.irq_work_single
0.04 ?109% +0.1 0.17 ? 36% perf-profile.children.cycles-pp.printk
0.13 ? 28% +0.1 0.27 ? 21% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.11 ? 16% +0.1 0.25 ? 17% perf-profile.children.cycles-pp.update_rq_clock
0.10 ? 20% +0.1 0.24 ? 18% perf-profile.children.cycles-pp.update_blocked_averages
0.04 ?112% +0.1 0.18 ? 36% perf-profile.children.cycles-pp.irq_work_run_list
0.28 ? 15% +0.1 0.43 ? 14% perf-profile.children.cycles-pp.load_balance
0.12 ? 10% +0.1 0.26 ? 8% perf-profile.children.cycles-pp.native_sched_clock
0.04 ?130% +0.2 0.20 ? 52% perf-profile.children.cycles-pp.__remove_hrtimer
0.10 ? 16% +0.2 0.26 ? 54% perf-profile.children.cycles-pp.lru_add_drain
0.10 ? 12% +0.2 0.26 ? 55% perf-profile.children.cycles-pp.lru_add_drain_cpu
0.10 ? 24% +0.2 0.26 ? 19% perf-profile.children.cycles-pp.run_rebalance_domains
0.09 ? 19% +0.2 0.25 ? 44% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.04 ?115% +0.2 0.20 ? 39% perf-profile.children.cycles-pp.rcu_idle_exit
0.18 ? 15% +0.2 0.35 ? 27% perf-profile.children.cycles-pp.lapic_next_deadline
0.14 ? 14% +0.2 0.31 ? 8% perf-profile.children.cycles-pp.sched_clock_cpu
0.05 ? 45% +0.2 0.25 ? 41% perf-profile.children.cycles-pp.page_add_new_anon_rmap
0.04 ? 72% +0.2 0.27 ? 53% perf-profile.children.cycles-pp.uncharge_page
0.22 ? 29% +0.2 0.46 ? 43% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.22 ? 20% +0.2 0.46 ? 35% perf-profile.children.cycles-pp.irqtime_account_irq
0.12 ? 53% +0.3 0.37 ? 54% perf-profile.children.cycles-pp.tick_irq_enter
0.12 ? 54% +0.3 0.39 ? 51% perf-profile.children.cycles-pp.irq_enter_rcu
0.44 ? 11% +0.3 0.71 ? 17% perf-profile.children.cycles-pp.native_irq_return_iret
0.60 ? 10% +0.3 0.92 ? 9% perf-profile.children.cycles-pp.page_remove_rmap
0.28 ? 10% +0.4 0.63 ? 11% perf-profile.children.cycles-pp.rebalance_domains
1.26 ? 10% +0.4 1.62 ? 7% perf-profile.children.cycles-pp.__x64_sys_brk
0.16 ? 15% +0.4 0.54 ? 51% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.35 ? 9% +0.4 0.73 ? 42% perf-profile.children.cycles-pp.lru_cache_add
0.46 ? 9% +0.4 0.84 ? 26% perf-profile.children.cycles-pp.do_anonymous_page
0.18 ? 18% +0.4 0.56 ? 50% perf-profile.children.cycles-pp.__count_memcg_events
0.14 ? 10% +0.4 0.54 ? 38% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.22 ? 8% +0.4 0.65 ? 41% perf-profile.children.cycles-pp.__delete_from_page_cache
0.14 ? 11% +0.4 0.57 ? 47% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.31 ? 8% +0.4 0.75 ? 36% perf-profile.children.cycles-pp.truncate_inode_page
0.26 ? 9% +0.4 0.70 ? 38% perf-profile.children.cycles-pp.delete_from_page_cache
1.40 ? 10% +0.5 1.85 ? 9% perf-profile.children.cycles-pp.zap_pte_range
0.75 ? 7% +0.5 1.20 ? 9% perf-profile.children.cycles-pp.scheduler_tick
1.46 ? 10% +0.5 1.91 ? 8% perf-profile.children.cycles-pp.unmap_page_range
1.52 ? 10% +0.5 1.99 ? 8% perf-profile.children.cycles-pp.unmap_vmas
0.47 ? 11% +0.5 1.00 ? 32% perf-profile.children.cycles-pp.wp_page_copy
0.42 ? 10% +0.5 0.94 ? 47% perf-profile.children.cycles-pp.__pagevec_lru_add
0.28 ? 17% +0.5 0.82 ? 44% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
2.45 ? 12% +0.6 3.07 ? 6% perf-profile.children.cycles-pp.irq_exit_rcu
0.66 ? 17% +0.7 1.37 ? 36% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.69 ? 15% +0.9 1.55 ? 32% perf-profile.children.cycles-pp.clockevents_program_event
1.16 ? 7% +0.9 2.06 ? 10% perf-profile.children.cycles-pp.update_process_times
3.16 ? 11% +0.9 4.06 ? 10% perf-profile.children.cycles-pp.__handle_mm_fault
1.18 ? 7% +1.0 2.17 ? 8% perf-profile.children.cycles-pp.tick_sched_handle
1.24 ? 7% +1.1 2.32 ? 7% perf-profile.children.cycles-pp.tick_sched_timer
3.70 ? 11% +1.1 4.85 ? 11% perf-profile.children.cycles-pp.asm_exc_page_fault
3.31 ? 11% +1.2 4.48 ? 12% perf-profile.children.cycles-pp.handle_mm_fault
3.56 ? 11% +1.2 4.74 ? 11% perf-profile.children.cycles-pp.do_user_addr_fault
3.57 ? 11% +1.2 4.75 ? 11% perf-profile.children.cycles-pp.exc_page_fault
1.06 ? 11% +1.2 2.24 ? 25% perf-profile.children.cycles-pp.menu_select
0.83 ? 19% +1.2 2.01 ? 25% perf-profile.children.cycles-pp.ktime_get
0.73 ? 15% +1.2 1.95 ? 41% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.71 ? 16% +1.3 2.02 ? 42% perf-profile.children.cycles-pp.mem_cgroup_charge
2.87 ? 11% +1.4 4.25 ? 22% perf-profile.children.cycles-pp.shmem_getpage_gfp
2.60 ? 11% +1.4 4.00 ? 24% perf-profile.children.cycles-pp.shmem_write_begin
1.58 ? 11% +1.8 3.39 ? 15% perf-profile.children.cycles-pp.__hrtimer_run_queues
2.57 ? 9% +3.0 5.61 ? 10% perf-profile.children.cycles-pp.hrtimer_interrupt
2.60 ? 10% +3.1 5.70 ? 9% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.39 ? 15% +3.6 3.95 ? 51% perf-profile.children.cycles-pp.__mod_memcg_state
0.59 ? 12% +3.6 4.20 ? 47% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
5.52 ? 11% +4.5 9.99 ? 7% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
5.86 ? 11% +4.9 10.72 ? 7% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
32.58 ? 10% +14.5 47.10 ? 8% perf-profile.children.cycles-pp.intel_idle
37.70 ? 8% +20.4 58.12 ? 6% perf-profile.children.cycles-pp.cpuidle_enter_state
37.71 ? 8% +20.4 58.13 ? 6% perf-profile.children.cycles-pp.cpuidle_enter
39.04 ? 8% +21.9 60.94 ? 5% perf-profile.children.cycles-pp.start_secondary
39.29 ? 8% +22.0 61.29 ? 5% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
39.29 ? 8% +22.0 61.29 ? 5% perf-profile.children.cycles-pp.cpu_startup_entry
39.29 ? 8% +22.0 61.29 ? 5% perf-profile.children.cycles-pp.do_idle
7.81 ? 56% -6.5 1.32 ? 43% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
3.73 ? 90% -3.5 0.25 ? 30% perf-profile.self.cycles-pp.osq_lock
1.23 ? 13% -0.7 0.54 ? 12% perf-profile.self.cycles-pp.common_file_perm
2.15 ? 15% -0.6 1.56 ? 17% perf-profile.self.cycles-pp.kmem_cache_free
0.56 ? 20% -0.3 0.26 ? 12% perf-profile.self.cycles-pp.propagate_protected_usage
0.83 ? 15% -0.3 0.56 ? 11% perf-profile.self.cycles-pp.page_counter_try_charge
0.59 ? 14% -0.2 0.38 ? 7% perf-profile.self.cycles-pp.page_counter_cancel
0.41 ? 15% -0.2 0.20 ? 15% perf-profile.self.cycles-pp.drain_obj_stock
0.37 ? 14% -0.2 0.19 ? 14% perf-profile.self.cycles-pp.refill_obj_stock
0.30 ? 13% -0.2 0.15 ? 17% perf-profile.self.cycles-pp.apparmor_file_alloc_security
0.42 ? 13% -0.1 0.29 ? 9% perf-profile.self.cycles-pp.workingset_age_nonresident
0.18 ? 18% -0.1 0.06 ? 47% perf-profile.self.cycles-pp.aa_get_task_label
0.22 ? 13% -0.1 0.11 ? 17% perf-profile.self.cycles-pp.apparmor_file_free_security
0.20 ? 14% -0.1 0.08 ? 16% perf-profile.self.cycles-pp.apparmor_file_open
0.14 ? 33% -0.1 0.05 ? 45% perf-profile.self.cycles-pp.rwsem_down_write_slowpath
0.18 ? 7% -0.1 0.09 ? 12% perf-profile.self.cycles-pp.update_cfs_group
0.16 ? 11% -0.1 0.09 ? 11% perf-profile.self.cycles-pp.lockref_put_or_lock
0.13 ? 12% -0.0 0.09 ? 7% perf-profile.self.cycles-pp.lockref_get_not_dead
0.09 ? 14% -0.0 0.04 ? 75% perf-profile.self.cycles-pp.update_load_avg
0.08 ? 8% -0.0 0.05 ? 9% perf-profile.self.cycles-pp.rcu_cblist_dequeue
0.08 ? 13% -0.0 0.06 ? 11% perf-profile.self.cycles-pp.lockref_put_return
0.04 ? 72% +0.0 0.08 ? 17% perf-profile.self.cycles-pp.__softirqentry_text_start
0.17 ? 14% +0.1 0.23 ? 9% perf-profile.self.cycles-pp.update_sd_lb_stats
0.04 ? 73% +0.1 0.09 ? 18% perf-profile.self.cycles-pp.update_rq_clock
0.04 ? 73% +0.1 0.11 ? 19% perf-profile.self.cycles-pp.update_blocked_averages
0.00 +0.1 0.07 ? 21% perf-profile.self.cycles-pp.trigger_load_balance
0.00 +0.1 0.08 ? 13% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.07 ? 14% +0.1 0.15 ? 20% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.04 ? 73% +0.1 0.12 ? 22% perf-profile.self.cycles-pp.update_irq_load_avg
0.02 ?142% +0.1 0.11 ? 18% perf-profile.self.cycles-pp.hrtimer_interrupt
0.03 ?108% +0.1 0.13 ? 31% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.06 ? 11% +0.1 0.16 ? 10% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.18 ? 10% +0.1 0.28 ? 9% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.03 ? 99% +0.1 0.13 ? 41% perf-profile.self.cycles-pp.workingset_activation
0.01 ?223% +0.1 0.12 ? 46% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.13 ? 28% +0.1 0.24 ? 17% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.11 ? 14% +0.1 0.23 ? 11% perf-profile.self.cycles-pp.read_tsc
0.06 ? 60% +0.1 0.19 ? 24% perf-profile.self.cycles-pp.do_idle
0.11 ? 9% +0.1 0.24 ? 8% perf-profile.self.cycles-pp.native_sched_clock
0.06 ? 46% +0.1 0.19 ? 38% perf-profile.self.cycles-pp.obj_cgroup_charge
0.18 ? 15% +0.2 0.35 ? 27% perf-profile.self.cycles-pp.lapic_next_deadline
0.17 ? 25% +0.2 0.36 ? 44% perf-profile.self.cycles-pp.update_process_times
0.04 ? 72% +0.2 0.26 ? 53% perf-profile.self.cycles-pp.uncharge_page
0.44 ? 11% +0.3 0.71 ? 17% perf-profile.self.cycles-pp.native_irq_return_iret
0.18 ? 17% +0.4 0.56 ? 50% perf-profile.self.cycles-pp.__count_memcg_events
0.13 ? 11% +0.4 0.54 ? 39% perf-profile.self.cycles-pp.__mod_lruvec_page_state
0.27 ? 18% +0.5 0.81 ? 44% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.38 ? 16% +0.6 0.96 ? 8% perf-profile.self.cycles-pp.cpuidle_enter_state
0.16 ? 23% +0.8 0.93 ? 47% perf-profile.self.cycles-pp.mem_cgroup_charge
0.74 ? 20% +1.1 1.82 ? 28% perf-profile.self.cycles-pp.ktime_get
0.39 ? 15% +3.5 3.94 ? 51% perf-profile.self.cycles-pp.__mod_memcg_state
32.58 ? 10% +14.5 47.10 ? 8% perf-profile.self.cycles-pp.intel_idle
1391 ? 20% -58.2% 581.50 ? 27% interrupts.CPU0.NMI:Non-maskable_interrupts
1391 ? 20% -58.2% 581.50 ? 27% interrupts.CPU0.PMI:Performance_monitoring_interrupts
336.00 ? 6% +27.5% 428.33 ? 7% interrupts.CPU0.RES:Rescheduling_interrupts
1149 ? 15% -43.6% 647.83 ? 36% interrupts.CPU1.NMI:Non-maskable_interrupts
1149 ? 15% -43.6% 647.83 ? 36% interrupts.CPU1.PMI:Performance_monitoring_interrupts
1196 ? 16% -58.3% 498.67 ? 30% interrupts.CPU10.NMI:Non-maskable_interrupts
1196 ? 16% -58.3% 498.67 ? 30% interrupts.CPU10.PMI:Performance_monitoring_interrupts
1127 ? 15% -64.2% 403.83 ? 51% interrupts.CPU100.NMI:Non-maskable_interrupts
1127 ? 15% -64.2% 403.83 ? 51% interrupts.CPU100.PMI:Performance_monitoring_interrupts
303.00 ? 6% +19.1% 360.83 ? 6% interrupts.CPU100.RES:Rescheduling_interrupts
1206 ? 23% -66.0% 409.83 ? 35% interrupts.CPU101.NMI:Non-maskable_interrupts
1206 ? 23% -66.0% 409.83 ? 35% interrupts.CPU101.PMI:Performance_monitoring_interrupts
292.17 ? 7% +28.8% 376.17 ? 8% interrupts.CPU101.RES:Rescheduling_interrupts
1060 ? 17% -55.8% 468.50 ? 43% interrupts.CPU102.NMI:Non-maskable_interrupts
1060 ? 17% -55.8% 468.50 ? 43% interrupts.CPU102.PMI:Performance_monitoring_interrupts
301.17 ? 5% +26.2% 380.00 ? 7% interrupts.CPU102.RES:Rescheduling_interrupts
1163 ? 20% -58.8% 478.67 ? 23% interrupts.CPU103.NMI:Non-maskable_interrupts
1163 ? 20% -58.8% 478.67 ? 23% interrupts.CPU103.PMI:Performance_monitoring_interrupts
313.67 ? 5% +23.5% 387.50 ? 4% interrupts.CPU103.RES:Rescheduling_interrupts
1228 ? 26% -50.6% 606.67 ? 30% interrupts.CPU104.NMI:Non-maskable_interrupts
1228 ? 26% -50.6% 606.67 ? 30% interrupts.CPU104.PMI:Performance_monitoring_interrupts
1209 ? 25% -59.7% 487.67 ? 39% interrupts.CPU105.NMI:Non-maskable_interrupts
1209 ? 25% -59.7% 487.67 ? 39% interrupts.CPU105.PMI:Performance_monitoring_interrupts
293.00 ? 6% +28.5% 376.50 ? 6% interrupts.CPU105.RES:Rescheduling_interrupts
1030 ? 30% -56.0% 453.00 ? 32% interrupts.CPU106.NMI:Non-maskable_interrupts
1030 ? 30% -56.0% 453.00 ? 32% interrupts.CPU106.PMI:Performance_monitoring_interrupts
310.50 ? 5% +21.9% 378.50 ? 4% interrupts.CPU106.RES:Rescheduling_interrupts
325.67 ? 6% +19.1% 388.00 ? 6% interrupts.CPU108.RES:Rescheduling_interrupts
1093 ? 37% -59.4% 443.83 ? 31% interrupts.CPU109.NMI:Non-maskable_interrupts
1093 ? 37% -59.4% 443.83 ? 31% interrupts.CPU109.PMI:Performance_monitoring_interrupts
321.17 ? 3% +22.3% 392.67 ? 7% interrupts.CPU109.RES:Rescheduling_interrupts
248.17 ? 5% +27.7% 317.00 ? 11% interrupts.CPU11.RES:Rescheduling_interrupts
315.17 ? 3% +19.1% 375.50 ? 5% interrupts.CPU110.RES:Rescheduling_interrupts
320.67 ? 4% +18.7% 380.50 ? 6% interrupts.CPU111.RES:Rescheduling_interrupts
312.33 ? 4% +23.9% 387.00 ? 8% interrupts.CPU112.RES:Rescheduling_interrupts
1448 ? 6% -14.9% 1232 ? 12% interrupts.CPU113.CAL:Function_call_interrupts
304.33 ? 7% +20.1% 365.50 ? 7% interrupts.CPU113.RES:Rescheduling_interrupts
1192 ? 22% -61.2% 463.17 ? 20% interrupts.CPU114.NMI:Non-maskable_interrupts
1192 ? 22% -61.2% 463.17 ? 20% interrupts.CPU114.PMI:Performance_monitoring_interrupts
1207 ? 25% -52.5% 573.17 ? 19% interrupts.CPU116.NMI:Non-maskable_interrupts
1207 ? 25% -52.5% 573.17 ? 19% interrupts.CPU116.PMI:Performance_monitoring_interrupts
329.67 ? 6% +20.8% 398.33 ? 5% interrupts.CPU116.RES:Rescheduling_interrupts
304.17 ? 4% +26.0% 383.33 ? 8% interrupts.CPU117.RES:Rescheduling_interrupts
1146 ? 23% -45.6% 624.00 ? 27% interrupts.CPU118.NMI:Non-maskable_interrupts
1146 ? 23% -45.6% 624.00 ? 27% interrupts.CPU118.PMI:Performance_monitoring_interrupts
1145 ? 21% -51.5% 555.67 ? 19% interrupts.CPU119.NMI:Non-maskable_interrupts
1145 ? 21% -51.5% 555.67 ? 19% interrupts.CPU119.PMI:Performance_monitoring_interrupts
282.67 ? 4% +30.3% 368.33 ? 9% interrupts.CPU119.RES:Rescheduling_interrupts
1313 ? 17% -54.6% 596.17 ? 32% interrupts.CPU12.NMI:Non-maskable_interrupts
1313 ? 17% -54.6% 596.17 ? 32% interrupts.CPU12.PMI:Performance_monitoring_interrupts
276.33 ? 6% +21.8% 336.50 ? 9% interrupts.CPU12.RES:Rescheduling_interrupts
310.33 ? 3% +26.9% 393.83 ? 9% interrupts.CPU120.RES:Rescheduling_interrupts
315.17 ? 6% +23.1% 388.00 ? 4% interrupts.CPU121.RES:Rescheduling_interrupts
314.33 ? 4% +22.9% 386.33 ? 7% interrupts.CPU122.RES:Rescheduling_interrupts
299.00 ? 6% +25.4% 375.00 ? 6% interrupts.CPU123.RES:Rescheduling_interrupts
313.00 ? 6% +18.8% 372.00 ? 2% interrupts.CPU124.RES:Rescheduling_interrupts
290.83 ? 6% +26.6% 368.33 ? 8% interrupts.CPU125.RES:Rescheduling_interrupts
305.17 ? 8% +21.4% 370.33 ? 6% interrupts.CPU126.RES:Rescheduling_interrupts
290.67 ? 7% +31.1% 381.17 ? 5% interrupts.CPU127.RES:Rescheduling_interrupts
293.17 ? 8% +28.9% 378.00 ? 8% interrupts.CPU128.RES:Rescheduling_interrupts
305.50 ? 6% +18.5% 362.17 ? 7% interrupts.CPU129.RES:Rescheduling_interrupts
1268 ? 20% -62.3% 478.33 ? 16% interrupts.CPU13.NMI:Non-maskable_interrupts
1268 ? 20% -62.3% 478.33 ? 16% interrupts.CPU13.PMI:Performance_monitoring_interrupts
289.00 ? 6% +10.5% 319.33 ? 8% interrupts.CPU13.RES:Rescheduling_interrupts
298.83 ? 8% +23.8% 370.00 ? 7% interrupts.CPU130.RES:Rescheduling_interrupts
295.17 ? 6% +24.5% 367.50 ? 7% interrupts.CPU131.RES:Rescheduling_interrupts
328.17 ? 5% +19.1% 391.00 ? 4% interrupts.CPU132.RES:Rescheduling_interrupts
307.83 ? 8% +26.5% 389.50 ? 6% interrupts.CPU133.RES:Rescheduling_interrupts
312.67 ? 9% +22.2% 382.17 ? 7% interrupts.CPU134.RES:Rescheduling_interrupts
304.83 ? 4% +21.2% 369.50 ? 5% interrupts.CPU135.RES:Rescheduling_interrupts
299.83 ? 6% +24.9% 374.50 ? 7% interrupts.CPU136.RES:Rescheduling_interrupts
290.50 ? 4% +26.5% 367.50 ? 4% interrupts.CPU137.RES:Rescheduling_interrupts
313.17 ? 6% +25.0% 391.50 ? 8% interrupts.CPU139.RES:Rescheduling_interrupts
1338 ? 9% -62.2% 505.50 ? 22% interrupts.CPU14.NMI:Non-maskable_interrupts
1338 ? 9% -62.2% 505.50 ? 22% interrupts.CPU14.PMI:Performance_monitoring_interrupts
975.67 ? 32% -45.9% 527.50 ? 28% interrupts.CPU140.NMI:Non-maskable_interrupts
975.67 ? 32% -45.9% 527.50 ? 28% interrupts.CPU140.PMI:Performance_monitoring_interrupts
296.50 ? 9% +27.7% 378.67 ? 7% interrupts.CPU140.RES:Rescheduling_interrupts
301.17 ? 7% +26.1% 379.67 ? 4% interrupts.CPU141.RES:Rescheduling_interrupts
295.83 ? 4% +22.8% 363.17 ? 10% interrupts.CPU142.RES:Rescheduling_interrupts
303.00 ? 11% +29.2% 391.50 ? 6% interrupts.CPU143.RES:Rescheduling_interrupts
314.33 ? 5% +20.1% 377.50 ? 6% interrupts.CPU144.RES:Rescheduling_interrupts
311.83 ? 8% +20.4% 375.50 ? 3% interrupts.CPU145.RES:Rescheduling_interrupts
310.00 ? 4% +21.8% 377.67 ? 7% interrupts.CPU146.RES:Rescheduling_interrupts
292.33 ? 8% +30.1% 380.33 ? 4% interrupts.CPU147.RES:Rescheduling_interrupts
294.50 ? 12% +22.8% 361.67 ? 7% interrupts.CPU148.RES:Rescheduling_interrupts
289.50 ? 8% +28.2% 371.00 ? 5% interrupts.CPU149.RES:Rescheduling_interrupts
1110 ? 21% -56.0% 488.50 ? 45% interrupts.CPU15.NMI:Non-maskable_interrupts
1110 ? 21% -56.0% 488.50 ? 45% interrupts.CPU15.PMI:Performance_monitoring_interrupts
306.17 ? 6% +23.1% 376.83 ? 6% interrupts.CPU150.RES:Rescheduling_interrupts
310.50 ? 5% +24.5% 386.50 ? 6% interrupts.CPU151.RES:Rescheduling_interrupts
1097 ? 35% -46.5% 586.83 ? 16% interrupts.CPU152.NMI:Non-maskable_interrupts
1097 ? 35% -46.5% 586.83 ? 16% interrupts.CPU152.PMI:Performance_monitoring_interrupts
312.50 ? 5% +26.6% 395.50 ? 3% interrupts.CPU152.RES:Rescheduling_interrupts
301.17 ? 10% +21.0% 364.50 ? 6% interrupts.CPU153.RES:Rescheduling_interrupts
309.67 ? 4% +25.9% 390.00 ? 5% interrupts.CPU154.RES:Rescheduling_interrupts
298.67 ? 9% +25.1% 373.50 ? 7% interrupts.CPU155.RES:Rescheduling_interrupts
285.00 ? 9% +35.5% 386.17 ? 2% interrupts.CPU156.RES:Rescheduling_interrupts
294.33 ? 7% +34.5% 396.00 ? 10% interrupts.CPU157.RES:Rescheduling_interrupts
315.83 ? 4% +21.8% 384.83 interrupts.CPU158.RES:Rescheduling_interrupts
292.50 ? 7% +21.6% 355.67 ? 6% interrupts.CPU160.RES:Rescheduling_interrupts
304.67 ? 9% +24.7% 379.83 ? 3% interrupts.CPU161.RES:Rescheduling_interrupts
278.67 ? 10% +31.4% 366.17 ? 7% interrupts.CPU162.RES:Rescheduling_interrupts
312.83 ? 9% +19.2% 373.00 ? 5% interrupts.CPU163.RES:Rescheduling_interrupts
297.33 ? 3% +24.3% 369.67 ? 7% interrupts.CPU164.RES:Rescheduling_interrupts
1022 ? 40% -48.1% 530.83 ? 23% interrupts.CPU166.NMI:Non-maskable_interrupts
1022 ? 40% -48.1% 530.83 ? 23% interrupts.CPU166.PMI:Performance_monitoring_interrupts
295.67 ? 9% +23.2% 364.33 ? 3% interrupts.CPU166.RES:Rescheduling_interrupts
303.67 ? 8% +24.1% 377.00 ? 10% interrupts.CPU167.RES:Rescheduling_interrupts
319.00 ? 7% +20.4% 384.17 ? 5% interrupts.CPU168.RES:Rescheduling_interrupts
319.00 ? 10% +18.1% 376.83 ? 6% interrupts.CPU169.RES:Rescheduling_interrupts
1067 ? 19% -58.6% 442.17 ? 29% interrupts.CPU17.NMI:Non-maskable_interrupts
1067 ? 19% -58.6% 442.17 ? 29% interrupts.CPU17.PMI:Performance_monitoring_interrupts
293.17 ? 5% +21.5% 356.33 ? 8% interrupts.CPU170.RES:Rescheduling_interrupts
301.33 ? 7% +21.3% 365.50 ? 6% interrupts.CPU171.RES:Rescheduling_interrupts
962.17 ? 35% -39.3% 584.50 ? 22% interrupts.CPU172.NMI:Non-maskable_interrupts
962.17 ? 35% -39.3% 584.50 ? 22% interrupts.CPU172.PMI:Performance_monitoring_interrupts
292.33 ? 8% +29.2% 377.83 ? 6% interrupts.CPU172.RES:Rescheduling_interrupts
303.50 ? 9% +19.7% 363.33 ? 4% interrupts.CPU173.RES:Rescheduling_interrupts
286.67 ? 7% +22.5% 351.17 ? 4% interrupts.CPU174.RES:Rescheduling_interrupts
1050 ? 27% -41.7% 612.00 ? 32% interrupts.CPU175.NMI:Non-maskable_interrupts
1050 ? 27% -41.7% 612.00 ? 32% interrupts.CPU175.PMI:Performance_monitoring_interrupts
307.17 ? 5% +19.4% 366.67 ? 5% interrupts.CPU175.RES:Rescheduling_interrupts
316.00 ? 7% +13.6% 359.00 ? 3% interrupts.CPU176.RES:Rescheduling_interrupts
1088 ? 27% -40.4% 648.33 ? 22% interrupts.CPU178.NMI:Non-maskable_interrupts
1088 ? 27% -40.4% 648.33 ? 22% interrupts.CPU178.PMI:Performance_monitoring_interrupts
280.17 ? 3% +25.8% 352.33 ? 8% interrupts.CPU178.RES:Rescheduling_interrupts
303.83 ? 5% +18.0% 358.50 ? 5% interrupts.CPU179.RES:Rescheduling_interrupts
1153 ? 21% -55.6% 512.00 ? 37% interrupts.CPU18.NMI:Non-maskable_interrupts
1153 ? 21% -55.6% 512.00 ? 37% interrupts.CPU18.PMI:Performance_monitoring_interrupts
294.33 ? 3% +25.7% 369.83 ? 7% interrupts.CPU183.RES:Rescheduling_interrupts
1011 ? 42% -46.9% 536.50 ? 17% interrupts.CPU184.NMI:Non-maskable_interrupts
1011 ? 42% -46.9% 536.50 ? 17% interrupts.CPU184.PMI:Performance_monitoring_interrupts
293.50 ? 8% +19.3% 350.17 ? 7% interrupts.CPU184.RES:Rescheduling_interrupts
303.67 ? 6% +14.5% 347.67 ? 6% interrupts.CPU185.RES:Rescheduling_interrupts
322.33 ? 5% +18.5% 382.00 ? 4% interrupts.CPU186.RES:Rescheduling_interrupts
304.00 ? 10% +26.8% 385.50 ? 4% interrupts.CPU187.RES:Rescheduling_interrupts
1010 ? 37% -44.2% 563.33 ? 15% interrupts.CPU188.NMI:Non-maskable_interrupts
1010 ? 37% -44.2% 563.33 ? 15% interrupts.CPU188.PMI:Performance_monitoring_interrupts
291.00 ? 7% +20.2% 349.83 ? 8% interrupts.CPU188.RES:Rescheduling_interrupts
273.50 ? 4% +25.9% 344.33 ? 7% interrupts.CPU189.RES:Rescheduling_interrupts
293.83 ? 7% +25.1% 367.50 ? 13% interrupts.CPU190.RES:Rescheduling_interrupts
283.17 ? 9% +26.5% 358.33 ? 8% interrupts.CPU191.RES:Rescheduling_interrupts
1145 ? 35% -53.1% 537.33 ? 47% interrupts.CPU2.NMI:Non-maskable_interrupts
1145 ? 35% -53.1% 537.33 ? 47% interrupts.CPU2.PMI:Performance_monitoring_interrupts
1146 ? 23% -55.1% 514.33 ? 24% interrupts.CPU20.NMI:Non-maskable_interrupts
1146 ? 23% -55.1% 514.33 ? 24% interrupts.CPU20.PMI:Performance_monitoring_interrupts
1142 ? 17% -54.0% 525.50 ? 35% interrupts.CPU21.NMI:Non-maskable_interrupts
1142 ? 17% -54.0% 525.50 ? 35% interrupts.CPU21.PMI:Performance_monitoring_interrupts
1166 ? 25% -49.0% 594.67 ? 27% interrupts.CPU22.NMI:Non-maskable_interrupts
1166 ? 25% -49.0% 594.67 ? 27% interrupts.CPU22.PMI:Performance_monitoring_interrupts
256.17 ? 5% +20.0% 307.33 ? 4% interrupts.CPU22.RES:Rescheduling_interrupts
1210 ? 24% -55.7% 536.00 ? 45% interrupts.CPU23.NMI:Non-maskable_interrupts
1210 ? 24% -55.7% 536.00 ? 45% interrupts.CPU23.PMI:Performance_monitoring_interrupts
267.17 ? 6% +24.8% 333.50 ? 7% interrupts.CPU24.RES:Rescheduling_interrupts
277.17 ? 4% +12.4% 311.67 ? 4% interrupts.CPU25.RES:Rescheduling_interrupts
255.17 ? 10% +26.6% 323.00 ? 4% interrupts.CPU28.RES:Rescheduling_interrupts
244.50 ? 8% +21.1% 296.00 ? 8% interrupts.CPU29.RES:Rescheduling_interrupts
1088 ? 25% -42.7% 624.00 ? 33% interrupts.CPU3.NMI:Non-maskable_interrupts
1088 ? 25% -42.7% 624.00 ? 33% interrupts.CPU3.PMI:Performance_monitoring_interrupts
990.17 ? 20% -45.7% 537.50 ? 48% interrupts.CPU30.NMI:Non-maskable_interrupts
990.17 ? 20% -45.7% 537.50 ? 48% interrupts.CPU30.PMI:Performance_monitoring_interrupts
971.67 ? 28% -48.2% 503.67 ? 43% interrupts.CPU31.NMI:Non-maskable_interrupts
971.67 ? 28% -48.2% 503.67 ? 43% interrupts.CPU31.PMI:Performance_monitoring_interrupts
1109 ? 24% -61.0% 433.00 ? 38% interrupts.CPU32.NMI:Non-maskable_interrupts
1109 ? 24% -61.0% 433.00 ? 38% interrupts.CPU32.PMI:Performance_monitoring_interrupts
1217 ? 10% -58.3% 507.83 ? 47% interrupts.CPU33.NMI:Non-maskable_interrupts
1217 ? 10% -58.3% 507.83 ? 47% interrupts.CPU33.PMI:Performance_monitoring_interrupts
252.83 ? 7% +22.1% 308.83 ? 9% interrupts.CPU33.RES:Rescheduling_interrupts
1046 ? 15% -42.7% 599.50 ? 32% interrupts.CPU34.NMI:Non-maskable_interrupts
1046 ? 15% -42.7% 599.50 ? 32% interrupts.CPU34.PMI:Performance_monitoring_interrupts
1023 ? 18% -52.4% 486.83 ? 43% interrupts.CPU35.NMI:Non-maskable_interrupts
1023 ? 18% -52.4% 486.83 ? 43% interrupts.CPU35.PMI:Performance_monitoring_interrupts
258.50 ? 4% +17.3% 303.17 ? 5% interrupts.CPU35.RES:Rescheduling_interrupts
251.33 ? 11% +23.7% 310.83 ? 5% interrupts.CPU36.RES:Rescheduling_interrupts
953.17 ? 18% -49.3% 483.67 ? 27% interrupts.CPU39.NMI:Non-maskable_interrupts
953.17 ? 18% -49.3% 483.67 ? 27% interrupts.CPU39.PMI:Performance_monitoring_interrupts
1202 ? 8% -56.8% 519.00 ? 25% interrupts.CPU4.NMI:Non-maskable_interrupts
1202 ? 8% -56.8% 519.00 ? 25% interrupts.CPU4.PMI:Performance_monitoring_interrupts
860.33 ? 35% -46.9% 456.50 ? 37% interrupts.CPU41.NMI:Non-maskable_interrupts
860.33 ? 35% -46.9% 456.50 ? 37% interrupts.CPU41.PMI:Performance_monitoring_interrupts
268.67 ? 9% +19.3% 320.50 ? 9% interrupts.CPU42.RES:Rescheduling_interrupts
978.67 ? 23% -47.3% 515.67 ? 40% interrupts.CPU43.NMI:Non-maskable_interrupts
978.67 ? 23% -47.3% 515.67 ? 40% interrupts.CPU43.PMI:Performance_monitoring_interrupts
248.67 ? 7% +20.4% 299.33 ? 6% interrupts.CPU43.RES:Rescheduling_interrupts
921.17 ? 29% -50.5% 456.00 ? 35% interrupts.CPU44.NMI:Non-maskable_interrupts
921.17 ? 29% -50.5% 456.00 ? 35% interrupts.CPU44.PMI:Performance_monitoring_interrupts
937.33 ? 28% -44.4% 521.17 ? 37% interrupts.CPU45.NMI:Non-maskable_interrupts
937.33 ? 28% -44.4% 521.17 ? 37% interrupts.CPU45.PMI:Performance_monitoring_interrupts
260.50 ? 4% +15.8% 301.67 ? 7% interrupts.CPU45.RES:Rescheduling_interrupts
253.17 ? 9% +17.9% 298.50 ? 6% interrupts.CPU46.RES:Rescheduling_interrupts
993.00 ? 27% -47.8% 518.67 ? 19% interrupts.CPU48.NMI:Non-maskable_interrupts
993.00 ? 27% -47.8% 518.67 ? 19% interrupts.CPU48.PMI:Performance_monitoring_interrupts
300.17 ? 6% +12.3% 337.17 ? 3% interrupts.CPU48.RES:Rescheduling_interrupts
272.67 ? 5% +21.5% 331.33 ? 6% interrupts.CPU49.RES:Rescheduling_interrupts
1302 ? 15% -57.8% 550.17 ? 54% interrupts.CPU5.NMI:Non-maskable_interrupts
1302 ? 15% -57.8% 550.17 ? 54% interrupts.CPU5.PMI:Performance_monitoring_interrupts
1154 ? 11% -45.1% 634.00 ? 38% interrupts.CPU50.NMI:Non-maskable_interrupts
1154 ? 11% -45.1% 634.00 ? 38% interrupts.CPU50.PMI:Performance_monitoring_interrupts
1280 ? 17% -53.0% 602.33 ? 32% interrupts.CPU51.NMI:Non-maskable_interrupts
1280 ? 17% -53.0% 602.33 ? 32% interrupts.CPU51.PMI:Performance_monitoring_interrupts
1248 ? 12% -51.6% 603.67 ? 33% interrupts.CPU52.NMI:Non-maskable_interrupts
1248 ? 12% -51.6% 603.67 ? 33% interrupts.CPU52.PMI:Performance_monitoring_interrupts
254.00 ? 5% +18.0% 299.83 ? 9% interrupts.CPU52.RES:Rescheduling_interrupts
1288 ? 19% -60.9% 503.17 ? 31% interrupts.CPU53.NMI:Non-maskable_interrupts
1288 ? 19% -60.9% 503.17 ? 31% interrupts.CPU53.PMI:Performance_monitoring_interrupts
247.17 ? 6% +16.5% 288.00 ? 5% interrupts.CPU53.RES:Rescheduling_interrupts
1201 ? 6% -44.0% 672.83 ? 25% interrupts.CPU54.NMI:Non-maskable_interrupts
1201 ? 6% -44.0% 672.83 ? 25% interrupts.CPU54.PMI:Performance_monitoring_interrupts
1200 ? 20% -53.3% 560.50 ? 49% interrupts.CPU55.NMI:Non-maskable_interrupts
1200 ? 20% -53.3% 560.50 ? 49% interrupts.CPU55.PMI:Performance_monitoring_interrupts
254.50 ? 8% +16.0% 295.17 ? 11% interrupts.CPU55.RES:Rescheduling_interrupts
1116 ? 26% -61.0% 435.67 ? 30% interrupts.CPU56.NMI:Non-maskable_interrupts
1116 ? 26% -61.0% 435.67 ? 30% interrupts.CPU56.PMI:Performance_monitoring_interrupts
264.33 ? 6% +15.1% 304.17 ? 3% interrupts.CPU56.RES:Rescheduling_interrupts
249.67 ? 9% +17.8% 294.17 ? 6% interrupts.CPU57.RES:Rescheduling_interrupts
89.00 ? 14% -19.5% 71.67 ? 13% interrupts.CPU58.TLB:TLB_shootdowns
1059 ? 5% -56.8% 457.67 ? 43% interrupts.CPU59.NMI:Non-maskable_interrupts
1059 ? 5% -56.8% 457.67 ? 43% interrupts.CPU59.PMI:Performance_monitoring_interrupts
1659 ? 18% -25.2% 1242 ? 12% interrupts.CPU6.CAL:Function_call_interrupts
1172 ? 3% -53.4% 546.33 ? 27% interrupts.CPU6.NMI:Non-maskable_interrupts
1172 ? 3% -53.4% 546.33 ? 27% interrupts.CPU6.PMI:Performance_monitoring_interrupts
1107 ? 17% -46.9% 588.00 ? 31% interrupts.CPU60.NMI:Non-maskable_interrupts
1107 ? 17% -46.9% 588.00 ? 31% interrupts.CPU60.PMI:Performance_monitoring_interrupts
238.33 ? 6% +22.7% 292.50 ? 9% interrupts.CPU60.RES:Rescheduling_interrupts
1075 ? 19% -49.3% 545.00 ? 10% interrupts.CPU61.NMI:Non-maskable_interrupts
1075 ? 19% -49.3% 545.00 ? 10% interrupts.CPU61.PMI:Performance_monitoring_interrupts
257.83 ? 7% +21.4% 313.00 ? 10% interrupts.CPU61.RES:Rescheduling_interrupts
1148 ? 14% -58.6% 475.33 ? 47% interrupts.CPU62.NMI:Non-maskable_interrupts
1148 ? 14% -58.6% 475.33 ? 47% interrupts.CPU62.PMI:Performance_monitoring_interrupts
1113 ? 16% -51.4% 541.00 ? 62% interrupts.CPU63.NMI:Non-maskable_interrupts
1113 ? 16% -51.4% 541.00 ? 62% interrupts.CPU63.PMI:Performance_monitoring_interrupts
1061 ? 15% -48.1% 551.00 ? 35% interrupts.CPU64.NMI:Non-maskable_interrupts
1061 ? 15% -48.1% 551.00 ? 35% interrupts.CPU64.PMI:Performance_monitoring_interrupts
252.50 ? 5% +19.5% 301.83 ? 5% interrupts.CPU64.RES:Rescheduling_interrupts
257.83 ? 9% +14.5% 295.17 ? 4% interrupts.CPU65.RES:Rescheduling_interrupts
250.00 ? 5% +19.5% 298.83 ? 5% interrupts.CPU66.RES:Rescheduling_interrupts
1171 ? 24% -51.7% 565.67 ? 35% interrupts.CPU67.NMI:Non-maskable_interrupts
1171 ? 24% -51.7% 565.67 ? 35% interrupts.CPU67.PMI:Performance_monitoring_interrupts
238.83 ? 7% +25.5% 299.83 ? 4% interrupts.CPU67.RES:Rescheduling_interrupts
1134 ? 19% -52.2% 542.50 ? 33% interrupts.CPU68.NMI:Non-maskable_interrupts
1134 ? 19% -52.2% 542.50 ? 33% interrupts.CPU68.PMI:Performance_monitoring_interrupts
1180 ? 11% -56.5% 513.67 ? 26% interrupts.CPU69.NMI:Non-maskable_interrupts
1180 ? 11% -56.5% 513.67 ? 26% interrupts.CPU69.PMI:Performance_monitoring_interrupts
235.67 ? 9% +29.0% 304.00 ? 8% interrupts.CPU69.RES:Rescheduling_interrupts
1224 ? 21% -63.3% 449.17 ? 23% interrupts.CPU7.NMI:Non-maskable_interrupts
1224 ? 21% -63.3% 449.17 ? 23% interrupts.CPU7.PMI:Performance_monitoring_interrupts
258.17 ? 6% +29.7% 334.83 ? 12% interrupts.CPU7.RES:Rescheduling_interrupts
1201 ? 14% -52.6% 569.67 ? 25% interrupts.CPU70.NMI:Non-maskable_interrupts
1201 ? 14% -52.6% 569.67 ? 25% interrupts.CPU70.PMI:Performance_monitoring_interrupts
252.83 ? 5% +19.4% 302.00 ? 7% interrupts.CPU70.RES:Rescheduling_interrupts
1273 ? 20% -60.5% 503.17 ? 35% interrupts.CPU71.NMI:Non-maskable_interrupts
1273 ? 20% -60.5% 503.17 ? 35% interrupts.CPU71.PMI:Performance_monitoring_interrupts
1347 ? 13% -57.2% 577.00 ? 33% interrupts.CPU72.NMI:Non-maskable_interrupts
1347 ? 13% -57.2% 577.00 ? 33% interrupts.CPU72.PMI:Performance_monitoring_interrupts
283.00 ? 6% +20.7% 341.67 ? 12% interrupts.CPU72.RES:Rescheduling_interrupts
276.83 ? 3% +14.2% 316.17 ? 5% interrupts.CPU73.RES:Rescheduling_interrupts
1197 ? 18% -55.4% 534.33 ? 25% interrupts.CPU74.NMI:Non-maskable_interrupts
1197 ? 18% -55.4% 534.33 ? 25% interrupts.CPU74.PMI:Performance_monitoring_interrupts
279.33 ? 5% +13.2% 316.17 ? 5% interrupts.CPU74.RES:Rescheduling_interrupts
1075 ? 26% -48.5% 553.67 ? 53% interrupts.CPU76.NMI:Non-maskable_interrupts
1075 ? 26% -48.5% 553.67 ? 53% interrupts.CPU76.PMI:Performance_monitoring_interrupts
997.17 ? 23% -48.8% 510.67 ? 42% interrupts.CPU77.NMI:Non-maskable_interrupts
997.17 ? 23% -48.8% 510.67 ? 42% interrupts.CPU77.PMI:Performance_monitoring_interrupts
1259 ? 10% -50.8% 620.00 ? 36% interrupts.CPU78.NMI:Non-maskable_interrupts
1259 ? 10% -50.8% 620.00 ? 36% interrupts.CPU78.PMI:Performance_monitoring_interrupts
243.50 ? 6% +19.9% 292.00 ? 8% interrupts.CPU78.RES:Rescheduling_interrupts
1204 ? 13% -59.0% 494.50 ? 33% interrupts.CPU79.NMI:Non-maskable_interrupts
1204 ? 13% -59.0% 494.50 ? 33% interrupts.CPU79.PMI:Performance_monitoring_interrupts
253.33 ? 6% +17.1% 296.67 ? 6% interrupts.CPU79.RES:Rescheduling_interrupts
1211 ? 19% -50.9% 595.33 ? 34% interrupts.CPU8.NMI:Non-maskable_interrupts
1211 ? 19% -50.9% 595.33 ? 34% interrupts.CPU8.PMI:Performance_monitoring_interrupts
273.17 ? 4% +17.6% 321.33 ? 5% interrupts.CPU8.RES:Rescheduling_interrupts
1152 ? 19% -56.4% 501.83 ? 39% interrupts.CPU80.NMI:Non-maskable_interrupts
1152 ? 19% -56.4% 501.83 ? 39% interrupts.CPU80.PMI:Performance_monitoring_interrupts
261.67 ? 6% +12.1% 293.33 ? 5% interrupts.CPU80.RES:Rescheduling_interrupts
1254 ? 7% -58.2% 524.33 ? 37% interrupts.CPU81.NMI:Non-maskable_interrupts
1254 ? 7% -58.2% 524.33 ? 37% interrupts.CPU81.PMI:Performance_monitoring_interrupts
1302 ? 9% -64.4% 463.83 ? 24% interrupts.CPU82.NMI:Non-maskable_interrupts
1302 ? 9% -64.4% 463.83 ? 24% interrupts.CPU82.PMI:Performance_monitoring_interrupts
246.00 ? 5% +16.5% 286.50 ? 7% interrupts.CPU82.RES:Rescheduling_interrupts
1233 ? 15% -53.8% 569.83 ? 34% interrupts.CPU83.NMI:Non-maskable_interrupts
1233 ? 15% -53.8% 569.83 ? 34% interrupts.CPU83.PMI:Performance_monitoring_interrupts
1246 ? 22% -58.7% 514.83 ? 61% interrupts.CPU84.NMI:Non-maskable_interrupts
1246 ? 22% -58.7% 514.83 ? 61% interrupts.CPU84.PMI:Performance_monitoring_interrupts
1263 ? 21% -53.2% 591.00 ? 27% interrupts.CPU85.NMI:Non-maskable_interrupts
1263 ? 21% -53.2% 591.00 ? 27% interrupts.CPU85.PMI:Performance_monitoring_interrupts
1297 ? 15% -63.6% 471.83 ? 37% interrupts.CPU86.NMI:Non-maskable_interrupts
1297 ? 15% -63.6% 471.83 ? 37% interrupts.CPU86.PMI:Performance_monitoring_interrupts
1262 ? 12% -63.1% 465.83 ? 40% interrupts.CPU87.NMI:Non-maskable_interrupts
1262 ? 12% -63.1% 465.83 ? 40% interrupts.CPU87.PMI:Performance_monitoring_interrupts
1251 ? 14% -57.0% 537.33 ? 35% interrupts.CPU88.NMI:Non-maskable_interrupts
1251 ? 14% -57.0% 537.33 ? 35% interrupts.CPU88.PMI:Performance_monitoring_interrupts
1152 ? 17% -54.5% 524.67 ? 44% interrupts.CPU89.NMI:Non-maskable_interrupts
1152 ? 17% -54.5% 524.67 ? 44% interrupts.CPU89.PMI:Performance_monitoring_interrupts
1335 ? 15% -57.7% 564.83 ? 30% interrupts.CPU9.NMI:Non-maskable_interrupts
1335 ? 15% -57.7% 564.83 ? 30% interrupts.CPU9.PMI:Performance_monitoring_interrupts
1241 ? 19% -51.2% 605.17 ? 46% interrupts.CPU90.NMI:Non-maskable_interrupts
1241 ? 19% -51.2% 605.17 ? 46% interrupts.CPU90.PMI:Performance_monitoring_interrupts
1226 ? 21% -56.2% 537.33 ? 39% interrupts.CPU91.NMI:Non-maskable_interrupts
1226 ? 21% -56.2% 537.33 ? 39% interrupts.CPU91.PMI:Performance_monitoring_interrupts
245.33 ? 7% +20.9% 296.50 ? 7% interrupts.CPU91.RES:Rescheduling_interrupts
1188 ? 11% -50.0% 594.33 ? 37% interrupts.CPU93.NMI:Non-maskable_interrupts
1188 ? 11% -50.0% 594.33 ? 37% interrupts.CPU93.PMI:Performance_monitoring_interrupts
1177 ? 17% -45.6% 640.17 ? 33% interrupts.CPU94.NMI:Non-maskable_interrupts
1177 ? 17% -45.6% 640.17 ? 33% interrupts.CPU94.PMI:Performance_monitoring_interrupts
236.50 ? 11% +20.7% 285.50 ? 7% interrupts.CPU94.RES:Rescheduling_interrupts
241.67 ? 3% +37.2% 331.67 ? 18% interrupts.CPU95.RES:Rescheduling_interrupts
1223 ? 25% -60.3% 486.17 ? 26% interrupts.CPU96.NMI:Non-maskable_interrupts
1223 ? 25% -60.3% 486.17 ? 26% interrupts.CPU96.PMI:Performance_monitoring_interrupts
313.00 ? 6% +28.7% 402.83 ? 7% interrupts.CPU96.RES:Rescheduling_interrupts
1155 ? 20% -53.7% 535.17 ? 53% interrupts.CPU97.NMI:Non-maskable_interrupts
1155 ? 20% -53.7% 535.17 ? 53% interrupts.CPU97.PMI:Performance_monitoring_interrupts
313.50 ? 11% +22.0% 382.50 ? 7% interrupts.CPU97.RES:Rescheduling_interrupts
299.17 ? 4% +27.4% 381.00 ? 9% interrupts.CPU98.RES:Rescheduling_interrupts
1190 ? 19% -61.6% 457.17 ? 32% interrupts.CPU99.NMI:Non-maskable_interrupts
1190 ? 19% -61.6% 457.17 ? 32% interrupts.CPU99.PMI:Performance_monitoring_interrupts
303.50 ? 8% +28.9% 391.33 ? 9% interrupts.CPU99.RES:Rescheduling_interrupts
204424 ? 8% -45.2% 112016 ? 9% interrupts.NMI:Non-maskable_interrupts
204424 ? 8% -45.2% 112016 ? 9% interrupts.PMI:Performance_monitoring_interrupts
54352 +20.3% 65372 ? 3% interrupts.RES:Rescheduling_interrupts





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation

Thanks,
Oliver Sang


Attachments:
(No filename) (100.17 kB)
config-5.12.0-11208-g2d146aa3aa84 (176.51 kB)
job-script (8.01 kB)
job.yaml (5.52 kB)
reproduce (44.40 kB)
Download all attachments