2021-07-25 15:24:27

by kernel test robot

[permalink] [raw]

Subject: [xfs] eef983ffea: stress-ng.fallocate.ops_per_sec -15.2% regression

---

Greeting,

FYI, we noticed a -15.2% regression of stress-ng.fallocate.ops_per_sec due to commit:


commit: eef983ffeae7a1cdde8c3338155ae2dd74b8621b ("xfs: journal IO cache flush reductions")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: stress-ng
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
with following parameters:

nr_threads: 10%
disk: 1HDD
testtime: 60s
fs: xfs
class: filesystem
test: fallocate
cpufreq_governor: performance
ucode: 0x5003006


In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------------+
| testcase: change | fxmark: fxmark.hdd_xfs_MWUM_72_bufferedio.works/sec 27.5% improvement |
| test machine | 272 threads 2 sockets Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory |
| test parameters | cpufreq_governor=performance |
| | directio=bufferedio |
| | disk=1HDD |
| | fstype=xfs |
| | media=hdd |
| | test=MWUM |
| | ucode=0x11 |
+------------------+---------------------------------------------------------------------------------+
| testcase: change | fxmark: fxmark.hdd_xfs_MWUL_72_bufferedio.works/sec 17.4% improvement |
| test machine | 272 threads 2 sockets Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory |
| test parameters | cpufreq_governor=performance |
| | directio=bufferedio |
| | disk=1HDD |
| | fstype=xfs |
| | media=hdd |
| | test=MWUL |
| | ucode=0x11 |
+------------------+---------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
bin/lkp run generated-yaml-file

=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
filesystem/gcc-9/performance/1HDD/xfs/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/fallocate/stress-ng/60s/0x5003006

commit:
3468bb1ca6 ("xfs: remove need_start_rec parameter from xlog_write()")
eef983ffea ("xfs: journal IO cache flush reductions")

3468bb1ca6e88407 eef983ffeae7a1cdde8c3338155
---------------- ---------------------------
%stddev %change %stddev
\ | \
538.00 -15.3% 455.83 stress-ng.fallocate.ops
8.96 -15.2% 7.60 stress-ng.fallocate.ops_per_sec
5262 +63.9% 8626 ? 4% stress-ng.time.involuntary_context_switches
102363 -19.9% 82031 stress-ng.time.voluntary_context_switches
870.67 ? 32% -31.1% 599.83 ? 24% interrupts.CPU16.CAL:Function_call_interrupts
1.59 ? 3% +44.8% 2.30 iostat.cpu.iowait
1.63 ? 3% +0.7 2.36 mpstat.cpu.all.iowait%
32539 ? 3% -13.4% 28168 proc-vmstat.pgpgout
15408 -20.8% 12198 softirqs.BLOCK
177637 ? 18% -40.9% 104995 ? 34% numa-numastat.node0.local_node
66118 ? 22% -56.2% 28966 ? 78% numa-numastat.node1.other_node
23431 ? 70% +164.3% 61929 ? 37% numa-vmstat.node0.numa_other
75268 ? 22% -51.1% 36837 ? 63% numa-vmstat.node1.numa_other
2688 ? 6% -10.2% 2415 ? 3% slabinfo.kmalloc-rcl-96.active_objs
2688 ? 6% -10.2% 2415 ? 3% slabinfo.kmalloc-rcl-96.num_objs
498.50 ? 3% -12.9% 434.00 vmstat.io.bo
1.00 +100.0% 2.00 vmstat.procs.b
7210 -11.7% 6366 vmstat.system.cs
18.41 ? 15% -26.2% 13.60 ? 19% perf-sched.total_wait_and_delay.max.ms
17.47 ? 11% -24.8% 13.14 ? 17% perf-sched.total_wait_time.max.ms
17.88 ? 18% -37.2% 11.23 ? 37% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.59 ?183% -98.0% 0.01 ?223% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_release_kernel.perf_release
1.14 ?191% -97.9% 0.02 ?223% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_release_kernel.perf_release
17.17 ? 12% -34.6% 11.22 ? 37% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
14.84 ? 54% -7.5 7.34 ? 20% perf-stat.i.cache-miss-rate%
6993 -12.6% 6113 perf-stat.i.context-switches
1.583e+08 ? 2% -2.4% 1.544e+08 perf-stat.i.dTLB-stores
918685 ? 15% -13.8% 792034 perf-stat.i.iTLB-load-misses
14.90 ? 48% -7.0 7.92 ? 22% perf-stat.overall.cache-miss-rate%
6881 -12.6% 6015 perf-stat.ps.context-switches
1.558e+08 ? 2% -2.4% 1.52e+08 perf-stat.ps.dTLB-stores
904016 ? 15% -13.8% 779391 perf-stat.ps.iTLB-load-misses
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.__libc_start_main
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.main
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.run_builtin
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.cmd_stat
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.dispatch_events
0.17 ? 21% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.process_interval
0.16 ? 22% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.read_counters
0.17 ? 41% -0.1 0.10 ? 31% perf-profile.children.cycles-pp.do_sys_ftruncate
0.17 ? 40% -0.1 0.10 ? 31% perf-profile.children.cycles-pp.do_truncate
0.16 ? 43% -0.1 0.10 ? 28% perf-profile.children.cycles-pp.xfs_itruncate_extents_flags
0.17 ? 41% -0.1 0.10 ? 31% perf-profile.children.cycles-pp.notify_change
0.17 ? 42% -0.1 0.10 ? 31% perf-profile.children.cycles-pp.xfs_vn_setattr
0.04 ? 72% +0.1 0.10 ? 20% perf-profile.children.cycles-pp.can_stop_idle_tick
0.06 ? 7% +0.1 0.13 ? 66% perf-profile.children.cycles-pp.perf_event_task_tick
0.56 ? 8% +0.2 0.72 ? 9% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.04 ? 71% +0.1 0.09 ? 22% perf-profile.self.cycles-pp.can_stop_idle_tick
0.50 ? 8% +0.1 0.61 ? 12% perf-profile.self.cycles-pp.__hrtimer_next_event_base



stress-ng.fallocate.ops_per_sec

12 +----------------------------------------------------------------------+
| |
10 |.+.+.+.+.+.++.+.+ +.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++ |
| : : + .+.+. .|
| O : : +.+ + |
8 |-O O O O O O O O:O : O O O O O O OO O O O O O O O O O |
| : : |
6 |-+ : : |
| : : |
4 |-+ : : |
| : : |
| :: |
2 |-+ : |
| : |
0 +----------------------------------------------------------------------+


[*] bisect-good sample
[O] bisect-bad sample

***************************************************************************************************
lkp-knm02: 272 threads 2 sockets Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory
=========================================================================================
compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/bufferedio/1HDD/xfs/x86_64-rhel-8.3/hdd/debian-10.4-x86_64-20200603.cgz/lkp-knm02/MWUM/fxmark/0x11

commit:
3468bb1ca6 ("xfs: remove need_start_rec parameter from xlog_write()")
eef983ffea ("xfs: journal IO cache flush reductions")

3468bb1ca6e88407 eef983ffeae7a1cdde8c3338155
---------------- ---------------------------
%stddev %change %stddev
\ | \
264.33 ? 8% -17.8% 217.23 ? 6% fxmark.hdd_xfs_MWUM_18_bufferedio.idle_sec
6.79 ? 7% -14.0% 5.83 ? 5% fxmark.hdd_xfs_MWUM_18_bufferedio.irq_sec
16.63 ? 7% -16.3% 13.92 ? 5% fxmark.hdd_xfs_MWUM_18_bufferedio.real_sec
16.22 ? 3% -15.2% 13.75 ? 5% fxmark.hdd_xfs_MWUM_18_bufferedio.secs
0.36 +8.9% 0.40 ? 4% fxmark.hdd_xfs_MWUM_18_bufferedio.softirq_util
7.23 ? 4% +17.3% 8.48 ? 2% fxmark.hdd_xfs_MWUM_18_bufferedio.sys_util
0.63 ? 3% +17.2% 0.74 ? 4% fxmark.hdd_xfs_MWUM_18_bufferedio.user_util
3206 ? 2% +17.9% 3780 ? 5% fxmark.hdd_xfs_MWUM_18_bufferedio.works/sec
0.11 ? 34% -74.8% 0.03 ?141% fxmark.hdd_xfs_MWUM_1_bufferedio.idle_util
0.04 ? 10% -76.9% 0.01 fxmark.hdd_xfs_MWUM_2_bufferedio.iowait_sec
0.19 ? 11% -75.8% 0.05 ? 2% fxmark.hdd_xfs_MWUM_2_bufferedio.iowait_util
0.56 ? 10% +26.0% 0.71 ? 9% fxmark.hdd_xfs_MWUM_36_bufferedio.iowait_util
22.42 ? 4% -20.7% 17.78 fxmark.hdd_xfs_MWUM_36_bufferedio.secs
0.26 +11.3% 0.29 ? 2% fxmark.hdd_xfs_MWUM_36_bufferedio.softirq_util
2.56 +18.4% 3.03 ? 3% fxmark.hdd_xfs_MWUM_36_bufferedio.sys_util
2357 ? 4% +25.5% 2958 ? 2% fxmark.hdd_xfs_MWUM_36_bufferedio.works/sec
0.36 ? 25% -35.5% 0.23 ? 9% fxmark.hdd_xfs_MWUM_4_bufferedio.iowait_sec
0.81 ? 20% -29.4% 0.57 ? 10% fxmark.hdd_xfs_MWUM_4_bufferedio.iowait_util
1.53 ? 3% +43.7% 2.20 ? 18% fxmark.hdd_xfs_MWUM_54_bufferedio.sys_util
3.25 ? 3% +116.4% 7.04 ? 28% fxmark.hdd_xfs_MWUM_54_bufferedio.user_sec
0.29 ? 6% +462.4% 1.64 ? 85% fxmark.hdd_xfs_MWUM_54_bufferedio.user_util
0.37 ? 4% +18.8% 0.44 fxmark.hdd_xfs_MWUM_72_bufferedio.iowait_util
29.50 ? 2% -21.4% 23.20 ? 2% fxmark.hdd_xfs_MWUM_72_bufferedio.secs
1.01 ? 5% +34.8% 1.37 ? 8% fxmark.hdd_xfs_MWUM_72_bufferedio.sys_util
1833 ? 2% +27.5% 2337 fxmark.hdd_xfs_MWUM_72_bufferedio.works/sec
355.61 -5.1% 337.48 fxmark.time.elapsed_time
355.61 -5.1% 337.48 fxmark.time.elapsed_time.max
22015 -4.8% 20965 fxmark.time.involuntary_context_switches
81060 +2.8% 83310 fxmark.time.voluntary_context_switches
137.35 -1.2% 135.70 boot-time.boot
2.18 -2.8% 2.12 boot-time.smp_boot
53.58 -4.1% 51.36 iostat.cpu.idle
28.35 +4.5% 29.62 iostat.cpu.system
43.67 ? 25% -56.5% 19.00 ? 34% numa-numastat.node1.numa_hit
43.67 ? 25% -56.5% 19.00 ? 34% numa-numastat.node1.other_node
10882 +26.9% 13814 vmstat.io.bo
7623 +4.0% 7927 vmstat.system.cs
12126 ? 6% -13.5% 10484 ? 5% meminfo.Active
12094 ? 6% -13.6% 10449 ? 5% meminfo.Active(anon)
697.67 +227.7% 2286 meminfo.Mlocked
12212 ? 6% -16.3% 10221 ? 5% numa-meminfo.node0.Active
12180 ? 6% -16.4% 10185 ? 5% numa-meminfo.node0.Active(anon)
694.67 +229.4% 2288 numa-meminfo.node0.Mlocked
1294 +13.1% 1463 ? 6% slabinfo.file_lock_cache.num_objs
1122 ? 4% -10.0% 1010 ? 9% slabinfo.kmalloc-rcl-96.active_objs
564.33 ? 5% +17.8% 664.67 ? 7% slabinfo.skbuff_fclone_cache.active_objs
764.33 ? 3% +11.3% 850.67 ? 6% slabinfo.skbuff_fclone_cache.num_objs
3047 ? 6% -16.4% 2549 ? 5% numa-vmstat.node0.nr_active_anon
271442 +22.2% 331698 numa-vmstat.node0.nr_dirtied
142.00 ? 10% +39.2% 197.67 ? 5% numa-vmstat.node0.nr_dirty
173.67 +229.2% 571.67 numa-vmstat.node0.nr_mlock
271273 +22.2% 331494 numa-vmstat.node0.nr_written
3047 ? 6% -16.4% 2549 ? 5% numa-vmstat.node0.nr_zone_active_anon
3089 ? 6% -15.5% 2611 ? 4% proc-vmstat.nr_active_anon
68420 +0.6% 68849 proc-vmstat.nr_anon_pages
465673 +19.9% 558162 proc-vmstat.nr_dirtied
134.00 ? 7% +54.7% 207.33 ? 3% proc-vmstat.nr_dirty
10656 +2.8% 10952 proc-vmstat.nr_mapped
173.00 +230.4% 571.67 proc-vmstat.nr_mlock
465673 +19.9% 558162 proc-vmstat.nr_written
3089 ? 6% -15.5% 2611 ? 4% proc-vmstat.nr_zone_active_anon
169.33 ? 6% +45.9% 247.00 ? 3% proc-vmstat.nr_zone_write_pending
1511406 -2.5% 1474355 proc-vmstat.numa_hit
1511362 -2.4% 1474336 proc-vmstat.numa_local
53181 ? 17% -25.7% 39487 ? 34% proc-vmstat.numa_pte_updates
1768573 -1.6% 1740840 proc-vmstat.pgalloc_normal
1103721 -4.6% 1052829 proc-vmstat.pgfault
1704090 -1.8% 1673299 proc-vmstat.pgfree
3714826 +20.1% 4462188 proc-vmstat.pgpgout
107504 -3.0% 104277 proc-vmstat.pgreuse
0.56 ? 7% +16.8% 0.66 perf-stat.i.MPKI
0.20 ? 5% +0.0 0.23 ? 3% perf-stat.i.branch-miss-rate%
0.42 ? 4% +0.0 0.46 ? 5% perf-stat.i.cache-miss-rate%
2763955 ? 4% +8.9% 3008571 ? 5% perf-stat.i.cache-references
6783 +3.2% 6999 perf-stat.i.context-switches
0.12 ? 8% +13.0% 0.14 ? 6% perf-stat.i.cpi
29.24 ? 7% +14.5% 33.49 ? 2% perf-stat.i.cycles-between-cache-misses
1.197e+08 ? 3% +6.8% 1.278e+08 ? 5% perf-stat.i.iTLB-loads
1.90 ? 3% +20.5% 2.29 ? 9% perf-stat.i.instructions-per-iTLB-miss
0.00 ? 3% +16.6% 0.01 ? 7% perf-stat.i.ipc
1.12 ? 7% +10.2% 1.24 ? 6% perf-stat.i.major-faults
11.53 ? 4% +8.5% 12.50 ? 5% perf-stat.i.metric.K/sec
0.52 ? 3% +7.0% 0.56 ? 5% perf-stat.i.metric.M/sec
23.11 +2.1% 23.59 perf-stat.overall.MPKI
2811536 ? 4% +8.5% 3051404 ? 6% perf-stat.ps.cache-references
6725 +3.2% 6940 perf-stat.ps.context-switches
1.216e+08 ? 3% +6.5% 1.294e+08 ? 5% perf-stat.ps.iTLB-loads
1.12 ? 7% +10.1% 1.23 ? 6% perf-stat.ps.major-faults
24319 -10.4% 21779 ? 2% softirqs.CPU10.SCHED
5160 ? 77% -59.3% 2098 ? 2% softirqs.CPU106.RCU
24458 ? 3% -9.4% 22160 softirqs.CPU14.SCHED
25258 ? 2% -11.6% 22339 softirqs.CPU15.SCHED
1034 ? 54% +350.0% 4656 ? 82% softirqs.CPU16.NET_RX
24844 -11.5% 21992 ? 6% softirqs.CPU17.SCHED
21245 -12.6% 18578 ? 5% softirqs.CPU19.SCHED
31656 ? 5% -12.1% 27818 softirqs.CPU2.SCHED
21064 -9.2% 19127 softirqs.CPU20.SCHED
21553 -15.0% 18319 ? 4% softirqs.CPU22.SCHED
6980 ? 59% -56.9% 3007 ? 3% softirqs.CPU232.RCU
20768 ? 4% -9.5% 18791 ? 3% softirqs.CPU24.SCHED
21076 -11.9% 18566 ? 2% softirqs.CPU26.SCHED
20452 ? 5% -13.3% 17735 ? 7% softirqs.CPU28.SCHED
21113 ? 3% -10.0% 19011 softirqs.CPU29.SCHED
21302 -13.6% 18416 ? 8% softirqs.CPU31.SCHED
21184 -11.0% 18849 ? 2% softirqs.CPU32.SCHED
19825 ? 4% -12.6% 17321 ? 3% softirqs.CPU34.SCHED
20150 ? 4% -8.6% 18412 ? 4% softirqs.CPU35.SCHED
16534 -12.4% 14477 ? 8% softirqs.CPU38.SCHED
16494 ? 2% -8.4% 15107 ? 2% softirqs.CPU39.SCHED
16517 -14.7% 14087 ? 8% softirqs.CPU42.SCHED
16473 -13.0% 14329 ? 5% softirqs.CPU43.SCHED
16370 ? 4% -9.0% 14896 softirqs.CPU44.SCHED
25366 ? 2% -10.3% 22741 ? 3% softirqs.CPU5.SCHED
15927 ? 4% -11.1% 14165 ? 8% softirqs.CPU50.SCHED
13790 ? 14% -14.9% 11737 ? 8% softirqs.CPU52.SCHED
11412 -9.9% 10278 softirqs.CPU56.SCHED
11935 ? 3% -20.4% 9506 ? 13% softirqs.CPU57.SCHED
11474 -8.9% 10450 ? 2% softirqs.CPU58.SCHED
11480 -13.1% 9972 ? 5% softirqs.CPU59.SCHED
24830 -10.8% 22142 softirqs.CPU6.SCHED
11474 -9.9% 10339 ? 3% softirqs.CPU65.SCHED
11310 ? 2% -17.5% 9334 ? 9% softirqs.CPU66.SCHED
11451 ? 2% -17.0% 9508 ? 8% softirqs.CPU67.SCHED
13860 ? 10% -18.1% 11352 ? 2% softirqs.CPU69.SCHED
25199 -9.7% 22746 ? 4% softirqs.CPU7.SCHED
24591 ? 3% -11.5% 21752 ? 3% softirqs.CPU8.SCHED
24328 ? 2% -8.0% 22387 ? 2% softirqs.CPU9.SCHED
1035077 +2.0% 1056227 interrupts.CAL:Function_call_interrupts
3097 -49.3% 1570 interrupts.CPU0.171:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
432.33 ? 66% +1399.3% 6482 ? 67% interrupts.CPU0.36:IR-PCI-MSI.2621442-edge.eth1-TxRx-1
2944 ? 28% -55.7% 1303 ? 7% interrupts.CPU0.NMI:Non-maskable_interrupts
2944 ? 28% -55.7% 1303 ? 7% interrupts.CPU0.PMI:Performance_monitoring_interrupts
6993 ? 10% -14.9% 5954 ? 9% interrupts.CPU0.RES:Rescheduling_interrupts
9085 ? 3% -53.5% 4220 ? 40% interrupts.CPU1.171:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
3570 ? 2% -62.7% 1331 ? 7% interrupts.CPU1.NMI:Non-maskable_interrupts
3570 ? 2% -62.7% 1331 ? 7% interrupts.CPU1.PMI:Performance_monitoring_interrupts
9354 ? 12% -26.8% 6845 ? 9% interrupts.CPU10.CAL:Function_call_interrupts
3480 ? 3% -49.8% 1747 ? 39% interrupts.CPU12.NMI:Non-maskable_interrupts
3480 ? 3% -49.8% 1747 ? 39% interrupts.CPU12.PMI:Performance_monitoring_interrupts
1748 ? 3% -27.1% 1274 ? 5% interrupts.CPU15.NMI:Non-maskable_interrupts
1748 ? 3% -27.1% 1274 ? 5% interrupts.CPU15.PMI:Performance_monitoring_interrupts
2294 ? 31% -43.9% 1287 ? 5% interrupts.CPU17.NMI:Non-maskable_interrupts
2294 ? 31% -43.9% 1287 ? 5% interrupts.CPU17.PMI:Performance_monitoring_interrupts
2917 ? 25% -55.3% 1305 ? 6% interrupts.CPU18.NMI:Non-maskable_interrupts
2917 ? 25% -55.3% 1305 ? 6% interrupts.CPU18.PMI:Performance_monitoring_interrupts
12689 ? 7% +16.1% 14727 ? 2% interrupts.CPU2.CAL:Function_call_interrupts
2372 ? 32% -42.7% 1358 ? 6% interrupts.CPU2.NMI:Non-maskable_interrupts
2372 ? 32% -42.7% 1358 ? 6% interrupts.CPU2.PMI:Performance_monitoring_interrupts
2757 ? 28% -52.5% 1310 ? 8% interrupts.CPU20.NMI:Non-maskable_interrupts
2757 ? 28% -52.5% 1310 ? 8% interrupts.CPU20.PMI:Performance_monitoring_interrupts
2884 ? 31% -54.8% 1302 ? 4% interrupts.CPU28.NMI:Non-maskable_interrupts
2884 ? 31% -54.8% 1302 ? 4% interrupts.CPU28.PMI:Performance_monitoring_interrupts
5032 ? 10% +22.0% 6137 ? 15% interrupts.CPU29.CAL:Function_call_interrupts
2830 ? 29% -54.9% 1276 ? 6% interrupts.CPU29.NMI:Non-maskable_interrupts
2830 ? 29% -54.9% 1276 ? 6% interrupts.CPU29.PMI:Performance_monitoring_interrupts
1175 ? 7% -25.9% 870.67 ? 11% interrupts.CPU3.RES:Rescheduling_interrupts
2867 ? 29% -55.2% 1285 ? 6% interrupts.CPU33.NMI:Non-maskable_interrupts
2867 ? 29% -55.2% 1285 ? 6% interrupts.CPU33.PMI:Performance_monitoring_interrupts
2876 ? 29% -55.8% 1271 ? 4% interrupts.CPU34.NMI:Non-maskable_interrupts
2876 ? 29% -55.8% 1271 ? 4% interrupts.CPU34.PMI:Performance_monitoring_interrupts
5063 ? 14% +33.3% 6750 ? 21% interrupts.CPU35.CAL:Function_call_interrupts
2907 ? 30% -56.9% 1254 ? 5% interrupts.CPU35.NMI:Non-maskable_interrupts
2907 ? 30% -56.9% 1254 ? 5% interrupts.CPU35.PMI:Performance_monitoring_interrupts
3814 +58.2% 6033 ? 13% interrupts.CPU36.CAL:Function_call_interrupts
3756 ? 3% +48.6% 5583 ? 12% interrupts.CPU37.CAL:Function_call_interrupts
2285 ? 33% -43.6% 1290 ? 6% interrupts.CPU37.NMI:Non-maskable_interrupts
2285 ? 33% -43.6% 1290 ? 6% interrupts.CPU37.PMI:Performance_monitoring_interrupts
2259 ? 33% -43.1% 1285 ? 7% interrupts.CPU38.NMI:Non-maskable_interrupts
2259 ? 33% -43.1% 1285 ? 7% interrupts.CPU38.PMI:Performance_monitoring_interrupts
3398 -62.3% 1280 ? 9% interrupts.CPU39.NMI:Non-maskable_interrupts
3398 -62.3% 1280 ? 9% interrupts.CPU39.PMI:Performance_monitoring_interrupts
1747 ? 2% +49.9% 2619 ? 6% interrupts.CPU4.NMI:Non-maskable_interrupts
1747 ? 2% +49.9% 2619 ? 6% interrupts.CPU4.PMI:Performance_monitoring_interrupts
5027 ? 14% -16.6% 4194 ? 14% interrupts.CPU40.CAL:Function_call_interrupts
2850 ? 29% -41.9% 1656 ? 25% interrupts.CPU40.NMI:Non-maskable_interrupts
2850 ? 29% -41.9% 1656 ? 25% interrupts.CPU40.PMI:Performance_monitoring_interrupts
3913 ? 8% +13.9% 4458 ? 9% interrupts.CPU43.CAL:Function_call_interrupts
2855 ? 25% -55.8% 1262 ? 7% interrupts.CPU43.NMI:Non-maskable_interrupts
2855 ? 25% -55.8% 1262 ? 7% interrupts.CPU43.PMI:Performance_monitoring_interrupts
2250 ? 34% -37.8% 1400 ? 14% interrupts.CPU47.NMI:Non-maskable_interrupts
2250 ? 34% -37.8% 1400 ? 14% interrupts.CPU47.PMI:Performance_monitoring_interrupts
2915 ? 32% -40.8% 1725 ? 39% interrupts.CPU50.NMI:Non-maskable_interrupts
2915 ? 32% -40.8% 1725 ? 39% interrupts.CPU50.PMI:Performance_monitoring_interrupts
3451 ? 4% -37.0% 2173 ? 26% interrupts.CPU51.NMI:Non-maskable_interrupts
3451 ? 4% -37.0% 2173 ? 26% interrupts.CPU51.PMI:Performance_monitoring_interrupts
2376 ? 40% -44.4% 1321 ? 5% interrupts.CPU53.NMI:Non-maskable_interrupts
2376 ? 40% -44.4% 1321 ? 5% interrupts.CPU53.PMI:Performance_monitoring_interrupts
3284 +20.4% 3954 ? 16% interrupts.CPU65.CAL:Function_call_interrupts
217388 -11.9% 191509 ? 6% interrupts.CPU68.LOC:Local_timer_interrupts
4542 ? 9% +24.8% 5670 ? 9% interrupts.CPU69.CAL:Function_call_interrupts
6208 ? 3% +27.5% 7914 ? 17% interrupts.CPU9.CAL:Function_call_interrupts
133580 ? 4% -31.2% 91950 ? 5% interrupts.NMI:Non-maskable_interrupts
133580 ? 4% -31.2% 91950 ? 5% interrupts.PMI:Performance_monitoring_interrupts
63.06 ? 70% -63.1 0.00 perf-profile.calltrace.cycles-pp.open64
63.04 ? 70% -63.0 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.open64
63.03 ? 70% -63.0 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
63.02 ? 70% -63.0 0.00 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
63.02 ? 70% -63.0 0.00 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
89.31 -4.1 85.16 perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
92.25 -3.6 88.67 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2.do_sys_open
94.89 -3.3 91.57 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
94.88 -3.3 91.57 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
0.89 ? 2% +0.1 0.95 ? 5% perf-profile.calltrace.cycles-pp.xfs_dialloc.xfs_create.xfs_generic_create.path_openat.do_filp_open
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.calltrace.cycles-pp.__libc_start_main
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.calltrace.cycles-pp.main.__libc_start_main
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.calltrace.cycles-pp.run_builtin.main.__libc_start_main
1.93 ? 2% +0.2 2.14 ? 5% perf-profile.calltrace.cycles-pp.xfs_generic_create.path_openat.do_filp_open.do_sys_openat2.do_sys_open
1.89 +0.2 2.10 ? 5% perf-profile.calltrace.cycles-pp.xfs_create.xfs_generic_create.path_openat.do_filp_open.do_sys_openat2
0.19 ?141% +0.4 0.60 ? 11% perf-profile.calltrace.cycles-pp.perf_session__process_events.record__finish_output.cmd_record.cmd_sched.run_builtin
0.19 ?141% +0.4 0.60 ? 12% perf-profile.calltrace.cycles-pp.record__finish_output.cmd_record.cmd_sched.run_builtin.main
0.19 ?141% +0.4 0.63 ? 11% perf-profile.calltrace.cycles-pp.cmd_record.cmd_sched.run_builtin.main.__libc_start_main
0.19 ?141% +0.4 0.63 ? 11% perf-profile.calltrace.cycles-pp.cmd_sched.run_builtin.main.__libc_start_main
2.66 +0.5 3.13 ? 4% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
1.99 ? 11% +0.6 2.60 ? 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.7 0.65 ? 13% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.66 ? 15% +0.7 1.33 ? 12% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.46 ? 70% +0.8 1.25 ? 12% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.07 ? 10% +1.0 2.11 ? 12% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
1.13 ? 10% +1.1 2.25 ? 12% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
1.46 ? 10% +1.5 2.95 ? 12% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
1.51 ? 10% +1.5 3.05 ? 11% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.10 ? 9% +2.1 4.23 ? 11% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.10 ? 9% +2.1 4.23 ? 11% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
2.09 ? 9% +2.1 4.22 ? 11% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.16 ? 9% +2.1 4.29 ? 11% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
63.06 ? 70% -63.1 0.00 perf-profile.children.cycles-pp.open64
89.36 -4.1 85.22 perf-profile.children.cycles-pp.osq_lock
92.26 -3.3 88.92 perf-profile.children.cycles-pp.rwsem_down_write_slowpath
95.00 -3.3 91.68 perf-profile.children.cycles-pp.do_sys_open
95.00 -3.3 91.68 perf-profile.children.cycles-pp.do_sys_openat2
94.94 -3.3 91.62 perf-profile.children.cycles-pp.do_filp_open
94.93 -3.3 91.62 perf-profile.children.cycles-pp.path_openat
96.10 -2.5 93.63 perf-profile.children.cycles-pp.do_syscall_64
96.11 -2.5 93.64 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.05 +0.0 0.06 ? 7% perf-profile.children.cycles-pp.iomap_apply
0.05 +0.0 0.06 ? 7% perf-profile.children.cycles-pp.iomap_file_buffered_write
0.07 ? 11% +0.0 0.09 ? 5% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.06 ? 7% +0.0 0.08 ? 10% perf-profile.children.cycles-pp.do_fault
0.05 ? 8% +0.0 0.07 perf-profile.children.cycles-pp.xfs_buf_item_format_segment
0.06 ? 7% +0.0 0.08 ? 5% perf-profile.children.cycles-pp.do_iter_readv_writev
0.06 ? 7% +0.0 0.08 ? 5% perf-profile.children.cycles-pp.xfs_file_buffered_write
0.10 ? 4% +0.0 0.12 ? 10% perf-profile.children.cycles-pp._raw_spin_lock
0.09 ? 5% +0.0 0.11 ? 4% perf-profile.children.cycles-pp.walk_component
0.07 ? 7% +0.0 0.09 ? 5% perf-profile.children.cycles-pp.xfs_buf_item_format
0.16 ? 3% +0.0 0.18 ? 2% perf-profile.children.cycles-pp.xfs_da3_node_read
0.06 ? 7% +0.0 0.09 ? 5% perf-profile.children.cycles-pp.do_iter_write
0.05 ? 8% +0.0 0.08 ? 12% perf-profile.children.cycles-pp.ttwu_do_activate
0.05 ? 8% +0.0 0.08 ? 12% perf-profile.children.cycles-pp.enqueue_task_fair
0.11 ? 8% +0.0 0.13 ? 12% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.08 ? 10% +0.0 0.11 ? 8% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.06 ? 7% +0.0 0.09 ? 9% perf-profile.children.cycles-pp.lo_write_bvec
0.12 ? 10% +0.0 0.15 ? 3% perf-profile.children.cycles-pp.__handle_mm_fault
0.11 ? 8% +0.0 0.14 ? 13% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.05 +0.0 0.08 ? 10% perf-profile.children.cycles-pp.__hrtimer_get_next_event
0.21 ? 3% +0.0 0.24 ? 3% perf-profile.children.cycles-pp.xfs_da_read_buf
0.09 ? 5% +0.0 0.12 ? 15% perf-profile.children.cycles-pp.link_path_walk
0.06 ? 8% +0.0 0.09 ? 19% perf-profile.children.cycles-pp.xfs_next_bit
0.05 +0.0 0.08 ? 14% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle
0.17 ? 2% +0.0 0.21 ? 11% perf-profile.children.cycles-pp.__libc_write
0.17 ? 4% +0.0 0.20 ? 12% perf-profile.children.cycles-pp.__generic_file_write_iter
0.16 ? 7% +0.0 0.20 ? 8% perf-profile.children.cycles-pp.do_user_addr_fault
0.13 ? 9% +0.0 0.16 ? 5% perf-profile.children.cycles-pp.build_id__mark_dso_hit
0.05 ? 8% +0.0 0.09 ? 10% perf-profile.children.cycles-pp.timerqueue_del
0.03 ? 70% +0.0 0.07 ? 14% perf-profile.children.cycles-pp.sysvec_call_function_single
0.08 ? 10% +0.0 0.11 ? 16% perf-profile.children.cycles-pp.update_irq_load_avg
0.08 ? 16% +0.0 0.11 ? 19% perf-profile.children.cycles-pp.execve
0.12 +0.0 0.15 ? 6% perf-profile.children.cycles-pp.wake_up_q
0.19 ? 4% +0.0 0.23 ? 11% perf-profile.children.cycles-pp.asm_exc_page_fault
0.16 ? 5% +0.0 0.20 ? 14% perf-profile.children.cycles-pp.generic_perform_write
0.17 ? 4% +0.0 0.21 ? 4% perf-profile.children.cycles-pp.kmem_cache_alloc
0.13 ? 3% +0.0 0.16 ? 5% perf-profile.children.cycles-pp.rwsem_wake
0.08 ? 6% +0.0 0.11 ? 18% perf-profile.children.cycles-pp.xfs_inobt_irec_to_allocmask
0.07 ? 7% +0.0 0.10 ? 12% perf-profile.children.cycles-pp.read_tsc
0.07 ? 25% +0.0 0.10 ? 12% perf-profile.children.cycles-pp.xfs_file_fsync
0.04 ? 71% +0.0 0.07 ? 17% perf-profile.children.cycles-pp.xfsaild
0.06 ? 8% +0.0 0.09 ? 5% perf-profile.children.cycles-pp.rcu_core
0.03 ? 70% +0.0 0.07 ? 11% perf-profile.children.cycles-pp.xfsaild_push
0.13 ? 9% +0.0 0.17 ? 4% perf-profile.children.cycles-pp.handle_mm_fault
0.17 ? 4% +0.0 0.21 ? 10% perf-profile.children.cycles-pp.exc_page_fault
0.16 ? 8% +0.0 0.20 ? 7% perf-profile.children.cycles-pp.__xfs_btree_check_sblock
0.13 ? 13% +0.0 0.17 ? 7% perf-profile.children.cycles-pp.xfs_dir2_node_addname_int
0.06 ? 16% +0.0 0.10 ? 9% perf-profile.children.cycles-pp.__poll
0.06 ? 16% +0.0 0.10 ? 9% perf-profile.children.cycles-pp.__x64_sys_poll
0.06 ? 16% +0.0 0.10 ? 9% perf-profile.children.cycles-pp.do_sys_poll
0.02 ?141% +0.0 0.06 ? 8% perf-profile.children.cycles-pp.rb_insert_color
0.02 ?141% +0.0 0.06 ? 8% perf-profile.children.cycles-pp.schedule_timeout
0.03 ? 70% +0.0 0.07 ? 17% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.17 ? 7% +0.0 0.21 ? 11% perf-profile.children.cycles-pp.generic_file_write_iter
0.13 ? 3% +0.0 0.17 ? 7% perf-profile.children.cycles-pp.xfs_btree_lookup
0.07 ? 20% +0.0 0.11 ? 12% perf-profile.children.cycles-pp.lapic_next_deadline
0.08 ? 6% +0.0 0.12 ? 11% perf-profile.children.cycles-pp.uart_console_write
0.31 ? 6% +0.0 0.35 ? 3% perf-profile.children.cycles-pp.xfs_dir_createname
0.14 ? 5% +0.0 0.18 ? 10% perf-profile.children.cycles-pp.xlog_cil_insert_items
0.12 ? 6% +0.0 0.16 ? 16% perf-profile.children.cycles-pp.__percpu_counter_sum
0.04 ? 71% +0.0 0.08 ? 10% perf-profile.children.cycles-pp.native_irq_return_iret
0.11 ? 7% +0.0 0.15 ? 8% perf-profile.children.cycles-pp.xfs_buf_find
0.22 ? 7% +0.0 0.26 ? 8% perf-profile.children.cycles-pp.machines__deliver_event
0.07 ? 6% +0.0 0.12 ? 11% perf-profile.children.cycles-pp.serial8250_console_putchar
0.17 ? 2% +0.0 0.22 ? 12% perf-profile.children.cycles-pp.xfs_trans_alloc_icreate
0.08 ? 6% +0.0 0.12 ? 13% perf-profile.children.cycles-pp.wait_for_xmitr
0.12 ? 3% +0.0 0.17 ? 12% perf-profile.children.cycles-pp.__percpu_counter_compare
0.07 ? 11% +0.0 0.12 ? 14% perf-profile.children.cycles-pp.tick_irq_enter
0.23 ? 12% +0.0 0.28 ? 3% perf-profile.children.cycles-pp.update_load_avg
0.27 ? 8% +0.0 0.32 ? 2% perf-profile.children.cycles-pp.xfs_dir2_node_addname
0.15 ? 3% +0.0 0.20 ? 6% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.08 ? 14% +0.1 0.13 ? 19% perf-profile.children.cycles-pp.irq_enter_rcu
0.08 ? 6% +0.1 0.13 ? 14% perf-profile.children.cycles-pp.serial8250_console_write
0.02 ?141% +0.1 0.07 ? 14% perf-profile.children.cycles-pp.enqueue_entity
0.02 ?141% +0.1 0.07 ? 7% perf-profile.children.cycles-pp.error_entry
0.00 +0.1 0.05 perf-profile.children.cycles-pp.do_dentry_open
0.00 +0.1 0.05 perf-profile.children.cycles-pp.xlog_cil_committed
0.00 +0.1 0.05 perf-profile.children.cycles-pp.iomap_write_actor
0.00 +0.1 0.05 perf-profile.children.cycles-pp.xlog_cil_process_committed
0.00 +0.1 0.05 perf-profile.children.cycles-pp.kblockd_mod_delayed_work_on
0.00 +0.1 0.05 perf-profile.children.cycles-pp.mod_delayed_work_on
0.20 ? 4% +0.1 0.25 ? 8% perf-profile.children.cycles-pp.xfs_btree_get_rec
0.15 ? 12% +0.1 0.20 ? 10% perf-profile.children.cycles-pp.perf_trace_sched_switch
0.13 ? 6% +0.1 0.18 ? 12% perf-profile.children.cycles-pp.xfs_mod_fdblocks
0.15 ? 3% +0.1 0.20 ? 7% perf-profile.children.cycles-pp.xfs_buf_read_map
0.11 ? 7% +0.1 0.16 ? 7% perf-profile.children.cycles-pp.irqtime_account_irq
0.04 ? 70% +0.1 0.09 ? 13% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.00 +0.1 0.05 ? 8% perf-profile.children.cycles-pp.__kmalloc
0.00 +0.1 0.05 ? 8% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.13 ? 7% +0.1 0.19 ? 5% perf-profile.children.cycles-pp.xfs_buf_get_map
0.02 ?141% +0.1 0.07 ? 11% perf-profile.children.cycles-pp.idle_cpu
0.19 ? 4% +0.1 0.25 ? 10% perf-profile.children.cycles-pp.xfs_btree_increment
0.10 +0.1 0.16 ? 7% perf-profile.children.cycles-pp.native_sched_clock
0.06 ? 7% +0.1 0.12 ? 6% perf-profile.children.cycles-pp.__remove_hrtimer
0.10 ? 9% +0.1 0.16 ? 5% perf-profile.children.cycles-pp.dequeue_entity
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.__cond_resched
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.sched_ttwu_pending
0.02 ?141% +0.1 0.07 ? 17% perf-profile.children.cycles-pp.blk_complete_reqs
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.xfs_read_agi
0.04 ? 70% +0.1 0.10 ? 4% perf-profile.children.cycles-pp.io_serial_in
0.14 ? 15% +0.1 0.20 ? 4% perf-profile.children.cycles-pp.loop_queue_work
0.11 ? 16% +0.1 0.17 ? 15% perf-profile.children.cycles-pp.update_rq_clock
0.00 +0.1 0.06 ? 13% perf-profile.children.cycles-pp.xfs_bmapi_allocate
0.00 +0.1 0.06 perf-profile.children.cycles-pp.xlog_ioend_work
0.00 +0.1 0.06 perf-profile.children.cycles-pp.xlog_state_do_callback
0.11 ? 7% +0.1 0.17 ? 4% perf-profile.children.cycles-pp.schedule_idle
0.24 ? 5% +0.1 0.30 ? 4% perf-profile.children.cycles-pp.xfs_trans_read_buf_map
0.24 ? 5% +0.1 0.30 ? 9% perf-profile.children.cycles-pp.xfs_da3_node_lookup_int
0.23 ? 15% +0.1 0.29 ? 9% perf-profile.children.cycles-pp.perf_session__process_user_event
0.23 ? 15% +0.1 0.29 ? 9% perf-profile.children.cycles-pp.__ordered_events__flush
0.09 ? 10% +0.1 0.15 ? 5% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.09 ? 9% +0.1 0.15 ? 8% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.08 ? 5% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.printk
0.08 ? 14% +0.1 0.15 ? 8% perf-profile.children.cycles-pp.update_blocked_averages
0.00 +0.1 0.06 ? 14% perf-profile.children.cycles-pp.finish_task_switch
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.perf_poll
0.11 ? 12% +0.1 0.17 ? 7% perf-profile.children.cycles-pp.dequeue_task_fair
0.13 ? 3% +0.1 0.19 ? 4% perf-profile.children.cycles-pp.sched_clock_cpu
0.17 ? 24% +0.1 0.24 ? 7% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.14 ? 21% +0.1 0.21 ? 6% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.08 ? 12% +0.1 0.14 ? 14% perf-profile.children.cycles-pp.run_rebalance_domains
0.00 +0.1 0.07 ? 25% perf-profile.children.cycles-pp.xfs_dir2_leafn_lookup_for_entry
0.24 ? 7% +0.1 0.30 ? 9% perf-profile.children.cycles-pp.xfs_btree_check_sblock
0.09 ? 5% +0.1 0.15 ? 15% perf-profile.children.cycles-pp.vprintk_emit
0.09 ? 5% +0.1 0.15 ? 15% perf-profile.children.cycles-pp.console_unlock
0.89 ? 2% +0.1 0.95 ? 5% perf-profile.children.cycles-pp.xfs_dialloc
0.28 ? 9% +0.1 0.35 ? 8% perf-profile.children.cycles-pp.perf_session__deliver_event
0.15 ? 8% +0.1 0.22 ? 9% perf-profile.children.cycles-pp.newidle_balance
0.15 ? 14% +0.1 0.22 ? 3% perf-profile.children.cycles-pp.kthread_worker_fn
0.00 +0.1 0.07 perf-profile.children.cycles-pp.blk_mq_submit_bio
0.12 ? 8% +0.1 0.19 ? 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +0.1 0.07 ? 6% perf-profile.children.cycles-pp.__queue_work
0.16 ? 6% +0.1 0.23 ? 7% perf-profile.children.cycles-pp.pick_next_task_fair
0.00 +0.1 0.08 ? 26% perf-profile.children.cycles-pp.xfs_dir2_node_removename
0.16 ? 5% +0.1 0.24 ? 14% perf-profile.children.cycles-pp.xfs_trans_reserve
0.11 ? 7% +0.1 0.19 ? 11% perf-profile.children.cycles-pp.rebalance_domains
0.00 +0.1 0.08 ? 20% perf-profile.children.cycles-pp.xfs_dir_removename
0.18 ? 4% +0.1 0.26 ? 15% perf-profile.children.cycles-pp.xfs_trans_alloc
0.07 ? 23% +0.1 0.16 ? 10% perf-profile.children.cycles-pp.irq_work_run_list
0.15 ? 6% +0.1 0.25 ? 10% perf-profile.children.cycles-pp.update_sd_lb_stats
0.07 ? 23% +0.1 0.16 ? 10% perf-profile.children.cycles-pp.tick_nohz_next_event
0.05 ? 72% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.asm_sysvec_irq_work
0.05 ? 72% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.sysvec_irq_work
0.05 ? 72% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.__sysvec_irq_work
0.05 ? 72% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.irq_work_run
0.05 ? 72% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.irq_work_single
0.00 +0.1 0.09 ? 10% perf-profile.children.cycles-pp.submit_bio
0.00 +0.1 0.09 ? 10% perf-profile.children.cycles-pp.submit_bio_noacct
0.00 +0.1 0.09 ? 10% perf-profile.children.cycles-pp.xlog_state_release_iclog
0.48 ? 9% +0.1 0.57 ? 12% perf-profile.children.cycles-pp.process_simple
0.21 ? 2% +0.1 0.31 ? 4% perf-profile.children.cycles-pp.try_to_wake_up
0.16 ? 8% +0.1 0.26 ? 8% perf-profile.children.cycles-pp.find_busiest_group
0.53 ? 9% +0.1 0.63 ? 11% perf-profile.children.cycles-pp.cmd_record
0.30 ? 4% +0.1 0.40 ? 8% perf-profile.children.cycles-pp.xfs_log_commit_cil
0.50 ? 9% +0.1 0.60 ? 12% perf-profile.children.cycles-pp.record__finish_output
0.50 ? 9% +0.1 0.60 ? 12% perf-profile.children.cycles-pp.perf_session__process_events
0.18 ? 11% +0.1 0.28 perf-profile.children.cycles-pp.clockevents_program_event
0.16 ? 13% +0.1 0.27 ? 6% perf-profile.children.cycles-pp.ktime_get
0.37 ? 3% +0.1 0.48 ? 10% perf-profile.children.cycles-pp.xfs_inobt_get_rec
0.51 ? 9% +0.1 0.63 ? 11% perf-profile.children.cycles-pp.cmd_sched
0.32 ? 4% +0.1 0.44 ? 8% perf-profile.children.cycles-pp.__xfs_trans_commit
0.00 +0.1 0.12 ? 11% perf-profile.children.cycles-pp.xlog_write
0.04 ? 71% +0.1 0.16 ? 10% perf-profile.children.cycles-pp.xlog_cil_push_work
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.children.cycles-pp.__libc_start_main
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.children.cycles-pp.main
0.57 ? 10% +0.1 0.70 ? 10% perf-profile.children.cycles-pp.run_builtin
0.23 ? 9% +0.1 0.36 ? 6% perf-profile.children.cycles-pp.load_balance
0.00 +0.1 0.13 ? 33% perf-profile.children.cycles-pp.xfs_difree_finobt
0.10 ? 19% +0.1 0.24 ? 13% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.2 0.16 ? 31% perf-profile.children.cycles-pp.xfs_difree
0.64 ? 3% +0.2 0.81 ? 9% perf-profile.children.cycles-pp.xfs_check_agi_freecount
0.38 ? 7% +0.2 0.56 ? 9% perf-profile.children.cycles-pp.schedule
0.00 +0.2 0.18 ? 34% perf-profile.children.cycles-pp.xfs_ifree
1.94 ? 2% +0.2 2.14 ? 5% perf-profile.children.cycles-pp.xfs_generic_create
1.89 +0.2 2.10 ? 5% perf-profile.children.cycles-pp.xfs_create
0.24 ? 12% +0.2 0.47 ? 14% perf-profile.children.cycles-pp.intel_idle
0.00 +0.2 0.22 ? 20% perf-profile.children.cycles-pp.xfs_vn_unlink
0.00 +0.2 0.22 ? 20% perf-profile.children.cycles-pp.xfs_remove
0.00 +0.2 0.23 ? 33% perf-profile.children.cycles-pp.xfs_inactive
0.00 +0.2 0.23 ? 33% perf-profile.children.cycles-pp.xfs_inactive_ifree
0.00 +0.2 0.23 ? 33% perf-profile.children.cycles-pp.xfs_fs_destroy_inode
0.00 +0.2 0.23 ? 19% perf-profile.children.cycles-pp.vfs_unlink
0.00 +0.2 0.23 ? 34% perf-profile.children.cycles-pp.destroy_inode
0.33 ? 5% +0.2 0.57 ? 9% perf-profile.children.cycles-pp.__softirqentry_text_start
0.51 ? 7% +0.2 0.76 ? 7% perf-profile.children.cycles-pp.__schedule
0.43 ? 6% +0.3 0.72 ? 8% perf-profile.children.cycles-pp.irq_exit_rcu
0.31 ? 9% +0.4 0.67 ? 13% perf-profile.children.cycles-pp.menu_select
2.62 ? 10% +0.4 3.05 ? 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
2.67 +0.7 3.33 ? 4% perf-profile.children.cycles-pp.rwsem_spin_on_owner
3.17 ? 9% +0.7 3.90 ? 4% perf-profile.children.cycles-pp.hrtimer_interrupt
0.00 +0.7 0.74 ? 26% perf-profile.children.cycles-pp.do_unlinkat
3.24 ? 9% +0.8 4.01 ? 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
3.85 ? 7% +1.2 5.03 ? 5% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
4.08 ? 8% +1.2 5.31 ? 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
1.55 ? 10% +1.5 3.07 ? 11% perf-profile.children.cycles-pp.cpuidle_enter_state
1.55 ? 11% +1.5 3.09 ? 11% perf-profile.children.cycles-pp.cpuidle_enter
2.10 ? 9% +2.1 4.23 ? 11% perf-profile.children.cycles-pp.start_secondary
2.16 ? 9% +2.1 4.29 ? 11% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
2.16 ? 9% +2.1 4.29 ? 11% perf-profile.children.cycles-pp.cpu_startup_entry
2.16 ? 9% +2.1 4.29 ? 11% perf-profile.children.cycles-pp.do_idle
86.82 -4.0 82.82 perf-profile.self.cycles-pp.osq_lock
0.06 ? 7% +0.0 0.08 ? 10% perf-profile.self.cycles-pp.kmem_cache_alloc
0.08 ? 10% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.update_load_avg
0.06 ? 13% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.09 ? 9% +0.0 0.11 ? 7% perf-profile.self.cycles-pp._raw_spin_lock
0.09 ? 10% +0.0 0.12 ? 4% perf-profile.self.cycles-pp.__xfs_btree_check_sblock
0.06 ? 16% +0.0 0.08 ? 17% perf-profile.self.cycles-pp.update_cfs_group
0.06 ? 8% +0.0 0.08 ? 17% perf-profile.self.cycles-pp.xfs_next_bit
0.07 ? 6% +0.0 0.10 ? 8% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.10 ? 4% +0.0 0.13 ? 16% perf-profile.self.cycles-pp.__percpu_counter_sum
0.07 ? 17% +0.0 0.11 ? 15% perf-profile.self.cycles-pp.update_irq_load_avg
0.06 ? 13% +0.0 0.10 ? 31% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.06 ? 14% +0.0 0.10 ? 12% perf-profile.self.cycles-pp.read_tsc
0.06 ? 7% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.hrtimer_interrupt
0.02 ?141% +0.0 0.06 ? 8% perf-profile.self.cycles-pp.update_curr
0.07 ? 20% +0.0 0.11 ? 12% perf-profile.self.cycles-pp.lapic_next_deadline
0.07 ? 6% +0.0 0.12 ? 14% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.04 ? 71% +0.0 0.08 ? 10% perf-profile.self.cycles-pp.native_irq_return_iret
0.02 ?141% +0.0 0.07 ? 18% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.10 ? 4% +0.0 0.15 ? 8% perf-profile.self.cycles-pp.native_sched_clock
0.02 ?141% +0.1 0.07 ? 25% perf-profile.self.cycles-pp.irqtime_account_irq
0.02 ?141% +0.1 0.07 ? 7% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.1 0.05 perf-profile.self.cycles-pp.irq_exit_rcu
0.00 +0.1 0.05 perf-profile.self.cycles-pp.xfs_btree_get_rec
0.03 ? 70% +0.1 0.09 ? 14% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.00 +0.1 0.05 ? 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.02 ?141% +0.1 0.07 ? 11% perf-profile.self.cycles-pp.idle_cpu
0.08 ? 14% +0.1 0.14 ? 17% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.1 0.06 ? 8% perf-profile.self.cycles-pp.error_entry
0.00 +0.1 0.06 ? 16% perf-profile.self.cycles-pp.scheduler_tick
0.04 ? 70% +0.1 0.10 ? 4% perf-profile.self.cycles-pp.io_serial_in
0.16 ? 21% +0.1 0.21 ? 9% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.03 ? 70% +0.1 0.10 ? 12% perf-profile.self.cycles-pp.do_idle
0.10 +0.1 0.17 ? 23% perf-profile.self.cycles-pp.update_process_times
0.10 ? 8% +0.1 0.17 ? 11% perf-profile.self.cycles-pp.update_sd_lb_stats
0.00 +0.1 0.07 ? 14% perf-profile.self.cycles-pp.tick_nohz_next_event
0.08 ? 10% +0.1 0.16 ? 10% perf-profile.self.cycles-pp.cpuidle_enter_state
0.10 ? 16% +0.1 0.18 ? 12% perf-profile.self.cycles-pp.ktime_get
0.18 ? 10% +0.2 0.35 ? 12% perf-profile.self.cycles-pp.menu_select
0.24 ? 11% +0.2 0.47 ? 14% perf-profile.self.cycles-pp.intel_idle
2.59 +0.6 3.21 ? 4% perf-profile.self.cycles-pp.rwsem_spin_on_owner
0.03 ? 3% -100.0% 0.00 perf-sched.sch_delay.avg.ms.blk_mq_freeze_queue_wait.lo_release.__blkdev_put.blkdev_put
0.06 ? 18% +2809.6% 1.62 ? 77% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.03 ? 9% -100.0% 0.00 perf-sched.sch_delay.avg.ms.devtmpfs_work_loop.devtmpfsd.kthread.ret_from_fork
0.02 ? 22% +100.0% 0.05 ? 16% perf-sched.sch_delay.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.04 ? 5% -27.2% 0.03 ? 5% perf-sched.sch_delay.avg.ms.kthread_worker_fn.kthread.ret_from_fork
0.20 ?138% -100.0% 0.00 perf-sched.sch_delay.avg.ms.percpu_down_write._cpu_down.cpu_down.device_offline
0.25 ? 7% -100.0% 0.00 perf-sched.sch_delay.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
6.73 ?140% -100.0% 0.00 perf-sched.sch_delay.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
0.27 ?124% -100.0% 0.00 perf-sched.sch_delay.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_start
0.00 ? 10% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__alloc_pages.pte_alloc_one.__pte_alloc
0.48 ? 77% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.lru_add_drain_all.khugepaged
0.02 ? 63% +1336.7% 0.23 ?125% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xfs_dir_lookup
0.04 ? 35% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xlog_alloc_log
0.03 ? 53% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.dput.simple_recursive_removal.debugfs_remove
0.01 ? 43% -85.0% 0.00 ?141% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.vm_area_dup.__split_vma
0.51 ?132% -97.1% 0.02 ? 23% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
0.02 ? 38% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.crypto_exit_scomp_ops_async.crypto_destroy_tfm
0.02 ? 36% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.device_del.device_unregister
0.03 ? 2% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.flush_workqueue.xfs_buftarg_drain
0.03 ? 4% -63.9% 0.01 ? 14% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.submit_bio_checks.submit_bio_noacct.submit_bio
0.02 ? 2% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.synchronize_rcu_expedited.namespace_unlock.path_umount
0.00 ? 28% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.vfree.free_vm_stack_cache.cpuhp_invoke_callback
0.01 ?107% -88.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
0.03 ? 11% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.xfs_rw_bdev
0.04 ? 28% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.write_inode_now.iput.xfs_rtunmount_inodes
0.04 ? 19% -100.0% 0.00 perf-sched.sch_delay.avg.ms.rescuer_thread.kthread.ret_from_fork
0.12 ? 15% -52.7% 0.06 ? 24% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.10 ? 80% -81.7% 0.02 ? 7% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.04 -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
0.06 ? 37% -51.1% 0.03 ? 7% perf-sched.sch_delay.avg.ms.schedule_timeout.__down.down.xfs_buf_lock
0.02 ? 4% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_direct_write
0.03 -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_read_iter
0.03 ? 4% +25.6% 0.03 ? 4% perf-sched.sch_delay.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
0.05 ? 3% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.sync_rcu_exp_select_cpus
0.03 ? 12% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
0.14 ? 9% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__wait_rcu_gp.synchronize_rcu
0.05 ? 60% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.cpuhp_kick_ap.cpuhp_kick_ap_work
0.03 -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.devtmpfs_submit_req.devtmpfs_delete_node
0.01 ? 38% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.kthread_stop.destroy_workqueue
0.05 ? 20% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.kthread_stop.xfs_trans_ail_destroy
0.04 ? 20% +150.8% 0.10 ? 41% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion_killable.__kthread_create_on_node.kthread_create_on_node
0.09 ? 26% -48.2% 0.05 ? 22% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.02 -19.0% 0.02 ? 8% perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
0.08 ? 3% +18.0% 0.09 ? 2% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.07 ? 22% -100.0% 0.00 perf-sched.sch_delay.avg.ms.xfs_ail_push_all_sync.xfs_log_quiesce.xfs_log_unmount.xfs_unmountfs
0.05 ? 55% -100.0% 0.00 perf-sched.sch_delay.avg.ms.xfsaild.kthread.ret_from_fork
0.05 ? 12% -37.6% 0.03 ? 3% perf-sched.sch_delay.avg.ms.xlog_state_get_iclog_space.xlog_write.xlog_cil_push_work.process_one_work
0.06 ? 79% -100.0% 0.00 perf-sched.sch_delay.avg.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.__xfs_trans_commit
0.02 ? 11% -100.0% 0.00 perf-sched.sch_delay.avg.ms.xlog_wait_on_iclog.xfs_log_unmount_write.xfs_log_unmount.xfs_unmountfs
0.06 ? 3% -94.5% 0.00 ? 47% perf-sched.sch_delay.avg.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread
0.03 ? 5% -100.0% 0.00 perf-sched.sch_delay.max.ms.blk_mq_freeze_queue_wait.lo_release.__blkdev_put.blkdev_put
0.48 ? 43% +2728.2% 13.53 ? 71% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.05 ? 23% -100.0% 0.00 perf-sched.sch_delay.max.ms.devtmpfs_work_loop.devtmpfsd.kthread.ret_from_fork
0.06 ? 30% -40.7% 0.04 ? 47% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
0.40 ? 48% -56.5% 0.17 ? 25% perf-sched.sch_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
0.17 ? 43% -67.6% 0.05 ? 15% perf-sched.sch_delay.max.ms.io_schedule.put_and_wait_on_page_locked.filemap_get_pages.filemap_read
1.02 ? 54% -77.0% 0.23 ? 50% perf-sched.sch_delay.max.ms.kthread_worker_fn.kthread.ret_from_fork
0.12 ? 60% -63.6% 0.04 ? 26% perf-sched.sch_delay.max.ms.kthreadd.ret_from_fork
3.46 ?139% -100.0% 0.00 perf-sched.sch_delay.max.ms.percpu_down_write._cpu_down.cpu_down.device_offline
0.43 ? 4% -100.0% 0.00 perf-sched.sch_delay.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
13.41 ?140% -100.0% 0.00 perf-sched.sch_delay.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
0.77 ?131% -100.0% 0.00 perf-sched.sch_delay.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_start
351.09 ?134% -96.6% 11.92 ? 14% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.02 ? 20% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.pte_alloc_one.__pte_alloc
0.48 ? 77% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__flush_work.lru_add_drain_all.khugepaged
0.03 +4333.7% 1.32 ?118% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xfs_dir_lookup
0.04 ? 35% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xlog_alloc_log
0.13 ? 86% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.dput.simple_recursive_removal.debugfs_remove
0.02 ? 73% +91.1% 0.03 ? 21% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.__alloc_file.alloc_empty_file
0.02 ? 9% -73.1% 0.01 ?141% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.vm_area_dup.__split_vma
2.53 ?133% -98.6% 0.04 ? 20% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
0.04 ? 25% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.crypto_exit_scomp_ops_async.crypto_destroy_tfm
0.03 ? 28% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.device_del.device_unregister
0.03 ? 2% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.flush_workqueue.xfs_buftarg_drain
0.22 ? 45% +874.8% 2.19 ? 34% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.submit_bio_checks.submit_bio_noacct.submit_bio
0.02 ? 2% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.synchronize_rcu_expedited.namespace_unlock.path_umount
0.03 ? 7% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.vfree.free_vm_stack_cache.cpuhp_invoke_callback
4.56 ?138% -98.3% 0.08 ? 27% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
0.12 ? 40% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.xfs_rw_bdev
0.13 ? 55% -62.4% 0.05 ? 14% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages
0.04 ? 28% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.write_inode_now.iput.xfs_rtunmount_inodes
0.09 ? 45% -100.0% 0.00 perf-sched.sch_delay.max.ms.rescuer_thread.kthread.ret_from_fork
0.04 ? 21% +336.4% 0.19 ? 57% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_create
15.73 ? 34% -98.8% 0.19 ? 80% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
4.15 ?112% -97.0% 0.12 ? 17% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.04 -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
0.06 ? 23% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_direct_write
0.04 ? 3% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_read_iter
0.05 ? 3% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.sync_rcu_exp_select_cpus
0.78 ? 4% -33.4% 0.52 ? 37% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
0.03 ? 7% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
0.61 ? 39% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__wait_rcu_gp.synchronize_rcu
0.33 ?121% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.cpuhp_kick_ap.cpuhp_kick_ap_work
0.05 ? 8% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.devtmpfs_submit_req.devtmpfs_delete_node
0.02 ? 3% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.kthread_stop.destroy_workqueue
0.05 ? 20% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.kthread_stop.xfs_trans_ail_destroy
0.55 ?121% -92.2% 0.04 ? 23% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.xfs_buf_iowait.__xfs_buf_submit
0.05 ? 70% +113.4% 0.12 ? 6% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
27.32 ? 21% -65.1% 9.52 ? 91% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.17 ? 35% -56.5% 0.08 ? 34% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
23.62 ? 24% +45.6% 34.39 ? 12% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
0.09 ? 34% -100.0% 0.00 perf-sched.sch_delay.max.ms.xfs_ail_push_all_sync.xfs_log_quiesce.xfs_log_unmount.xfs_unmountfs
0.05 ? 51% -100.0% 0.00 perf-sched.sch_delay.max.ms.xfsaild.kthread.ret_from_fork
0.09 ? 99% -100.0% 0.00 perf-sched.sch_delay.max.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.__xfs_trans_commit
0.02 ? 11% -100.0% 0.00 perf-sched.sch_delay.max.ms.xlog_wait_on_iclog.xfs_log_unmount_write.xfs_log_unmount.xfs_unmountfs
0.06 ? 20% +323.2% 0.27 ? 51% perf-sched.sch_delay.max.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
36572 ? 2% +17.3% 42889 ? 3% perf-sched.total_wait_and_delay.count.ms
798.63 ? 5% +8.4% 865.36 ? 5% perf-sched.wait_and_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
148.76 ? 20% +252.3% 524.14 ? 15% perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
152.01 ? 21% +262.1% 550.43 ? 21% perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
40.46 ? 30% -29.3% 28.62 ? 18% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
5.82 ? 71% -84.4% 0.91 ? 41% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
78.40 ? 39% +339.2% 344.36 ? 55% perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
3.92 -33.2% 2.62 perf-sched.wait_and_delay.avg.ms.kthread_worker_fn.kthread.ret_from_fork
229.81 ? 68% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
95.84 ? 31% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
42.95 ? 4% +45.0% 62.29 ? 12% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
11.98 ? 3% +19.5% 14.31 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
60.36 ? 22% -71.3% 17.32 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
0.35 ? 52% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages
6.15 ? 12% +111.7% 13.01 ? 11% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
129.71 ? 14% +100.3% 259.79 ? 15% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
230.79 +44.1% 332.67 ? 9% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
240.28 -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
6.59 ? 38% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
4.36 ? 72% +160.4% 11.35 ? 4% perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
70.14 -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
0.08 ? 28% +2103.6% 1.84 ? 7% perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
60.94 ? 5% -25.2% 45.61 perf-sched.wait_and_delay.avg.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
293.56 ? 4% +83.5% 538.81 perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
134.37 ? 7% -26.7% 98.46 ? 2% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
102.90 ? 14% -85.2% 15.25 ?141% perf-sched.wait_and_delay.avg.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
4.26 ?141% +270.7% 15.78 ? 2% perf-sched.wait_and_delay.avg.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
95.02 ? 8% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
34.33 ? 3% -72.8% 9.33 ? 13% perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64
33.67 ? 3% -73.3% 9.00 ? 15% perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read
301.67 ? 7% -18.3% 246.33 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
997.33 ? 15% -34.6% 652.00 ? 26% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
57.33 ? 7% -89.5% 6.00 ? 23% perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
380.33 ? 14% +187.2% 1092 ? 4% perf-sched.wait_and_delay.count.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range
547.33 ? 6% +126.9% 1242 ? 3% perf-sched.wait_and_delay.count.kthread_worker_fn.kthread.ret_from_fork
13.67 ? 9% -65.9% 4.67 ? 53% perf-sched.wait_and_delay.count.kthreadd.ret_from_fork
13.00 ? 10% -100.0% 0.00 perf-sched.wait_and_delay.count.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
1.67 ? 28% -100.0% 0.00 perf-sched.wait_and_delay.count.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
3699 ? 5% -28.4% 2647 ? 11% perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
205.33 ? 10% +84.4% 378.67 perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
378.67 ? 11% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages
14154 ? 18% +47.8% 20918 ? 4% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
284.67 ? 5% -92.9% 20.33 ? 6% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
90.00 ? 3% -36.7% 57.00 ? 10% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
1.00 -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
440.67 ? 2% -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
137.33 ? 71% +184.7% 391.00 perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
2.00 -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
206.33 ? 10% +98.2% 409.00 perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
2237 ? 6% -47.9% 1165 perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
4070 ? 4% +59.4% 6488 ? 2% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
66.33 ?141% +319.6% 278.33 perf-sched.wait_and_delay.count.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
10.33 ? 9% -100.0% 0.00 perf-sched.wait_and_delay.count.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
3711 ? 3% -57.2% 1590 ? 26% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
2889 ? 92% -90.0% 289.85 ? 84% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
232.01 -64.4% 82.63 ? 34% perf-sched.wait_and_delay.max.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range
74.01 ? 25% -42.0% 42.95 ? 16% perf-sched.wait_and_delay.max.ms.kthread_worker_fn.kthread.ret_from_fork
1110 ? 55% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
138.50 ? 42% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
1681 ? 24% -89.9% 170.60 ? 18% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
29.22 ? 63% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages
5403 ? 25% -54.9% 2435 ? 2% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
1544 ? 24% -35.2% 1000 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
240.28 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
822.36 ?103% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
139.05 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
5.80 ? 77% +1110.3% 70.22 ? 8% perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
2017 -96.0% 81.57 ? 13% perf-sched.wait_and_delay.max.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
137.05 ? 8% -82.3% 24.28 ?141% perf-sched.wait_and_delay.max.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
12.48 ?141% +279.5% 47.38 ? 13% perf-sched.wait_and_delay.max.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
148.05 ? 6% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
798.58 ? 5% +8.4% 865.32 ? 5% perf-sched.wait_time.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
13.03 ? 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.blk_mq_freeze_queue_wait.lo_release.__blkdev_put.blkdev_put
148.70 ? 20% +251.4% 522.52 ? 16% perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
43.36 ? 5% -100.0% 0.00 perf-sched.wait_time.avg.ms.devtmpfs_work_loop.devtmpfsd.kthread.ret_from_fork
151.92 ? 21% +260.9% 548.23 ? 21% perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
69.92 ?105% -100.0% 0.00 perf-sched.wait_time.avg.ms.do_task_dead.do_exit.kthread.ret_from_fork
40.39 ? 30% -29.3% 28.57 ? 18% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
23.00 ? 89% -87.8% 2.80 ? 61% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
5.81 ? 71% -84.5% 0.90 ? 40% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
78.32 ? 39% +339.5% 344.26 ? 55% perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
10.20 ? 7% -56.6% 4.42 ? 69% perf-sched.wait_time.avg.ms.io_schedule.put_and_wait_on_page_locked.filemap_get_pages.filemap_read
2.95 ? 88% +607.1% 20.89 ? 10% perf-sched.wait_time.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
3.88 -33.3% 2.59 perf-sched.wait_time.avg.ms.kthread_worker_fn.kthread.ret_from_fork
6.66 ? 56% -100.0% 0.00 perf-sched.wait_time.avg.ms.percpu_down_write._cpu_down.cpu_down.device_offline
229.56 ? 68% -100.0% 0.00 perf-sched.wait_time.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
89.11 ? 37% -100.0% 0.00 perf-sched.wait_time.avg.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
42.81 ? 4% +45.4% 62.23 ? 12% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.36 ?129% -93.0% 0.03 ? 13% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__get_user_pages.__get_user_pages_remote.get_arg_page
5.45 ? 37% +113.1% 11.61 ? 36% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.kmem_alloc_large
0.10 ? 14% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xlog_alloc_log
0.01 ?141% +437.5% 0.06 ? 51% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.apparmor_file_alloc_security.security_file_alloc.__alloc_file
0.01 ?141% +11751.4% 1.38 ?126% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.copy_page_from_iter.pipe_write.new_sync_write
0.01 ?141% +988.2% 0.06 ? 79% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.copy_page_to_iter.filemap_read.new_sync_read
0.27 ? 62% -89.9% 0.03 ? 26% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.copy_strings.isra.0
21.74 ? 85% -97.3% 0.59 ? 36% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.02 ? 71% +2686.0% 0.53 ?133% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_read.path_openat.do_filp_open
17.05 ? 72% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_write.simple_recursive_removal.debugfs_remove
0.02 ? 71% +146.9% 0.05 ? 47% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_write_killable.map_vdso.load_elf_binary
0.03 ? 15% -31.0% 0.02 ? 29% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.nd_jump_root.pick_link
63.11 ?141% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.path_put.vfs_statx
10.85 ? 25% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.simple_recursive_removal.debugfs_remove
0.03 ? 10% -17.6% 0.02 ? 14% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.exit_mmap.mmput.begin_new_exec
0.86 ?137% -99.0% 0.01 ?141% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.getname_flags.user_path_at_empty
6.61 ?133% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds
17.45 ? 42% +109.1% 36.50 ? 39% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_buf_item_init._xfs_trans_bjoin
7.07 ? 20% +95.7% 13.84 ? 35% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_trans_alloc.xfs_trans_alloc_icreate
0.52 ? 64% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.crypto_exit_scomp_ops_async.crypto_destroy_tfm
0.04 ? 20% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.device_del.device_unregister
230.88 ? 65% -76.6% 53.99 ?141% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.drm_gem_shmem_vunmap.drm_gem_vunmap
0.09 ? 4% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.flush_workqueue.xfs_buftarg_drain
0.36 ? 28% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_exit_cpu_context.perf_event_exit_cpu
0.90 ? 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.slub_cpu_dead.cpuhp_invoke_callback
9.12 ? 12% +72.4% 15.72 ? 11% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
0.04 ? 49% -75.0% 0.01 ?141% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.remove_vma.__do_munmap.mmap_region
0.06 ? 69% +88.5% 0.12 ? 42% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
0.62 ? 56% +131.7% 1.43 ? 45% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
8.01 ? 6% -48.8% 4.10 ? 3% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.submit_bio_checks.submit_bio_noacct.submit_bio
1.04 ? 10% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.vfree.free_vm_stack_cache.cpuhp_invoke_callback
1.26 ? 87% -98.7% 0.02 ?141% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.vfs_write.ksys_write.do_syscall_64
11.97 ? 3% +19.5% 14.30 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
0.03 ? 3% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.stop_machine_cpuslocked.takedown_cpu
1.90 ?141% +2221.3% 44.14 ? 36% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xfs_btree_split.xfs_btree_make_block_unfull
60.31 ? 22% -71.3% 17.28 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
0.18 ? 16% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.xfs_rw_bdev
0.42 -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.write_inode_now.iput.xfs_rtunmount_inodes
0.07 ? 19% -100.0% 0.00 perf-sched.wait_time.avg.ms.rescuer_thread.kthread.ret_from_fork
21.26 ? 15% +30.6% 27.76 ? 8% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_create
0.10 ? 48% -86.2% 0.01 ?141% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.xfs_map_blocks.iomap_writepage_map.write_cache_pages
6.08 ? 13% +112.8% 12.94 ? 11% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
129.60 ? 14% +100.4% 259.73 ? 15% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
230.69 +44.2% 332.65 ? 9% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
240.24 -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
5.01 ? 80% +648.5% 37.53 ? 6% perf-sched.wait_time.avg.ms.schedule_timeout.__down.down.xfs_buf_lock
0.46 -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_direct_write
6.74 ? 9% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_read_iter
6.56 ? 38% -81.2% 1.24 ? 26% perf-sched.wait_time.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
0.04 ? 15% -45.2% 0.02 ? 14% perf-sched.wait_time.avg.ms.schedule_timeout.khugepaged.kthread.ret_from_fork
6.46 ? 9% +74.6% 11.28 ? 4% perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
70.12 -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
148.48 ? 98% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__wait_rcu_gp.synchronize_rcu
45.56 ? 10% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.cpuhp_kick_ap.cpuhp_kick_ap_work
8.06 ? 5% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.devtmpfs_submit_req.devtmpfs_delete_node
17.37 ? 5% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.kthread_park.takedown_cpu
0.03 ? 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.kthread_stop.destroy_workqueue
35.73 ? 11% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.kthread_stop.xfs_trans_ail_destroy
1.69 ? 6% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.stop_machine_cpuslocked.takedown_cpu
2.04 -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.takedown_cpu.cpuhp_invoke_callback
5.50 ? 3% +149.9% 13.75 ? 32% perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.xfs_buf_iowait.__xfs_buf_submit
0.08 ? 28% +2098.8% 1.84 ? 7% perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
9.28 ? 17% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion_killable.__kthread_create_on_node.kthread_create_on_node
60.87 ? 5% -25.2% 45.56 perf-sched.wait_time.avg.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
293.47 ? 4% +83.6% 538.76 perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.14 ? 3% -100.0% 0.00 perf-sched.wait_time.avg.ms.synchronize_rcu_expedited.namespace_unlock.path_umount.ksys_umount
134.29 ? 7% -26.8% 98.37 ? 2% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
24.70 ? 9% -100.0% 0.00 perf-sched.wait_time.avg.ms.xfs_ail_push_all_sync.xfs_log_quiesce.xfs_log_unmount.xfs_unmountfs
20.17 ? 72% -100.0% 0.00 perf-sched.wait_time.avg.ms.xfsaild.kthread.ret_from_fork
102.86 ? 14% -59.0% 42.19 ? 9% perf-sched.wait_time.avg.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
14.35 ? 22% +69.9% 24.39 ? 11% perf-sched.wait_time.avg.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
13.80 ? 5% -41.7% 8.05 perf-sched.wait_time.avg.ms.xlog_state_get_iclog_space.xlog_write.xlog_cil_push_work.process_one_work
11.47 ? 32% -100.0% 0.00 perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.__xfs_trans_commit
12.51 +25.8% 15.74 ? 2% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
0.18 ? 18% -100.0% 0.00 perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xfs_log_unmount_write.xfs_log_unmount.xfs_unmountfs
94.98 ? 8% -56.4% 41.38 ? 4% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
23.03 ? 5% -96.4% 0.83 ? 21% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread
45.72 ? 8% -100.0% 0.00 perf-sched.wait_time.max.ms.blk_mq_freeze_queue_wait.lo_release.__blkdev_put.blkdev_put
179.00 ? 27% -100.0% 0.00 perf-sched.wait_time.max.ms.devtmpfs_work_loop.devtmpfsd.kthread.ret_from_fork
3710 ? 3% -57.1% 1590 ? 26% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
486.28 ?106% -100.0% 0.00 perf-sched.wait_time.max.ms.do_task_dead.do_exit.kthread.ret_from_fork
396.63 ?108% -95.7% 16.96 ? 65% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
2889 ? 92% -90.0% 289.85 ? 84% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
372.65 -97.4% 9.69 ? 39% perf-sched.wait_time.max.ms.io_schedule.put_and_wait_on_page_locked.filemap_get_pages.filemap_read
231.98 -64.4% 82.59 ? 34% perf-sched.wait_time.max.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range
73.96 ? 25% -42.0% 42.89 ? 15% perf-sched.wait_time.max.ms.kthread_worker_fn.kthread.ret_from_fork
57.20 ? 51% -100.0% 0.00 perf-sched.wait_time.max.ms.percpu_down_write._cpu_down.cpu_down.device_offline
1110 ? 55% -100.0% 0.00 perf-sched.wait_time.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.rebuild_sched_domains
125.09 ? 48% -100.0% 0.00 perf-sched.wait_time.max.ms.percpu_rwsem_wait.__percpu_down_read.cpus_read_lock.vmstat_shepherd
191.52 ? 77% +662.1% 1459 ? 91% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.shmem_alloc_page
0.04 ? 13% +166.4% 0.10 ? 54% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.wp_page_copy
1.71 ?137% -98.1% 0.03 ? 3% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__get_user_pages.__get_user_pages_remote.get_arg_page
18.74 ? 42% +370.6% 88.21 ? 52% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.kmem_alloc_large
16.66 ? 53% +181.4% 46.90 ? 42% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xfs_dir_lookup
0.10 ? 14% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xlog_alloc_log
0.01 ?141% +437.5% 0.06 ? 51% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.apparmor_file_alloc_security.security_file_alloc.__alloc_file
0.01 ?141% +19957.1% 2.34 ?114% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.copy_page_from_iter.pipe_write.new_sync_write
3.28 ? 75% -97.5% 0.08 ? 77% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.copy_strings.isra.0
533.33 ? 66% -98.3% 8.86 ? 45% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.03 ? 12% +498.9% 0.19 ?103% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.__vma_adjust.shift_arg_pages
19.17 ? 70% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.simple_recursive_removal.debugfs_remove
0.07 ? 12% -54.1% 0.03 ? 5% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.unlink_anon_vmas.free_pgtables
0.02 ? 71% +503.1% 0.13 ?102% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write_killable.map_vdso.load_elf_binary
13.50 ? 5% +2558.0% 358.82 ?126% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run
126.20 ?141% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.path_put.vfs_statx
28.66 ? 15% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.simple_recursive_removal.debugfs_remove
1.70 ?139% -99.5% 0.01 ?141% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.getname_flags.user_path_at_empty
6.61 ?133% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds
2.64 ?137% +137.1% 6.27 ? 72% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.security_file_alloc.__alloc_file
21.30 ? 12% +544.1% 137.19 ? 41% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_trans_alloc.xfs_trans_alloc_icreate
0.03 ? 5% -67.4% 0.01 ?141% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mmput.begin_new_exec.load_elf_binary
1.14 ? 73% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.crypto_exit_scomp_ops_async.crypto_destroy_tfm
0.04 ? 25% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.device_del.device_unregister
362.97 ? 56% -85.1% 53.99 ?141% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.drm_gem_shmem_vunmap.drm_gem_vunmap
0.09 ? 4% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.flush_workqueue.xfs_buftarg_drain
0.55 ? 47% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_exit_cpu_context.perf_event_exit_cpu
1.81 ? 8% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.slub_cpu_dead.cpuhp_invoke_callback
382.39 ? 20% +114.1% 818.63 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
0.04 ? 36% -77.4% 0.01 ?141% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.remove_vma.__do_munmap.mmap_region
1.93 ? 9% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.vfree.free_vm_stack_cache.cpuhp_invoke_callback
1.27 ? 86% -98.7% 0.02 ?141% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.vfs_write.ksys_write.do_syscall_64
0.05 ? 13% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.stop_machine_cpuslocked.takedown_cpu
1.90 ?141% +3242.9% 63.56 ? 37% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xfs_btree_split.xfs_btree_make_block_unfull
1681 ? 24% -89.9% 170.56 ? 18% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
0.56 ? 26% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.xfs_rw_bdev
0.42 -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.write_inode_now.iput.xfs_rtunmount_inodes
0.12 ? 39% -100.0% 0.00 perf-sched.wait_time.max.ms.rescuer_thread.kthread.ret_from_fork
5.23 ?137% +408.0% 26.59 ? 79% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_trans_roll
0.48 ? 63% -93.6% 0.03 ?141% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.xfs_map_blocks.iomap_writepage_map.write_cache_pages
135.13 ? 44% +50.3% 203.13 ? 17% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
5403 ? 25% -54.9% 2435 ? 2% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
1541 ? 24% -35.1% 1000 perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
240.24 -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.blkdev_get_by_dev
60.21 ? 56% +157.4% 154.98 ? 39% perf-sched.wait_time.max.ms.schedule_timeout.__down.down.xfs_buf_lock
1.51 ? 3% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_direct_write
19.99 ? 9% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.io_schedule_timeout.__blkdev_direct_IO_simple.generic_file_read_iter
822.32 ?103% -97.5% 20.19 ? 22% perf-sched.wait_time.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
0.04 ? 15% -45.2% 0.02 ? 14% perf-sched.wait_time.max.ms.schedule_timeout.khugepaged.kthread.ret_from_fork
169.84 ? 12% -48.0% 88.33 ? 9% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
139.02 -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__synchronize_srcu.part.0
1305 ? 72% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__wait_rcu_gp.synchronize_rcu
105.74 ? 23% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.cpuhp_kick_ap.cpuhp_kick_ap_work
30.33 ? 3% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.devtmpfs_submit_req.devtmpfs_delete_node
31.52 ? 3% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.kthread_park.takedown_cpu
0.05 ? 29% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.kthread_stop.destroy_workqueue
35.73 ? 11% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.kthread_stop.xfs_trans_ail_destroy
2.03 -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.stop_machine_cpuslocked.takedown_cpu
2.04 -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.takedown_cpu.cpuhp_invoke_callback
142.32 ? 2% -71.3% 40.86 ? 54% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.xfs_buf_iowait.__xfs_buf_submit
5.75 ? 77% +1121.0% 70.17 ? 8% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
45.42 ? 2% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion_killable.__kthread_create_on_node.kthread_create_on_node
2017 -96.0% 81.50 ? 13% perf-sched.wait_time.max.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
0.14 ? 3% -100.0% 0.00 perf-sched.wait_time.max.ms.synchronize_rcu_expedited.namespace_unlock.path_umount.ksys_umount
26.83 ? 10% -100.0% 0.00 perf-sched.wait_time.max.ms.xfs_ail_push_all_sync.xfs_log_quiesce.xfs_log_unmount.xfs_unmountfs
34.55 ? 70% -100.0% 0.00 perf-sched.wait_time.max.ms.xfsaild.kthread.ret_from_fork
137.01 ? 8% -33.2% 91.58 ? 22% perf-sched.wait_time.max.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
56.24 ? 36% +130.5% 129.66 ? 27% perf-sched.wait_time.max.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
21.56 ? 35% -100.0% 0.00 perf-sched.wait_time.max.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.__xfs_trans_commit
36.56 ? 8% +29.5% 47.34 ? 13% perf-sched.wait_time.max.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
0.18 ? 18% -100.0% 0.00 perf-sched.wait_time.max.ms.xlog_wait_on_iclog.xfs_log_unmount_write.xfs_log_unmount.xfs_unmountfs
148.02 ? 6% -38.1% 91.67 ? 3% perf-sched.wait_time.max.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
68.26 ? 14% -63.8% 24.72 perf-sched.wait_time.max.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread



***************************************************************************************************
lkp-knm02: 272 threads 2 sockets Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory
=========================================================================================
compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/bufferedio/1HDD/xfs/x86_64-rhel-8.3/hdd/debian-10.4-x86_64-20200603.cgz/lkp-knm02/MWUL/fxmark/0x11

commit:
3468bb1ca6 ("xfs: remove need_start_rec parameter from xlog_write()")
eef983ffea ("xfs: journal IO cache flush reductions")

3468bb1ca6e88407 eef983ffeae7a1cdde8c3338155
---------------- ---------------------------
%stddev %change %stddev
\ | \
85.12 -14.8% 72.56 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.idle_sec
0.20 ? 4% -52.5% 0.09 ? 5% fxmark.hdd_xfs_MWUL_18_bufferedio.iowait_sec
0.19 ? 4% -45.6% 0.10 ? 7% fxmark.hdd_xfs_MWUL_18_bufferedio.iowait_util
2.89 -10.2% 2.59 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.irq_sec
5.98 -12.4% 5.23 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.real_sec
5.66 -13.0% 4.93 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.secs
0.46 ? 2% -24.1% 0.35 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.softirq_sec
0.43 -13.1% 0.37 fxmark.hdd_xfs_MWUL_18_bufferedio.softirq_util
14.83 +13.0% 16.75 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.sys_util
9185 +15.0% 10566 ? 2% fxmark.hdd_xfs_MWUL_18_bufferedio.works/sec
0.19 ? 10% -15.5% 0.16 ? 7% fxmark.hdd_xfs_MWUL_1_bufferedio.softirq_sec
1.63 ? 10% -14.4% 1.39 ? 7% fxmark.hdd_xfs_MWUL_1_bufferedio.softirq_util
0.65 ? 36% -56.9% 0.28 ? 26% fxmark.hdd_xfs_MWUL_2_bufferedio.idle_sec
4.81 ? 35% -54.5% 2.19 ? 26% fxmark.hdd_xfs_MWUL_2_bufferedio.idle_util
0.17 -9.8% 0.15 ? 3% fxmark.hdd_xfs_MWUL_2_bufferedio.softirq_sec
0.40 ? 15% -38.7% 0.24 fxmark.hdd_xfs_MWUL_36_bufferedio.iowait_sec
0.11 ? 9% -45.0% 0.06 ? 2% fxmark.hdd_xfs_MWUL_36_bufferedio.iowait_util
5.29 ? 19% -64.7% 1.87 ? 22% fxmark.hdd_xfs_MWUL_4_bufferedio.idle_sec
25.40 ? 14% -55.1% 11.39 ? 19% fxmark.hdd_xfs_MWUL_4_bufferedio.idle_util
0.07 ? 44% -100.0% 0.00 fxmark.hdd_xfs_MWUL_4_bufferedio.iowait_sec
0.35 ? 42% -100.0% 0.00 fxmark.hdd_xfs_MWUL_4_bufferedio.iowait_util
0.96 ? 2% -16.3% 0.81 ? 2% fxmark.hdd_xfs_MWUL_4_bufferedio.irq_sec
5.28 ? 5% -22.0% 4.12 ? 3% fxmark.hdd_xfs_MWUL_4_bufferedio.real_sec
4.96 ? 5% -23.7% 3.78 ? 2% fxmark.hdd_xfs_MWUL_4_bufferedio.secs
0.20 ? 14% -48.3% 0.10 ? 9% fxmark.hdd_xfs_MWUL_4_bufferedio.softirq_sec
0.96 ? 10% -34.1% 0.63 ? 8% fxmark.hdd_xfs_MWUL_4_bufferedio.softirq_util
64.31 ? 5% +21.5% 78.13 ? 2% fxmark.hdd_xfs_MWUL_4_bufferedio.sys_util
4.31 ? 10% +13.5% 4.89 ? 3% fxmark.hdd_xfs_MWUL_4_bufferedio.user_util
10394 ? 6% +30.5% 13568 ? 2% fxmark.hdd_xfs_MWUL_4_bufferedio.works/sec
848.97 ? 7% -16.1% 712.05 ? 6% fxmark.hdd_xfs_MWUL_54_bufferedio.idle_sec
0.75 ? 6% -22.3% 0.58 ? 6% fxmark.hdd_xfs_MWUL_54_bufferedio.iowait_sec
21.07 ? 5% -18.1% 17.25 ? 7% fxmark.hdd_xfs_MWUL_54_bufferedio.irq_sec
16.91 ? 6% -16.3% 14.16 ? 6% fxmark.hdd_xfs_MWUL_54_bufferedio.real_sec
16.85 ? 5% -17.0% 13.98 ? 7% fxmark.hdd_xfs_MWUL_54_bufferedio.secs
2.59 ? 5% -22.0% 2.02 ? 7% fxmark.hdd_xfs_MWUL_54_bufferedio.softirq_sec
7.75 -58.0% 3.26 ? 2% fxmark.hdd_xfs_MWUL_54_bufferedio.user_sec
0.86 ? 8% -49.9% 0.43 ? 5% fxmark.hdd_xfs_MWUL_54_bufferedio.user_util
3185 ? 6% +20.3% 3832 ? 7% fxmark.hdd_xfs_MWUL_54_bufferedio.works/sec
1309 ? 3% -14.0% 1126 ? 2% fxmark.hdd_xfs_MWUL_72_bufferedio.idle_sec
0.09 +12.8% 0.10 ? 3% fxmark.hdd_xfs_MWUL_72_bufferedio.iowait_util
30.04 ? 2% -11.9% 26.47 ? 2% fxmark.hdd_xfs_MWUL_72_bufferedio.irq_sec
19.10 ? 3% -13.6% 16.49 ? 2% fxmark.hdd_xfs_MWUL_72_bufferedio.real_sec
19.61 ? 4% -15.2% 16.64 ? 2% fxmark.hdd_xfs_MWUL_72_bufferedio.secs
3.10 ? 3% -9.7% 2.80 fxmark.hdd_xfs_MWUL_72_bufferedio.softirq_sec
1.60 ? 4% +16.2% 1.86 fxmark.hdd_xfs_MWUL_72_bufferedio.sys_util
0.33 ? 2% +13.9% 0.37 fxmark.hdd_xfs_MWUL_72_bufferedio.user_util
2774 ? 4% +17.4% 3257 ? 2% fxmark.hdd_xfs_MWUL_72_bufferedio.works/sec
273.82 -5.5% 258.66 fxmark.time.elapsed_time
273.82 -5.5% 258.66 fxmark.time.elapsed_time.max
22.00 +4.5% 23.00 fxmark.time.percent_of_cpu_this_job_got
47980 ? 2% -22.5% 37196 ? 2% fxmark.time.voluntary_context_switches
0.11 ? 3% -0.0 0.09 ? 2% mpstat.cpu.all.iowait%
9432 ? 9% +28.9% 12160 ? 22% numa-meminfo.node1.KReclaimable
9432 ? 9% +28.9% 12160 ? 22% numa-meminfo.node1.SReclaimable
56.60 -3.1% 54.84 iostat.cpu.idle
20.21 +3.7% 20.96 iostat.cpu.system
2.76 -5.3% 2.61 iostat.cpu.user
138197 +32.4% 182972 numa-vmstat.node0.nr_dirtied
138039 +32.4% 182774 numa-vmstat.node0.nr_written
2358 ? 9% +28.9% 3040 ? 22% numa-vmstat.node1.nr_slab_reclaimable
7134 +39.6% 9960 vmstat.io.bo
10348 +3.4% 10700 vmstat.system.cs
131002 +3.7% 135808 vmstat.system.in
1471 ? 2% +12.4% 1653 slabinfo.PING.num_objs
6396 +16.0% 7418 slabinfo.proc_inode_cache.active_objs
6863 +17.8% 8084 slabinfo.proc_inode_cache.num_objs
715.67 ? 5% -19.4% 576.67 ? 8% slabinfo.skbuff_fclone_cache.active_objs
923.67 ? 3% -16.0% 776.00 ? 5% slabinfo.skbuff_fclone_cache.num_objs
9068 +2.7% 9312 perf-stat.i.context-switches
58455 +2.9% 60128 perf-stat.i.cpu-clock
110.25 +1.5% 111.90 perf-stat.i.cpu-migrations
2.32 ? 4% +4.5% 2.42 perf-stat.i.major-faults
2771 +3.3% 2862 perf-stat.i.minor-faults
2774 +3.3% 2864 perf-stat.i.page-faults
58455 +2.9% 60128 perf-stat.i.task-clock
9024 +2.5% 9249 perf-stat.ps.context-switches
58710 +2.6% 60231 perf-stat.ps.cpu-clock
110.31 +1.5% 111.98 perf-stat.ps.cpu-migrations
2757 +3.1% 2843 perf-stat.ps.minor-faults
2759 +3.1% 2845 perf-stat.ps.page-faults
58709 +2.6% 60231 perf-stat.ps.task-clock
70145 +0.9% 70790 proc-vmstat.nr_anon_pages
231942 +30.2% 301999 proc-vmstat.nr_dirtied
8977 +3.1% 9253 proc-vmstat.nr_inactive_file
61051 +1.3% 61816 proc-vmstat.nr_slab_unreclaimable
231942 +30.2% 301999 proc-vmstat.nr_written
8977 +3.1% 9253 proc-vmstat.nr_zone_inactive_file
187.33 ? 12% +22.1% 228.67 ? 11% proc-vmstat.nr_zone_write_pending
1372660 -1.9% 1346336 proc-vmstat.numa_hit
1372618 -1.9% 1346286 proc-vmstat.numa_local
24895 ? 7% -59.3% 10129 ?104% proc-vmstat.pgactivate
1631337 -1.4% 1608916 proc-vmstat.pgalloc_normal
930890 -2.1% 911419 proc-vmstat.pgfault
1575367 -1.7% 1548619 proc-vmstat.pgfree
1850163 +30.6% 2417156 proc-vmstat.pgpgout
93731 -2.5% 91424 proc-vmstat.pgreuse
28009 ? 13% -15.9% 23545 softirqs.CPU1.RCU
28414 ? 3% -11.3% 25216 softirqs.CPU1.SCHED
20483 -15.4% 17328 ? 7% softirqs.CPU11.SCHED
25094 -11.3% 22262 softirqs.CPU2.SCHED
23633 ? 2% -10.6% 21131 ? 4% softirqs.CPU3.SCHED
16271 -13.9% 14017 ? 3% softirqs.CPU34.SCHED
14823 ? 2% -10.7% 13235 ? 2% softirqs.CPU36.SCHED
15230 ? 2% -11.1% 13533 ? 4% softirqs.CPU37.SCHED
15258 ? 3% -13.7% 13163 ? 3% softirqs.CPU41.SCHED
14601 -9.6% 13193 softirqs.CPU42.SCHED
11341 ? 2% -9.6% 10251 ? 3% softirqs.CPU48.RCU
21521 -9.3% 19514 ? 2% softirqs.CPU5.SCHED
14550 -11.1% 12936 ? 2% softirqs.CPU50.SCHED
11269 ? 4% -9.8% 10163 ? 3% softirqs.CPU51.RCU
12578 ? 11% -17.3% 10407 ? 4% softirqs.CPU53.RCU
14481 ? 2% -9.0% 13182 ? 2% softirqs.CPU53.SCHED
10095 -11.6% 8922 ? 7% softirqs.CPU55.SCHED
7828 ? 2% +38.5% 10841 ? 19% softirqs.CPU56.RCU
37535 -25.6% 27919 ? 2% interrupts.171:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
968498 +1.0% 978318 interrupts.CAL:Function_call_interrupts
3150 -48.5% 1621 interrupts.CPU0.171:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
660.33 ? 12% -34.9% 430.00 ? 37% interrupts.CPU0.36:IR-PCI-MSI.2621442-edge.eth1-TxRx-1
10125 ? 6% -7.8% 9335 ? 7% interrupts.CPU0.CAL:Function_call_interrupts
270.33 ? 39% +230.0% 892.00 ? 21% interrupts.CPU0.NMI:Non-maskable_interrupts
270.33 ? 39% +230.0% 892.00 ? 21% interrupts.CPU0.PMI:Performance_monitoring_interrupts
237.67 ? 6% +123.8% 532.00 ? 32% interrupts.CPU1.NMI:Non-maskable_interrupts
237.67 ? 6% +123.8% 532.00 ? 32% interrupts.CPU1.PMI:Performance_monitoring_interrupts
3607 ? 23% -44.2% 2012 ? 33% interrupts.CPU1.RES:Rescheduling_interrupts
4360 ? 4% +31.3% 5725 ? 5% interrupts.CPU11.CAL:Function_call_interrupts
4817 ? 9% +24.2% 5984 ? 2% interrupts.CPU12.CAL:Function_call_interrupts
233.67 ? 36% +68.6% 394.00 ? 41% interrupts.CPU12.NMI:Non-maskable_interrupts
233.67 ? 36% +68.6% 394.00 ? 41% interrupts.CPU12.PMI:Performance_monitoring_interrupts
4950 ? 11% +15.4% 5713 ? 2% interrupts.CPU13.CAL:Function_call_interrupts
1298 ? 75% -74.9% 325.67 ? 22% interrupts.CPU17.37:IR-PCI-MSI.2621443-edge.eth1-TxRx-2
5720 ? 9% -12.9% 4981 ? 4% interrupts.CPU17.CAL:Function_call_interrupts
264.67 ? 24% +67.1% 442.33 ? 38% interrupts.CPU19.NMI:Non-maskable_interrupts
264.67 ? 24% +67.1% 442.33 ? 38% interrupts.CPU19.PMI:Performance_monitoring_interrupts
8948 ? 8% +21.3% 10852 ? 8% interrupts.CPU2.CAL:Function_call_interrupts
268.67 ? 35% +41.8% 381.00 ? 20% interrupts.CPU2.NMI:Non-maskable_interrupts
268.67 ? 35% +41.8% 381.00 ? 20% interrupts.CPU2.PMI:Performance_monitoring_interrupts
726.33 ? 14% -45.3% 397.33 ? 8% interrupts.CPU2.RES:Rescheduling_interrupts
440.67 ? 29% -35.9% 282.67 ? 24% interrupts.CPU25.NMI:Non-maskable_interrupts
440.67 ? 29% -35.9% 282.67 ? 24% interrupts.CPU25.PMI:Performance_monitoring_interrupts
3885 ? 3% +16.6% 4529 ? 9% interrupts.CPU27.CAL:Function_call_interrupts
236.00 ? 29% +54.5% 364.67 ? 14% interrupts.CPU28.NMI:Non-maskable_interrupts
236.00 ? 29% +54.5% 364.67 ? 14% interrupts.CPU28.PMI:Performance_monitoring_interrupts
859.67 ? 21% -37.5% 537.00 ? 10% interrupts.CPU3.RES:Rescheduling_interrupts
3875 +12.8% 4371 ? 5% interrupts.CPU32.CAL:Function_call_interrupts
230.00 ? 33% +33.8% 307.67 ? 23% interrupts.CPU32.NMI:Non-maskable_interrupts
230.00 ? 33% +33.8% 307.67 ? 23% interrupts.CPU32.PMI:Performance_monitoring_interrupts
4005 ? 2% +12.5% 4506 ? 9% interrupts.CPU33.CAL:Function_call_interrupts
313.00 ? 23% -30.9% 216.33 ? 26% interrupts.CPU34.NMI:Non-maskable_interrupts
313.00 ? 23% -30.9% 216.33 ? 26% interrupts.CPU34.PMI:Performance_monitoring_interrupts
184.67 ? 9% -39.2% 112.33 ? 9% interrupts.CPU36.NMI:Non-maskable_interrupts
184.67 ? 9% -39.2% 112.33 ? 9% interrupts.CPU36.PMI:Performance_monitoring_interrupts
239.67 ? 32% -53.5% 111.33 ? 36% interrupts.CPU37.NMI:Non-maskable_interrupts
239.67 ? 32% -53.5% 111.33 ? 36% interrupts.CPU37.PMI:Performance_monitoring_interrupts
398.00 ? 14% -60.0% 159.33 ? 50% interrupts.CPU39.NMI:Non-maskable_interrupts
398.00 ? 14% -60.0% 159.33 ? 50% interrupts.CPU39.PMI:Performance_monitoring_interrupts
4435 ? 10% +54.2% 6840 ? 20% interrupts.CPU4.CAL:Function_call_interrupts
262.00 ? 23% +100.5% 525.33 ? 33% interrupts.CPU4.NMI:Non-maskable_interrupts
262.00 ? 23% +100.5% 525.33 ? 33% interrupts.CPU4.PMI:Performance_monitoring_interrupts
87.00 ? 14% +78.2% 155.00 ? 24% interrupts.CPU4.RES:Rescheduling_interrupts
239.00 ? 33% -53.1% 112.00 ? 34% interrupts.CPU40.NMI:Non-maskable_interrupts
239.00 ? 33% -53.1% 112.00 ? 34% interrupts.CPU40.PMI:Performance_monitoring_interrupts
3861 ? 9% +23.5% 4768 ? 12% interrupts.CPU43.CAL:Function_call_interrupts
180.33 ? 5% -26.6% 132.33 ? 5% interrupts.CPU43.NMI:Non-maskable_interrupts
180.33 ? 5% -26.6% 132.33 ? 5% interrupts.CPU43.PMI:Performance_monitoring_interrupts
342.00 ? 21% -53.2% 160.00 ? 19% interrupts.CPU46.NMI:Non-maskable_interrupts
342.00 ? 21% -53.2% 160.00 ? 19% interrupts.CPU46.PMI:Performance_monitoring_interrupts
278.00 ? 35% -54.3% 127.00 ? 37% interrupts.CPU48.NMI:Non-maskable_interrupts
278.00 ? 35% -54.3% 127.00 ? 37% interrupts.CPU48.PMI:Performance_monitoring_interrupts
91.33 ? 28% +33.6% 122.00 ? 22% interrupts.CPU5.RES:Rescheduling_interrupts
5156 ? 7% +25.9% 6491 ? 14% interrupts.CPU6.CAL:Function_call_interrupts
248.00 ? 28% +31.3% 325.67 ? 22% interrupts.CPU6.NMI:Non-maskable_interrupts
248.00 ? 28% +31.3% 325.67 ? 22% interrupts.CPU6.PMI:Performance_monitoring_interrupts
3327 +10.6% 3681 ? 4% interrupts.CPU63.CAL:Function_call_interrupts
4531 ? 8% -18.0% 3716 ? 14% interrupts.CPU68.CAL:Function_call_interrupts
4740 ? 4% +21.7% 5769 ? 12% interrupts.CPU7.CAL:Function_call_interrupts
11894 ? 7% -18.3% 9712 ? 7% interrupts.RES:Rescheduling_interrupts
0.04 ? 6% +25.0% 0.05 ? 17% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.01 ? 44% -85.3% 0.00 ?141% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.04 ? 4% -20.8% 0.03 ? 9% perf-sched.sch_delay.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.03 ? 3% +13.0% 0.03 perf-sched.sch_delay.avg.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range
0.04 -16.0% 0.03 ? 7% perf-sched.sch_delay.avg.ms.kthread_worker_fn.kthread.ret_from_fork
0.11 ? 95% +399.4% 0.57 ? 23% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.kmem_alloc_large
0.00 ?141% +258.3% 0.01 ? 40% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.down_read.walk_component.link_path_walk
0.02 ? 7% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.down_write.__vma_adjust.__split_vma
0.14 ? 46% -56.2% 0.06 ? 41% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.down_write.xfs_ilock.xfs_lock_two_inodes
0.10 ? 98% +248.0% 0.35 ? 19% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.dput.do_unlinkat.do_syscall_64
0.02 ? 2% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.vm_area_dup.__split_vma
0.20 ? 54% +96.0% 0.39 ? 7% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_inobt_init_common.xfs_inobt_init_cursor
0.03 ? 16% -43.5% 0.02 ? 40% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
0.03 -77.1% 0.01 ? 12% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.submit_bio_checks.submit_bio_noacct.submit_bio
0.01 ? 37% -70.3% 0.00 ? 89% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.zap_pte_range.unmap_page_range.unmap_vmas
0.04 ? 11% -35.1% 0.02 ? 8% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
0.02 ? 19% +59.3% 0.03 ? 31% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.04 ? 26% -36.3% 0.03 ? 4% perf-sched.sch_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom
0.03 +56.6% 0.04 ? 43% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.05 +16.0% 0.06 ? 6% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
0.05 ? 28% +96.0% 0.10 ? 33% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.07 ? 3% +36.5% 0.10 ? 2% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.04 ? 4% -36.4% 0.03 ? 15% perf-sched.sch_delay.avg.ms.xlog_state_get_iclog_space.xlog_write.xlog_cil_push_work.process_one_work
0.04 ? 4% -94.3% 0.00 ? 20% perf-sched.sch_delay.avg.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread
0.05 ? 78% +176.6% 0.13 ? 33% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
3.07 ?130% -95.9% 0.13 ? 74% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.43 ? 37% +78350.5% 335.51 ?140% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.54 ? 97% +448.3% 2.98 ? 24% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.kmem_alloc_large
3.75 ? 71% -61.0% 1.46 ?141% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.kmem_alloc.xfs_dir_removename
0.02 ? 5% +80.4% 0.03 ? 41% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.01 ?141% +305.3% 0.03 ? 34% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.down_read.walk_component.link_path_walk
0.02 ? 23% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.down_write.__vma_adjust.__split_vma
0.02 ? 2% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.vm_area_dup.__split_vma
0.28 ?108% +256.7% 0.99 ? 79% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_buf_item_init._xfs_trans_bjoin
0.05 ? 31% -51.3% 0.02 ? 16% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
0.08 ? 18% -61.6% 0.03 ? 87% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
0.03 ? 5% +9.0% 0.03 ? 5% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.blkdev_issue_flush
0.04 ? 21% +31.5% 0.05 ? 19% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages
0.04 ? 6% +617.0% 0.32 ?111% perf-sched.sch_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
0.10 ? 39% +126.3% 0.22 ? 44% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree
0.04 ? 17% +1158.8% 0.50 ?108% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_remove
0.08 ? 25% +409.4% 0.40 ?107% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.04 ? 26% -36.3% 0.03 ? 4% perf-sched.sch_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom
0.09 ? 12% +13536.6% 11.68 ?123% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.04 ? 42% -58.7% 0.02 ? 70% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
0.07 ? 34% -49.3% 0.04 ? 41% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
0.19 ? 6% +54.2% 0.29 ? 22% perf-sched.sch_delay.max.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
45.44 ? 18% -54.4% 20.70 ? 52% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
0.14 ? 14% -25.6% 0.11 ? 18% perf-sched.sch_delay.max.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread
0.03 ? 2% +20.2% 0.04 ? 11% perf-sched.total_sch_delay.average.ms
273.48 -32.5% 184.68 ? 12% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
795.10 ? 12% -67.6% 257.37 ? 47% perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
4.72 ? 2% -46.0% 2.55 ? 3% perf-sched.wait_and_delay.avg.ms.kthread_worker_fn.kthread.ret_from_fork
33.82 ? 5% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
549.04 ? 38% -52.6% 260.33 ? 49% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
446.94 ? 2% -31.2% 307.28 ? 16% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
14.28 ? 2% -34.4% 9.37 ? 18% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
129.29 ? 3% -9.4% 117.19 ? 7% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
135.30 -74.7% 34.23 ? 7% perf-sched.wait_and_delay.avg.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
6.69 ? 10% -26.5% 4.91 ? 15% perf-sched.wait_and_delay.avg.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
140.19 ? 10% -68.1% 44.66 ? 12% perf-sched.wait_and_delay.avg.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
9.67 ? 12% +110.3% 20.33 ? 57% perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64
9.33 ? 13% +117.9% 20.33 ? 57% perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read
239.67 +43.3% 343.33 ? 14% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
7.67 ? 22% +365.2% 35.67 ? 62% perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
877.33 ? 3% +40.5% 1232 ? 12% perf-sched.wait_and_delay.count.kthread_worker_fn.kthread.ret_from_fork
15.00 ? 5% -100.0% 0.00 perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
28.33 ? 26% +342.4% 125.33 ? 94% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
40.33 ? 2% +66.1% 67.00 ? 24% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
662.67 ? 2% +58.5% 1050 ? 22% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
252.00 +17.9% 297.00 ? 5% perf-sched.wait_and_delay.count.schedule_timeout.xfsaild.kthread.ret_from_fork
4591 +10.9% 5090 ? 6% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
33.67 ? 6% +54.5% 52.00 ? 19% perf-sched.wait_and_delay.count.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
63.43 ? 16% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
380.57 ? 15% -33.8% 251.78 ? 9% perf-sched.wait_and_delay.max.ms.schedule_timeout.__down.down.xfs_buf_lock
52.05 +1141.3% 646.11 ?126% perf-sched.wait_and_delay.max.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
184.41 ? 7% -64.9% 64.75 ? 10% perf-sched.wait_and_delay.max.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
382.80 ? 14% -38.8% 234.21 ? 2% perf-sched.wait_and_delay.max.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
405.68 ? 4% -65.7% 139.00 ? 7% perf-sched.wait_and_delay.max.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
273.46 -32.5% 184.66 ? 12% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
4.91 +63.1% 8.01 ? 7% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
53.43 ? 86% -96.0% 2.16 ?128% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
795.03 ? 12% -67.6% 257.32 ? 47% perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
9.00 ? 22% +297.0% 35.75 ? 19% perf-sched.wait_time.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.67 ? 3% +125.1% 1.50 ? 24% perf-sched.wait_time.avg.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range
4.69 ? 2% -46.3% 2.52 ? 3% perf-sched.wait_time.avg.ms.kthread_worker_fn.kthread.ret_from_fork
16.50 ?109% -82.0% 2.97 ? 18% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_write.xfs_ilock.xfs_inactive_ifree
0.07 ? 6% +135.9% 0.16 ? 52% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.terminate_walk.path_openat
14.16 ? 5% -51.0% 6.94 ? 67% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_inobt_init_common.xfs_inobt_init_cursor
4.04 ? 46% -94.6% 0.22 ?100% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
6.94 ?135% -98.1% 0.13 ? 74% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.__fdget_pos.ksys_write
0.18 ? 24% +3086.3% 5.87 ?132% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
14.48 -71.2% 4.17 perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.submit_bio_checks.submit_bio_noacct.submit_bio
0.34 ? 20% +2192.7% 7.75 ? 65% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.blkdev_issue_flush
15.09 ? 8% -60.5% 5.97 ? 18% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree
33.78 ? 5% -65.8% 11.55 ? 49% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
548.66 ? 38% -52.6% 260.28 ? 49% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
446.92 ? 2% -31.3% 307.25 ? 16% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
1.02 ? 38% +389.0% 5.01 ? 3% perf-sched.wait_time.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
14.26 ? 2% -34.6% 9.33 ? 18% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
7.28 ? 3% +31.4% 9.57 ? 12% perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
129.22 ? 3% -9.4% 117.09 ? 7% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
135.28 -74.7% 34.21 ? 7% perf-sched.wait_time.avg.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
6.66 ? 10% -26.7% 4.88 ? 15% perf-sched.wait_time.avg.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
13.42 ? 2% -37.5% 8.39 ? 13% perf-sched.wait_time.avg.ms.xlog_state_get_iclog_space.xlog_write.xlog_cil_push_work.process_one_work
13.36 +11.5% 14.89 perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.__xfs_log_force_lsn.xfs_log_force_lsn.xfs_file_fsync
140.17 ? 10% -68.2% 44.64 ? 12% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
13.54 ? 5% -92.2% 1.05 ? 34% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xlog_cil_push_work.process_one_work.worker_thread
209.61 +174.9% 576.20 ? 52% perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
241.09 ? 71% -99.6% 1.07 ? 68% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
1840 ? 64% -96.6% 62.86 ?136% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
29.29 ? 26% +1823.1% 563.33 ? 18% perf-sched.wait_time.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
5.45 ? 26% +569.0% 36.44 ?112% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.xfs_ilock.xfs_lock_two_inodes
0.19 ? 44% +89.9% 0.37 ? 20% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.terminate_walk.path_openat
238.89 ? 6% -64.5% 84.71 ? 90% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xfs_inobt_init_common.xfs_inobt_init_cursor
7.49 ? 32% -89.1% 0.82 ?130% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.xlog_ticket_alloc.xfs_log_reserve
199.71 ? 70% -93.0% 13.95 ?138% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_poll.do_sys_poll
104.91 ? 8% +534.3% 665.44 ?108% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.xlog_cil_push_work.process_one_work
0.84 ? 80% +4305.6% 36.80 ? 68% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion_io_timeout.submit_bio_wait.blkdev_issue_flush
233.55 ? 6% -63.8% 84.49 ? 30% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree
3.31 ? 71% +859.1% 31.72 ?110% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.xfs_log_commit_cil.__xfs_trans_commit.xfs_trans_roll
63.38 ? 16% +271.8% 235.66 ? 89% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.xlog_cil_push_work.process_one_work.worker_thread
379.32 ? 15% -38.8% 232.15 ? 2% perf-sched.wait_time.max.ms.schedule_timeout.__down.down.xfs_buf_lock
29.29 ? 73% +2771.8% 841.27 ? 81% perf-sched.wait_time.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io_timeout.submit_bio_wait
187.12 ? 11% -51.6% 90.49 ? 4% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__flush_work.xlog_cil_force_lsn
49.67 -45.8% 26.91 ? 53% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.xlog_cil_push_work.process_one_work
52.01 +1142.2% 646.05 ?126% perf-sched.wait_time.max.ms.schedule_timeout.xfsaild.kthread.ret_from_fork
184.39 ? 7% -64.9% 64.73 ? 10% perf-sched.wait_time.max.ms.xlog_cil_force_lsn.xfs_log_force.xfsaild_push.xfsaild
380.22 ? 15% -38.6% 233.31 ? 2% perf-sched.wait_time.max.ms.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
405.66 ? 4% -65.7% 138.98 ? 7% perf-sched.wait_time.max.ms.xlog_wait_on_iclog.xfsaild_push.xfsaild.kthread
39.60 ? 2% -19.0 20.61 ? 38% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
40.73 ? 2% -17.9 22.78 ? 36% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
40.70 ? 2% -17.9 22.76 ? 36% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
23.97 -12.8 11.15 ? 39% perf-profile.calltrace.cycles-pp.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
23.72 -12.7 11.01 ? 39% perf-profile.calltrace.cycles-pp.xfs_fs_destroy_inode.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
23.65 -12.7 10.98 ? 39% perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.do_unlinkat.do_syscall_64
23.58 -12.6 10.95 ? 39% perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.do_unlinkat
16.15 -10.6 5.56 ? 41% perf-profile.calltrace.cycles-pp.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode.destroy_inode
14.15 ? 2% -9.9 4.28 ? 43% perf-profile.calltrace.cycles-pp.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode
12.58 ? 4% -9.2 3.34 ? 45% perf-profile.calltrace.cycles-pp.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive
12.02 ? 4% -9.0 3.06 ? 46% perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree
43.11 ? 2% -7.3 35.78 ? 10% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
43.10 ? 2% -7.3 35.78 ? 10% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
43.00 ? 2% -7.3 35.69 ? 10% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
43.95 ? 2% -7.2 36.76 ? 10% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
7.09 ? 4% -5.5 1.56 ? 70% perf-profile.calltrace.cycles-pp.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree
14.21 ? 5% -5.5 8.71 ? 38% perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.93 ? 5% -5.4 8.55 ? 38% perf-profile.calltrace.cycles-pp.xfs_vn_unlink.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.88 ? 5% -5.4 8.53 ? 38% perf-profile.calltrace.cycles-pp.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat.do_syscall_64
29.28 ? 3% -4.8 24.43 ? 11% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
28.30 ? 3% -4.8 23.55 ? 11% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
22.56 ? 4% -4.0 18.61 ? 12% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
20.51 ? 4% -3.5 17.01 ? 10% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
4.13 ? 8% -3.3 0.83 ? 71% perf-profile.calltrace.cycles-pp.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree
3.92 ? 4% -3.1 0.85 ? 70% perf-profile.calltrace.cycles-pp.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree
13.08 ? 4% -2.4 10.65 ? 11% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
12.41 ? 5% -2.3 10.12 ? 10% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
2.57 ? 8% -2.0 0.52 ? 71% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree
2.42 ? 8% -1.9 0.52 ? 70% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt
3.16 ? 4% -1.8 1.32 ? 43% perf-profile.calltrace.cycles-pp.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat
7.96 ? 5% -1.6 6.38 ? 10% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
5.02 ? 9% -1.4 3.57 ? 33% perf-profile.calltrace.cycles-pp.xfs_trans_alloc.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat
1.79 ? 9% -1.4 0.36 ? 70% perf-profile.calltrace.cycles-pp.__xfs_btree_check_sblock.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt
4.94 ? 9% -1.4 3.51 ? 33% perf-profile.calltrace.cycles-pp.xfs_trans_reserve.xfs_trans_alloc.xfs_remove.xfs_vn_unlink.vfs_unlink
6.77 ? 6% -1.3 5.46 ? 11% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
3.72 ? 7% -1.3 2.43 ? 40% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat
3.37 ? 4% -1.2 2.17 ? 35% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode.destroy_inode
3.49 ? 7% -1.2 2.31 ? 41% perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_remove.xfs_vn_unlink.vfs_unlink
1.60 ? 15% -1.2 0.43 ? 72% perf-profile.calltrace.cycles-pp.__xfs_btree_check_sblock.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount
4.83 ? 4% -1.1 3.70 ? 31% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
3.12 ? 4% -1.1 2.07 ? 36% perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode
4.92 ? 5% -1.0 3.91 ? 11% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
4.71 ? 4% -1.0 3.74 ? 6% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
4.39 ? 6% -0.9 3.51 ? 11% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
4.15 ? 6% -0.8 3.34 ? 11% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.50 ? 11% -0.8 0.73 ? 70% perf-profile.calltrace.cycles-pp.xfs_difree_finobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive
3.80 ? 3% -0.8 3.03 ? 6% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.21 ? 10% -0.6 0.56 ? 71% perf-profile.calltrace.cycles-pp.xfs_mod_fdblocks.xfs_trans_reserve.xfs_trans_alloc.xfs_remove.xfs_vn_unlink
1.19 ? 11% -0.6 0.54 ? 71% perf-profile.calltrace.cycles-pp.__percpu_counter_compare.xfs_mod_fdblocks.xfs_trans_reserve.xfs_trans_alloc.xfs_remove
4.68 ? 2% -0.6 4.04 ? 17% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
2.93 ? 4% -0.6 2.31 ? 23% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
1.71 ? 3% -0.5 1.17 ? 36% perf-profile.calltrace.cycles-pp.xfs_iunlink_remove.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode
0.90 ? 9% -0.5 0.40 ? 71% perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit.xfs_remove.xfs_vn_unlink
1.37 ? 6% -0.5 0.87 ? 13% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
1.15 ? 4% -0.5 0.66 ? 72% perf-profile.calltrace.cycles-pp.xfs_read_agi.xfs_iunlink_remove.xfs_ifree.xfs_inactive_ifree.xfs_inactive
1.29 ? 5% -0.5 0.81 ? 11% perf-profile.calltrace.cycles-pp.update_blocked_averages.run_rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt
0.96 ? 13% -0.5 0.49 ? 70% perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_difree_finobt.xfs_difree.xfs_ifree.xfs_inactive_ifree
1.09 ? 6% -0.5 0.63 ? 71% perf-profile.calltrace.cycles-pp.xfs_trans_read_buf_map.xfs_read_agi.xfs_iunlink_remove.xfs_ifree.xfs_inactive_ifree
1.94 ? 5% -0.4 1.54 ? 24% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
0.98 ? 8% -0.4 0.58 ? 71% perf-profile.calltrace.cycles-pp.xfs_buf_read_map.xfs_trans_read_buf_map.xfs_read_agi.xfs_iunlink_remove.xfs_ifree
0.95 ? 10% -0.4 0.57 ? 71% perf-profile.calltrace.cycles-pp.xfs_buf_get_map.xfs_buf_read_map.xfs_trans_read_buf_map.xfs_read_agi.xfs_iunlink_remove
1.44 ? 6% -0.4 1.08 ? 8% perf-profile.calltrace.cycles-pp.flush_smp_call_function_from_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.89 ? 5% -0.4 0.53 ? 71% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
1.41 ? 5% -0.3 1.06 ? 34% perf-profile.calltrace.cycles-pp.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive
1.36 ? 5% -0.3 1.03 ? 33% perf-profile.calltrace.cycles-pp.xfs_log_space_wake.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree
0.67 ? 9% -0.3 0.38 ? 70% perf-profile.calltrace.cycles-pp.hrtimer_next_event_without.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
1.16 ? 10% -0.3 0.87 ? 11% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__percpu_counter_compare.xfs_mod_fdblocks.xfs_trans_reserve.xfs_trans_alloc
1.54 ? 6% -0.3 1.27 ? 11% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
1.62 ? 6% -0.3 1.36 ? 12% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
1.74 ? 3% -0.3 1.48 ? 8% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
0.99 ? 6% -0.2 0.74 ? 13% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_from_idle.do_idle.cpu_startup_entry.start_secondary
0.60 ? 5% -0.2 0.36 ? 70% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.10 ? 11% -0.2 0.89 ? 8% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.29 ? 4% -0.2 1.10 ? 13% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt
0.69 ? 16% -0.1 0.55 perf-profile.calltrace.cycles-pp.hrtimer_update_next_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.88 -0.1 0.76 ? 13% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu
0.82 ? 3% -0.1 0.70 ? 12% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start
0.76 -0.1 0.65 ? 6% perf-profile.calltrace.cycles-pp.update_rq_clock.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
0.75 ? 3% +0.2 0.93 ? 20% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.calltrace.cycles-pp.execve
0.80 ? 17% +0.3 1.09 ? 20% perf-profile.calltrace.cycles-pp.loop_queue_work.kthread_worker_fn.kthread.ret_from_fork
0.54 +0.4 0.90 ? 14% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
0.52 ? 3% +0.4 0.87 ? 13% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
0.87 ? 21% +0.4 1.25 ? 16% perf-profile.calltrace.cycles-pp.kthread_worker_fn.kthread.ret_from_fork
0.19 ?141% +0.5 0.66 ? 5% perf-profile.calltrace.cycles-pp.xlog_cil_push_work.process_one_work.worker_thread.kthread.ret_from_fork
0.35 ? 70% +0.5 0.85 ? 12% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +0.6 0.62 ? 4% perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.7 0.67 ? 9% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +0.8 0.76 ? 14% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +1.1 1.07 ? 13% perf-profile.calltrace.cycles-pp.__invalidate_mapping_pages.drop_pagecache_sb.iterate_supers.drop_caches_sysctl_handler.proc_sys_call_handler
0.00 +1.1 1.08 ? 34% perf-profile.calltrace.cycles-pp.mnt_get_count.mntput_no_expire.simple_recursive_removal.debugfs_remove.update_sched_domain_debugfs
0.00 +1.1 1.10 ? 34% perf-profile.calltrace.cycles-pp.mntput_no_expire.simple_recursive_removal.debugfs_remove.update_sched_domain_debugfs.rebuild_sched_domains_locked
0.00 +1.3 1.31 ? 33% perf-profile.calltrace.cycles-pp.simple_recursive_removal.debugfs_remove.update_sched_domain_debugfs.rebuild_sched_domains_locked.rebuild_sched_domains
0.00 +1.3 1.32 ? 33% perf-profile.calltrace.cycles-pp.debugfs_remove.update_sched_domain_debugfs.rebuild_sched_domains_locked.rebuild_sched_domains.cpuset_hotplug_workfn
0.61 ? 15% +1.3 1.95 ? 9% perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +1.4 1.41 ? 13% perf-profile.calltrace.cycles-pp.drop_pagecache_sb.iterate_supers.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write
0.00 +1.4 1.41 ? 13% perf-profile.calltrace.cycles-pp.iterate_supers.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write.vfs_write
0.43 ? 72% +1.4 1.87 ? 9% perf-profile.calltrace.cycles-pp.drm_fb_memcpy_dstclip.mgag200_simple_display_pipe_update.drm_atomic_helper_commit_planes.drm_atomic_helper_commit_tail.commit_tail
0.43 ? 72% +1.4 1.88 ? 9% perf-profile.calltrace.cycles-pp.drm_atomic_helper_commit_planes.drm_atomic_helper_commit_tail.commit_tail.drm_atomic_helper_commit.drm_atomic_helper_dirtyfb
0.43 ? 72% +1.4 1.88 ? 9% perf-profile.calltrace.cycles-pp.mgag200_simple_display_pipe_update.drm_atomic_helper_commit_planes.drm_atomic_helper_commit_tail.commit_tail.drm_atomic_helper_commit
0.43 ? 72% +1.4 1.88 ? 9% perf-profile.calltrace.cycles-pp.drm_atomic_helper_commit_tail.commit_tail.drm_atomic_helper_commit.drm_atomic_helper_dirtyfb.drm_fb_helper_damage_work
0.43 ? 72% +1.5 1.89 ? 9% perf-profile.calltrace.cycles-pp.commit_tail.drm_atomic_helper_commit.drm_atomic_helper_dirtyfb.drm_fb_helper_damage_work.process_one_work
0.43 ? 71% +1.5 1.89 ? 9% perf-profile.calltrace.cycles-pp.drm_atomic_helper_commit.drm_atomic_helper_dirtyfb.drm_fb_helper_damage_work.process_one_work.worker_thread
0.43 ? 71% +1.5 1.90 ? 9% perf-profile.calltrace.cycles-pp.drm_atomic_helper_dirtyfb.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
0.00 +1.5 1.54 ? 33% perf-profile.calltrace.cycles-pp.update_sched_domain_debugfs.rebuild_sched_domains_locked.rebuild_sched_domains.cpuset_hotplug_workfn.process_one_work
0.00 +1.8 1.76 ? 14% perf-profile.calltrace.cycles-pp.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write.vfs_write.ksys_write
0.00 +1.8 1.78 ? 13% perf-profile.calltrace.cycles-pp.proc_sys_call_handler.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.00 +2.0 1.96 ? 31% perf-profile.calltrace.cycles-pp.cpuset_hotplug_workfn.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +2.0 1.96 ? 31% perf-profile.calltrace.cycles-pp.rebuild_sched_domains.cpuset_hotplug_workfn.process_one_work.worker_thread.kthread
0.00 +2.0 1.96 ? 31% perf-profile.calltrace.cycles-pp.rebuild_sched_domains_locked.rebuild_sched_domains.cpuset_hotplug_workfn.process_one_work.worker_thread
0.39 ?141% +2.3 2.73 ? 23% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.memcpy_toio.drm_fb_memcpy_dstclip.mgag200_simple_display_pipe_update.drm_atomic_helper_commit_planes
0.00 +2.5 2.48 ? 5% perf-profile.calltrace.cycles-pp.rcu_momentary_dyntick_idle.multi_cpu_stop.cpu_stopper_thread.smpboot_thread_fn.kthread
0.00 +2.5 2.52 ? 9% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.5 2.52 ? 9% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.5 2.52 ? 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
0.00 +2.5 2.52 ? 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.5 2.53 ? 9% perf-profile.calltrace.cycles-pp.write
0.73 ? 3% +2.6 3.36 ? 11% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +2.7 2.73 ? 7% perf-profile.calltrace.cycles-pp.multi_cpu_stop.cpu_stopper_thread.smpboot_thread_fn.kthread.ret_from_fork
0.00 +2.8 2.79 ? 8% perf-profile.calltrace.cycles-pp.cpu_stopper_thread.smpboot_thread_fn.kthread.ret_from_fork
0.00 +3.3 3.28 ? 9% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
2.33 ? 5% +3.5 5.81 ? 13% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
1.99 ? 4% +3.5 5.47 ? 13% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
4.11 ? 8% +6.8 10.89 ? 5% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
4.11 ? 8% +6.8 10.90 ? 5% perf-profile.calltrace.cycles-pp.ret_from_fork
0.00 +9.8 9.79 ?113% perf-profile.calltrace.cycles-pp.xfs_create.xfs_generic_create.path_openat.do_filp_open.do_sys_openat2
0.00 +10.0 9.96 ?112% perf-profile.calltrace.cycles-pp.xfs_generic_create.path_openat.do_filp_open.do_sys_openat2.do_sys_open
0.00 +11.8 11.83 ?108% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
0.00 +11.8 11.85 ?108% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +12.1 12.08 ?107% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
0.00 +12.1 12.09 ?107% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
0.00 +12.1 12.14 ?107% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
0.00 +12.1 12.15 ?107% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.open64
0.00 +12.3 12.29 ?107% perf-profile.calltrace.cycles-pp.open64
39.60 ? 2% -19.0 20.63 ? 38% perf-profile.children.cycles-pp.do_unlinkat
23.98 -12.8 11.18 ? 39% perf-profile.children.cycles-pp.destroy_inode
23.72 -12.7 11.01 ? 39% perf-profile.children.cycles-pp.xfs_fs_destroy_inode
23.66 -12.7 10.98 ? 39% perf-profile.children.cycles-pp.xfs_inactive
23.58 -12.6 10.95 ? 39% perf-profile.children.cycles-pp.xfs_inactive_ifree
16.15 -10.6 5.57 ? 41% perf-profile.children.cycles-pp.xfs_ifree
14.15 ? 2% -9.9 4.28 ? 43% perf-profile.children.cycles-pp.xfs_difree
12.58 ? 4% -9.2 3.34 ? 45% perf-profile.children.cycles-pp.xfs_difree_inobt
43.11 ? 2% -7.3 35.78 ? 10% perf-profile.children.cycles-pp.start_secondary
43.95 ? 2% -7.2 36.75 ? 10% perf-profile.children.cycles-pp.do_idle
43.95 ? 2% -7.2 36.76 ? 10% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
43.95 ? 2% -7.2 36.76 ? 10% perf-profile.children.cycles-pp.cpu_startup_entry
13.03 ? 3% -6.6 6.47 ? 30% perf-profile.children.cycles-pp.xfs_check_agi_freecount
14.21 ? 5% -5.5 8.72 ? 38% perf-profile.children.cycles-pp.vfs_unlink
13.93 ? 5% -5.4 8.55 ? 38% perf-profile.children.cycles-pp.xfs_vn_unlink
13.89 ? 5% -5.4 8.54 ? 38% perf-profile.children.cycles-pp.xfs_remove
29.75 ? 3% -4.7 25.03 ? 11% perf-profile.children.cycles-pp.cpuidle_enter_state
29.87 ? 3% -4.7 25.15 ? 11% perf-profile.children.cycles-pp.cpuidle_enter
7.80 ? 2% -4.2 3.62 ? 25% perf-profile.children.cycles-pp.xfs_inobt_get_rec
22.93 ? 4% -3.2 19.71 ? 7% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
5.43 ? 3% -3.0 2.43 ? 26% perf-profile.children.cycles-pp.xfs_btree_check_sblock
4.45 ? 7% -2.4 2.04 ? 36% perf-profile.children.cycles-pp.xfs_btree_increment
24.74 ? 4% -2.4 22.35 ? 8% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
4.30 ? 3% -2.4 1.95 ? 23% perf-profile.children.cycles-pp.xfs_btree_get_rec
2.31 ? 12% -2.2 0.14 ? 85% perf-profile.children.cycles-pp.xfs_dir2_node_removename
3.68 ? 7% -2.1 1.56 ? 22% perf-profile.children.cycles-pp.__xfs_btree_check_sblock
15.19 ? 4% -2.1 13.13 ? 9% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
14.47 ? 4% -1.9 12.57 ? 8% perf-profile.children.cycles-pp.hrtimer_interrupt
3.17 ? 4% -1.8 1.32 ? 43% perf-profile.children.cycles-pp.xfs_dir_removename
6.95 ? 6% -1.4 5.59 ? 12% perf-profile.children.cycles-pp.menu_select
9.68 ? 4% -1.2 8.52 ? 7% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.39 ? 16% -1.0 0.41 ? 98% perf-profile.children.cycles-pp.xfs_da3_node_lookup_int
5.28 ? 4% -0.9 4.34 ? 4% perf-profile.children.cycles-pp.irq_exit_rcu
4.79 ? 4% -0.8 4.00 perf-profile.children.cycles-pp.__softirqentry_text_start
1.34 ? 12% -0.8 0.56 ? 21% perf-profile.children.cycles-pp.xfs_inobt_get_maxrecs
0.81 ? 15% -0.7 0.08 ? 16% perf-profile.children.cycles-pp.xfs_dir2_leafn_lookup_for_entry
1.34 ? 5% -0.7 0.66 ? 19% perf-profile.children.cycles-pp.xfs_inobt_irec_to_allocmask
3.03 ? 5% -0.7 2.37 ? 23% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.50 ? 11% -0.6 0.88 ? 34% perf-profile.children.cycles-pp.xfs_difree_finobt
1.71 ? 3% -0.5 1.17 ? 36% perf-profile.children.cycles-pp.xfs_iunlink_remove
1.12 ? 12% -0.5 0.63 ? 30% perf-profile.children.cycles-pp.xfs_errortag_test
1.42 ? 6% -0.5 0.95 ? 11% perf-profile.children.cycles-pp.run_rebalance_domains
1.44 ? 7% -0.5 0.97 ? 9% perf-profile.children.cycles-pp.update_blocked_averages
0.93 ? 9% -0.4 0.49 ? 20% perf-profile.children.cycles-pp.xfs_next_bit
1.46 ? 6% -0.4 1.11 ? 7% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle
0.73 ? 14% -0.3 0.39 ? 49% perf-profile.children.cycles-pp.xfs_droplink
0.67 ? 6% -0.3 0.35 ? 24% perf-profile.children.cycles-pp.filename_parentat
1.76 ? 6% -0.3 1.46 ? 11% perf-profile.children.cycles-pp.clockevents_program_event
0.66 ? 9% -0.3 0.36 ? 25% perf-profile.children.cycles-pp.xfs_verify_agino
0.63 ? 9% -0.3 0.34 ? 26% perf-profile.children.cycles-pp.path_parentat
1.63 ? 7% -0.3 1.34 ? 11% perf-profile.children.cycles-pp.rebalance_domains
0.46 ? 14% -0.3 0.20 ? 34% perf-profile.children.cycles-pp.xfs_btree_get_block
0.73 ? 9% -0.2 0.48 ? 32% perf-profile.children.cycles-pp.hrtimer_next_event_without
1.84 ? 3% -0.2 1.60 ? 12% perf-profile.children.cycles-pp.ktime_get
1.07 ? 7% -0.2 0.84 ? 13% perf-profile.children.cycles-pp.sched_ttwu_pending
0.51 ? 11% -0.2 0.28 ? 41% perf-profile.children.cycles-pp.xfs_btree_check_block
1.08 ? 13% -0.2 0.85 ? 8% perf-profile.children.cycles-pp.update_load_avg
1.41 ? 6% -0.2 1.20 ? 12% perf-profile.children.cycles-pp.sched_clock_cpu
0.90 ? 7% -0.2 0.71 ? 5% perf-profile.children.cycles-pp.ttwu_do_activate
1.24 ? 6% -0.2 1.05 ? 8% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.75 ? 7% -0.2 0.57 ? 7% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.36 ? 20% -0.2 0.19 ? 24% perf-profile.children.cycles-pp.xfs_inobt_btrec_to_irec
0.76 ? 5% -0.2 0.60 ? 4% perf-profile.children.cycles-pp.enqueue_entity
0.69 ? 8% -0.2 0.53 ? 16% perf-profile.children.cycles-pp.xfs_imap_to_bp
1.13 ? 7% -0.2 0.97 ? 10% perf-profile.children.cycles-pp.native_sched_clock
1.12 -0.2 0.97 ? 5% perf-profile.children.cycles-pp.update_rq_clock
0.91 ? 4% -0.2 0.76 ? 7% perf-profile.children.cycles-pp.irqtime_account_irq
0.86 ? 5% -0.1 0.71 ? 2% perf-profile.children.cycles-pp.enqueue_task_fair
0.44 ? 7% -0.1 0.30 ? 33% perf-profile.children.cycles-pp.xfs_iunlink_update_inode
0.63 ? 7% -0.1 0.48 ? 4% perf-profile.children.cycles-pp.__remove_hrtimer
0.36 ? 8% -0.1 0.23 ? 29% perf-profile.children.cycles-pp.xfs_btree_rec_offset
0.43 ? 7% -0.1 0.31 ? 14% perf-profile.children.cycles-pp.blk_complete_reqs
1.11 ? 8% -0.1 0.99 ? 5% perf-profile.children.cycles-pp.irq_enter_rcu
0.46 ? 9% -0.1 0.34 ? 2% perf-profile.children.cycles-pp.timerqueue_del
0.17 ? 22% -0.1 0.06 ? 73% perf-profile.children.cycles-pp.xfs_inode_mark_reclaimable
0.30 ? 15% -0.1 0.19 ? 19% perf-profile.children.cycles-pp.do_softirq
0.22 ? 5% -0.1 0.11 ? 18% perf-profile.children.cycles-pp.update_dl_rq_load_avg
0.29 ? 10% -0.1 0.18 ? 7% perf-profile.children.cycles-pp.scsi_io_completion
0.41 ? 23% -0.1 0.31 ? 5% perf-profile.children.cycles-pp.xfs_file_fsync
0.28 ? 11% -0.1 0.18 ? 7% perf-profile.children.cycles-pp.scsi_end_request
0.44 ? 9% -0.1 0.34 ? 7% perf-profile.children.cycles-pp.__update_load_avg_se
0.20 ? 20% -0.1 0.10 ? 40% perf-profile.children.cycles-pp.xfs_btree_readahead
0.32 ? 10% -0.1 0.22 ? 11% perf-profile.children.cycles-pp.xfs_buf_item_init
0.68 ? 3% -0.1 0.58 ? 4% perf-profile.children.cycles-pp.lapic_next_deadline
0.21 ? 12% -0.1 0.11 ? 29% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.22 ? 16% -0.1 0.13 ? 30% perf-profile.children.cycles-pp.__x64_sys_unlink
0.16 ? 6% -0.1 0.07 ? 25% perf-profile.children.cycles-pp.flush_end_io
0.23 ? 4% -0.1 0.15 ? 17% perf-profile.children.cycles-pp.xfs_perag_get
0.14 ? 16% -0.1 0.06 ? 70% perf-profile.children.cycles-pp.xfs_lock_two_inodes
0.16 ? 20% -0.1 0.08 ? 71% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.40 -0.1 0.32 ? 19% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.12 ? 10% -0.1 0.04 ? 71% perf-profile.children.cycles-pp.blk_mq_requeue_work
0.63 ? 5% -0.1 0.55 ? 2% perf-profile.children.cycles-pp.update_irq_load_avg
0.57 ? 7% -0.1 0.50 ? 6% perf-profile.children.cycles-pp.perf_output_copy
0.16 ? 5% -0.1 0.08 ? 20% perf-profile.children.cycles-pp.blk_flush_complete_seq
0.17 ? 17% -0.1 0.10 ? 31% perf-profile.children.cycles-pp.string
0.13 ? 3% -0.1 0.05 ? 84% perf-profile.children.cycles-pp.xfs_dir2_data_make_free
0.14 ? 17% -0.1 0.07 ? 78% perf-profile.children.cycles-pp.__radix_tree_lookup
0.45 ? 8% -0.1 0.38 ? 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.21 ? 9% -0.1 0.15 ? 12% perf-profile.children.cycles-pp.xfs_buf_item_put
0.13 ? 3% -0.1 0.06 ? 72% perf-profile.children.cycles-pp.__blk_mq_delay_run_hw_queue
0.10 ? 21% -0.1 0.03 ? 70% perf-profile.children.cycles-pp.xfs_perag_set_inode_tag
0.23 ? 16% -0.1 0.16 ? 12% perf-profile.children.cycles-pp.get_cpu_device
0.16 ? 16% -0.1 0.10 ? 21% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.36 ? 8% -0.1 0.30 ? 4% perf-profile.children.cycles-pp.__wake_up_common
0.10 ? 9% -0.1 0.04 ? 76% perf-profile.children.cycles-pp.__lookup_hash
0.13 ? 37% -0.1 0.08 ? 30% perf-profile.children.cycles-pp.asm_common_interrupt
0.13 ? 37% -0.1 0.08 ? 30% perf-profile.children.cycles-pp.common_interrupt
0.15 ? 8% -0.1 0.09 ? 15% perf-profile.children.cycles-pp.__switch_to_asm
0.31 ? 3% -0.1 0.26 ? 6% perf-profile.children.cycles-pp.rb_erase
0.31 ? 8% -0.1 0.26 ? 5% perf-profile.children.cycles-pp.pipe_read
0.25 ? 6% -0.1 0.20 ? 16% perf-profile.children.cycles-pp.xfs_trans_brelse
0.09 ? 18% -0.1 0.04 ? 73% perf-profile.children.cycles-pp.scsi_queue_rq
0.14 ? 8% -0.1 0.09 ? 5% perf-profile.children.cycles-pp.send_call_function_single_ipi
0.09 ? 5% -0.1 0.04 ? 71% perf-profile.children.cycles-pp.put_prev_task_fair
0.17 ? 22% -0.0 0.12 ? 10% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.27 ? 9% -0.0 0.22 ? 13% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.22 ? 7% -0.0 0.17 ? 4% perf-profile.children.cycles-pp.rb_next
0.18 ? 6% -0.0 0.14 ? 20% perf-profile.children.cycles-pp.rcu_nmi_exit
0.16 ? 10% -0.0 0.11 ? 12% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.13 ? 25% -0.0 0.09 ? 10% perf-profile.children.cycles-pp.wait_for_completion
0.13 ? 25% -0.0 0.09 ? 10% perf-profile.children.cycles-pp.xfs_buf_inode_iodone
0.14 ? 10% -0.0 0.10 ? 24% perf-profile.children.cycles-pp.xfs_trans_buf_item_match
0.13 ? 9% -0.0 0.09 ? 21% perf-profile.children.cycles-pp.is_bpf_text_address
0.08 ? 14% -0.0 0.04 ? 71% perf-profile.children.cycles-pp.xfs_dir2_data_entry_tag_p
0.11 ? 12% -0.0 0.07 ? 20% perf-profile.children.cycles-pp.pm_qos_read_value
0.19 ? 9% -0.0 0.15 ? 8% perf-profile.children.cycles-pp.xfs_lookup_get_search_key
0.07 ? 17% -0.0 0.04 ? 71% perf-profile.children.cycles-pp.check_tsc_unstable
0.07 ? 11% -0.0 0.04 ? 71% perf-profile.children.cycles-pp.devkmsg_read
0.10 ? 21% -0.0 0.07 ? 25% perf-profile.children.cycles-pp.xfs_log_force_lsn
0.09 ? 5% -0.0 0.06 ? 19% perf-profile.children.cycles-pp.rcu_irq_enter
0.05 ? 8% +0.0 0.07 ? 12% perf-profile.children.cycles-pp.up_write
0.06 ? 29% +0.0 0.09 ? 10% perf-profile.children.cycles-pp.__note_gp_changes
0.06 ? 7% +0.0 0.09 ? 28% perf-profile.children.cycles-pp.current_time
0.06 ? 13% +0.0 0.09 ? 18% perf-profile.children.cycles-pp.__fxstat64
0.07 ? 23% +0.0 0.10 ? 19% perf-profile.children.cycles-pp.perf_tp_event_match
0.06 ? 14% +0.0 0.10 ? 12% perf-profile.children.cycles-pp.elf_map
0.07 ? 25% +0.0 0.10 perf-profile.children.cycles-pp.__slab_free
0.07 ? 11% +0.0 0.11 ? 14% perf-profile.children.cycles-pp.rcu_read_unlock_strict
0.03 ? 70% +0.0 0.07 ? 12% perf-profile.children.cycles-pp.clear_page_erms
0.05 ? 8% +0.0 0.10 ? 12% perf-profile.children.cycles-pp.__do_sys_wait4
0.05 ? 8% +0.0 0.10 ? 12% perf-profile.children.cycles-pp.kernel_wait4
0.06 ? 13% +0.0 0.11 ? 23% perf-profile.children.cycles-pp.map_id_range_down
0.02 ?141% +0.0 0.06 ? 7% perf-profile.children.cycles-pp.perf_output_put_handle
0.03 ? 70% +0.0 0.08 ? 11% perf-profile.children.cycles-pp.page_remove_rmap
0.03 ? 70% +0.0 0.08 ? 20% perf-profile.children.cycles-pp.get_obj_cgroup_from_current
0.04 ? 71% +0.1 0.09 ? 21% perf-profile.children.cycles-pp.__mem_cgroup_charge
0.06 ? 14% +0.1 0.11 ? 29% perf-profile.children.cycles-pp.balance_fair
0.02 ?141% +0.1 0.07 ? 11% perf-profile.children.cycles-pp.free_unref_page_list
0.19 ? 5% +0.1 0.24 ? 15% perf-profile.children.cycles-pp.xfs_trans_committed_bulk
0.02 ?141% +0.1 0.07 ? 23% perf-profile.children.cycles-pp.vma_interval_tree_remove
0.00 +0.1 0.05 ? 8% perf-profile.children.cycles-pp.create_elf_tables
0.14 ? 6% +0.1 0.20 ? 4% perf-profile.children.cycles-pp.__might_sleep
0.14 ? 29% +0.1 0.20 ? 17% perf-profile.children.cycles-pp.__open64_nocancel
0.09 ? 10% +0.1 0.15 ? 14% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.07 ? 11% +0.1 0.13 ? 26% perf-profile.children.cycles-pp.mem_cgroup_charge
0.08 ? 40% +0.1 0.13 ? 21% perf-profile.children.cycles-pp.tlb_finish_mmu
0.08 ? 40% +0.1 0.13 ? 15% perf-profile.children.cycles-pp.new_inode_pseudo
0.04 ? 71% +0.1 0.09 ? 18% perf-profile.children.cycles-pp.do_wait
0.03 ? 70% +0.1 0.09 ? 9% perf-profile.children.cycles-pp.prep_new_page
0.08 ? 31% +0.1 0.14 ? 11% perf-profile.children.cycles-pp.alloc_inode
0.00 +0.1 0.06 ? 13% perf-profile.children.cycles-pp.__kmalloc_node
0.00 +0.1 0.06 perf-profile.children.cycles-pp.copy_pte_range
0.09 ? 5% +0.1 0.16 ? 25% perf-profile.children.cycles-pp.__perf_sw_event
0.04 ? 71% +0.1 0.11 ? 19% perf-profile.children.cycles-pp.perf_iterate_sb
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.vma_interval_tree_insert
0.32 ? 6% +0.1 0.39 ? 11% perf-profile.children.cycles-pp.__module_text_address
0.37 ? 3% +0.1 0.44 ? 6% perf-profile.children.cycles-pp.__libc_fork
0.28 ? 16% +0.1 0.34 ? 15% perf-profile.children.cycles-pp.step_into
0.02 ?141% +0.1 0.08 ? 20% perf-profile.children.cycles-pp.unlink_file_vma
0.00 +0.1 0.07 ? 14% perf-profile.children.cycles-pp.do_mkdirat
0.07 ? 17% +0.1 0.14 ? 5% perf-profile.children.cycles-pp.__close_nocancel
0.05 ? 71% +0.1 0.12 ? 6% perf-profile.children.cycles-pp.blk_mq_sched_insert_request
0.00 +0.1 0.07 ? 30% perf-profile.children.cycles-pp.unlock_page
0.02 ?141% +0.1 0.09 ? 18% perf-profile.children.cycles-pp.proc_pid_make_inode
0.07 ? 30% +0.1 0.14 ? 23% perf-profile.children.cycles-pp.xfs_trans_ijoin
0.00 +0.1 0.07 ? 12% perf-profile.children.cycles-pp.mkdir
0.00 +0.1 0.07 ? 23% perf-profile.children.cycles-pp.do_faccessat
0.02 ?141% +0.1 0.09 ? 18% perf-profile.children.cycles-pp.call_rcu
0.30 ? 8% +0.1 0.37 ? 5% perf-profile.children.cycles-pp.submit_bio_noacct
0.30 ? 8% +0.1 0.37 ? 6% perf-profile.children.cycles-pp.submit_bio
0.00 +0.1 0.08 ? 16% perf-profile.children.cycles-pp.___d_drop
0.00 +0.1 0.08 ? 40% perf-profile.children.cycles-pp.epoll_wait
0.09 ? 18% +0.1 0.17 ? 20% perf-profile.children.cycles-pp.perf_event_mmap
0.02 ?141% +0.1 0.09 ? 44% perf-profile.children.cycles-pp.obj_cgroup_charge_pages
0.23 ? 11% +0.1 0.31 ? 13% perf-profile.children.cycles-pp.__cond_resched
0.10 ? 25% +0.1 0.18 ? 24% perf-profile.children.cycles-pp.free_pgtables
0.00 +0.1 0.08 ? 20% perf-profile.children.cycles-pp.__d_drop
0.00 +0.1 0.08 ? 44% perf-profile.children.cycles-pp.lru_add_drain
0.02 ?141% +0.1 0.10 ? 24% perf-profile.children.cycles-pp.submit_bio_checks
0.02 ?141% +0.1 0.10 ? 31% perf-profile.children.cycles-pp.finish_fault
0.02 ?141% +0.1 0.10 ? 8% perf-profile.children.cycles-pp.__do_fault
0.00 +0.1 0.08 ? 20% perf-profile.children.cycles-pp.update_tasks_root_domain
0.00 +0.1 0.08 ? 5% perf-profile.children.cycles-pp.copy_page_range
0.00 +0.1 0.08 ? 31% perf-profile.children.cycles-pp.xfs_buffered_write_iomap_begin
0.10 ? 21% +0.1 0.18 ? 18% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.29 ? 14% +0.1 0.37 ? 12% perf-profile.children.cycles-pp.getname_flags
0.17 ? 12% +0.1 0.26 ? 20% perf-profile.children.cycles-pp.evlist__parse_sample
0.00 +0.1 0.09 ? 38% perf-profile.children.cycles-pp.dentry_lru_isolate
0.00 +0.1 0.09 ? 5% perf-profile.children.cycles-pp.try_charge
0.02 ?141% +0.1 0.10 ? 25% perf-profile.children.cycles-pp.__vma_adjust
0.00 +0.1 0.09 ? 47% perf-profile.children.cycles-pp.inode_init_always
0.00 +0.1 0.09 ? 33% perf-profile.children.cycles-pp._PyEval_EvalFrameDefault
0.00 +0.1 0.09 ? 33% perf-profile.children.cycles-pp.lookup_one_len
0.00 +0.1 0.09 ? 18% perf-profile.children.cycles-pp.page_mapping
0.00 +0.1 0.09 ? 13% perf-profile.children.cycles-pp.openat64
0.14 ? 28% +0.1 0.24 ? 15% perf-profile.children.cycles-pp.get_page_from_freelist
0.00 +0.1 0.10 ? 17% perf-profile.children.cycles-pp.new_inode
0.18 ? 19% +0.1 0.28 ? 17% perf-profile.children.cycles-pp.do_anonymous_page
0.42 ? 11% +0.1 0.52 ? 5% perf-profile.children.cycles-pp.__module_address
0.10 ? 12% +0.1 0.20 ? 37% perf-profile.children.cycles-pp.next_uptodate_page
0.00 +0.1 0.10 ? 16% perf-profile.children.cycles-pp.vm_area_dup
0.16 ? 11% +0.1 0.26 ? 19% perf-profile.children.cycles-pp.zap_pte_range
0.02 ?141% +0.1 0.12 ? 20% perf-profile.children.cycles-pp.mprotect_fixup
0.20 ? 4% +0.1 0.31 ? 28% perf-profile.children.cycles-pp.iomap_write_actor
0.13 ? 37% +0.1 0.24 ? 15% perf-profile.children.cycles-pp.__do_munmap
0.00 +0.1 0.11 ? 37% perf-profile.children.cycles-pp.__debugfs_create_file
0.04 ? 76% +0.1 0.15 ? 39% perf-profile.children.cycles-pp.obj_cgroup_charge
0.03 ?141% +0.1 0.14 ? 51% perf-profile.children.cycles-pp.xfs_inode_free_callback
0.00 +0.1 0.11 ? 41% perf-profile.children.cycles-pp.__list_lru_walk_one
0.00 +0.1 0.11 ? 41% perf-profile.children.cycles-pp.list_lru_walk_one
0.00 +0.1 0.11 ? 32% perf-profile.children.cycles-pp.start_creating
0.02 ?141% +0.1 0.13 ? 65% perf-profile.children.cycles-pp.__x64_sys_close
0.20 ? 16% +0.1 0.31 ? 27% perf-profile.children.cycles-pp.delay_tsc
0.00 +0.1 0.11 ? 23% perf-profile.children.cycles-pp.xas_load
0.18 ? 10% +0.1 0.30 ? 13% perf-profile.children.cycles-pp.begin_new_exec
0.10 ? 25% +0.1 0.22 ? 5% perf-profile.children.cycles-pp.wp_page_copy
0.00 +0.1 0.11 ? 33% perf-profile.children.cycles-pp.invalidate_inode_page
0.02 ?141% +0.1 0.13 ? 18% perf-profile.children.cycles-pp.__x64_sys_mprotect
0.02 ?141% +0.1 0.13 ? 18% perf-profile.children.cycles-pp.do_mprotect_pkey
0.18 ? 13% +0.1 0.30 ? 16% perf-profile.children.cycles-pp.unmap_page_range
0.18 ? 31% +0.1 0.30 ? 11% perf-profile.children.cycles-pp.cpumask_next
0.17 ? 11% +0.1 0.28 ? 13% perf-profile.children.cycles-pp.setlocale
0.04 ? 71% +0.1 0.16 ? 10% perf-profile.children.cycles-pp.pagecache_get_page
0.11 ? 4% +0.1 0.23 ? 3% perf-profile.children.cycles-pp.dup_mmap
0.12 ? 11% +0.1 0.24 ? 10% perf-profile.children.cycles-pp.dup_mm
0.28 ? 18% +0.1 0.41 ? 16% perf-profile.children.cycles-pp.___might_sleep
0.20 ? 10% +0.1 0.33 ? 16% perf-profile.children.cycles-pp.unmap_vmas
0.52 ? 10% +0.1 0.66 ? 9% perf-profile.children.cycles-pp.__xstat64
0.00 +0.1 0.13 ? 33% perf-profile.children.cycles-pp.xfs_file_fallocate
0.00 +0.1 0.13 ? 33% perf-profile.children.cycles-pp.xfs_free_file_space
0.05 ? 81% +0.1 0.18 ? 31% perf-profile.children.cycles-pp.d_alloc
0.20 ? 20% +0.1 0.34 ? 9% perf-profile.children.cycles-pp.xlog_state_release_iclog
0.10 ? 9% +0.1 0.24 ? 3% perf-profile.children.cycles-pp.path_init
0.50 ? 9% +0.1 0.65 ? 4% perf-profile.children.cycles-pp.vfs_statx
0.02 ?141% +0.1 0.17 ? 22% perf-profile.children.cycles-pp.__split_vma
0.48 ? 15% +0.2 0.63 ? 10% perf-profile.children.cycles-pp.io_serial_in
0.05 ? 84% +0.2 0.20 ? 59% perf-profile.children.cycles-pp.do_dentry_open
0.20 ? 16% +0.2 0.35 ? 12% perf-profile.children.cycles-pp.__alloc_pages
0.51 ? 11% +0.2 0.66 ? 4% perf-profile.children.cycles-pp.__do_sys_newstat
0.06 ? 71% +0.2 0.21 ? 45% perf-profile.children.cycles-pp.__alloc_file
0.07 ? 23% +0.2 0.23 ? 39% perf-profile.children.cycles-pp.alloc_empty_file
0.50 ? 11% +0.2 0.66 ? 5% perf-profile.children.cycles-pp.xlog_cil_push_work
0.25 ? 5% +0.2 0.41 ? 26% perf-profile.children.cycles-pp.filemap_map_pages
0.33 ? 9% +0.2 0.49 ? 8% perf-profile.children.cycles-pp.xlog_write
0.00 +0.2 0.17 ? 38% perf-profile.children.cycles-pp.xfs_dir2_block_lookup
0.00 +0.2 0.17 ? 38% perf-profile.children.cycles-pp.xfs_dir2_block_lookup_int
0.30 ? 6% +0.2 0.46 ? 7% perf-profile.children.cycles-pp.__do_sys_clone
0.00 +0.2 0.17 ? 17% perf-profile.children.cycles-pp.shrink_dentry_list
0.30 ? 8% +0.2 0.47 ? 6% perf-profile.children.cycles-pp.kernel_clone
0.24 ? 5% +0.2 0.42 ? 28% perf-profile.children.cycles-pp.iomap_apply
0.98 ? 5% +0.2 1.15 ? 7% perf-profile.children.cycles-pp.task_tick_fair
0.16 ? 42% +0.2 0.33 ? 24% perf-profile.children.cycles-pp.perf_mmap__read_head
0.24 ? 5% +0.2 0.42 ? 28% perf-profile.children.cycles-pp.iomap_file_buffered_write
0.43 ? 18% +0.2 0.61 ? 10% perf-profile.children.cycles-pp.kmem_cache_free
0.08 ? 32% +0.2 0.26 ? 29% perf-profile.children.cycles-pp.d_alloc_parallel
0.43 ? 8% +0.2 0.62 perf-profile.children.cycles-pp.filename_lookup
1.15 ? 12% +0.2 1.34 ? 2% perf-profile.children.cycles-pp.xfs_buf_lock
0.21 ? 3% +0.2 0.40 ? 11% perf-profile.children.cycles-pp.copy_process
1.14 ? 12% +0.2 1.33 ? 2% perf-profile.children.cycles-pp.__down
1.35 ? 11% +0.2 1.54 ? 7% perf-profile.children.cycles-pp.newidle_balance
1.28 ? 11% +0.2 1.47 ? 2% perf-profile.children.cycles-pp.schedule_timeout
0.13 ? 16% +0.2 0.33 ? 15% perf-profile.children.cycles-pp.__lookup_slow
0.00 +0.2 0.20 ? 20% perf-profile.children.cycles-pp.__pagevec_release
1.14 ? 12% +0.2 1.34 ? 2% perf-profile.children.cycles-pp.down
0.25 ? 19% +0.2 0.46 ? 4% perf-profile.children.cycles-pp.mmap_region
0.71 ? 8% +0.2 0.92 ? 11% perf-profile.children.cycles-pp.walk_component
0.23 ? 9% +0.2 0.44 ? 10% perf-profile.children.cycles-pp.dput
0.38 ? 6% +0.2 0.59 ? 9% perf-profile.children.cycles-pp.load_elf_binary
0.07 ? 89% +0.2 0.29 ? 8% perf-profile.children.cycles-pp.release_pages
0.40 ? 5% +0.2 0.62 ? 8% perf-profile.children.cycles-pp.exec_binprm
0.43 ? 8% +0.2 0.65 ? 3% perf-profile.children.cycles-pp.path_lookupat
0.58 ? 4% +0.2 0.80 ? 9% perf-profile.children.cycles-pp.execve
0.27 ? 21% +0.2 0.49 ? 9% perf-profile.children.cycles-pp.do_mmap
0.06 ? 19% +0.2 0.29 ? 14% perf-profile.children.cycles-pp.__dentry_kill
0.33 ? 8% +0.2 0.56 ? 29% perf-profile.children.cycles-pp.xfs_file_buffered_write
0.37 ? 9% +0.2 0.60 ? 28% perf-profile.children.cycles-pp.do_iter_write
0.13 ? 64% +0.2 0.36 ? 59% perf-profile.children.cycles-pp.__fput
0.09 ? 70% +0.2 0.32 ? 18% perf-profile.children.cycles-pp.xfs_dir3_leaf_read
0.10 ? 39% +0.2 0.33 ? 61% perf-profile.children.cycles-pp.do_open
0.47 +0.2 0.71 ? 7% perf-profile.children.cycles-pp.bprm_execve
0.00 +0.2 0.24 ? 12% perf-profile.children.cycles-pp.xas_find
0.26 ? 16% +0.2 0.50 ? 9% perf-profile.children.cycles-pp.ksys_mmap_pgoff
0.37 ? 9% +0.2 0.61 ? 29% perf-profile.children.cycles-pp.lo_write_bvec
0.30 ? 17% +0.2 0.55 ? 7% perf-profile.children.cycles-pp.vm_mmap_pgoff
0.30 ? 14% +0.2 0.55 ? 7% perf-profile.children.cycles-pp.do_exit
0.30 ? 14% +0.3 0.55 ? 6% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.30 ? 14% +0.3 0.55 ? 6% perf-profile.children.cycles-pp.do_group_exit
0.33 ? 8% +0.3 0.59 ? 26% perf-profile.children.cycles-pp.do_iter_readv_writev
0.27 ? 37% +0.3 0.54 ? 25% perf-profile.children.cycles-pp.rcu_do_batch
0.21 ? 45% +0.3 0.48 ? 36% perf-profile.children.cycles-pp.perf_mmap__push
0.00 +0.3 0.27 ? 30% perf-profile.children.cycles-pp.common_cpu_die.cold
0.00 +0.3 0.27 ? 23% perf-profile.children.cycles-pp.prune_dcache_sb
0.35 ? 9% +0.3 0.63 ? 8% perf-profile.children.cycles-pp.exit_mmap
0.23 ? 46% +0.3 0.51 ? 36% perf-profile.children.cycles-pp.record__mmap_read_evlist
0.35 ? 9% +0.3 0.64 ? 9% perf-profile.children.cycles-pp.mmput
0.16 ? 52% +0.3 0.45 ? 51% perf-profile.children.cycles-pp.task_work_run
0.00 +0.3 0.30 ? 28% perf-profile.children.cycles-pp.takedown_cpu
0.80 ? 17% +0.3 1.10 ? 21% perf-profile.children.cycles-pp.loop_queue_work
0.63 ? 23% +0.3 0.93 ? 16% perf-profile.children.cycles-pp.rcu_core
0.00 +0.3 0.31 ? 35% perf-profile.children.cycles-pp.build_sched_domains
0.17 ? 70% +0.3 0.48 ? 8% perf-profile.children.cycles-pp.xfs_dir2_leaf_lookup_int
0.58 ? 4% +0.3 0.90 ? 11% perf-profile.children.cycles-pp.do_execveat_common
0.58 ? 4% +0.3 0.91 ? 12% perf-profile.children.cycles-pp.__x64_sys_execve
0.00 +0.3 0.33 ? 33% perf-profile.children.cycles-pp.partition_sched_domains_locked
0.37 ? 9% +0.3 0.70 ? 21% perf-profile.children.cycles-pp.do_fault
0.75 ? 9% +0.3 1.09 ? 26% perf-profile.children.cycles-pp.kmem_cache_alloc
0.77 ? 5% +0.3 1.12 ? 10% perf-profile.children.cycles-pp.vprintk_emit
0.77 ? 5% +0.3 1.12 ? 9% perf-profile.children.cycles-pp.console_unlock
0.00 +0.4 0.35 ? 29% perf-profile.children.cycles-pp.super_cache_scan
0.00 +0.4 0.35 ? 30% perf-profile.children.cycles-pp.drop_slab
0.00 +0.4 0.35 ? 30% perf-profile.children.cycles-pp.drop_slab_node
0.00 +0.4 0.35 ? 30% perf-profile.children.cycles-pp.shrink_slab
0.00 +0.4 0.35 ? 30% perf-profile.children.cycles-pp.do_shrink_slab
0.74 ? 2% +0.4 1.10 ? 11% perf-profile.children.cycles-pp.printk
0.57 ? 7% +0.4 0.93 ? 12% perf-profile.children.cycles-pp.serial8250_console_putchar
0.58 ? 7% +0.4 0.95 ? 13% perf-profile.children.cycles-pp.wait_for_xmitr
0.28 ? 31% +0.4 0.64 ? 43% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.58 ? 8% +0.4 0.95 ? 13% perf-profile.children.cycles-pp.uart_console_write
0.87 ? 21% +0.4 1.25 ? 16% perf-profile.children.cycles-pp.kthread_worker_fn
0.30 ? 28% +0.4 0.68 ? 41% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.59 ? 8% +0.4 0.98 ? 14% perf-profile.children.cycles-pp.serial8250_console_write
0.00 +0.4 0.43 ? 12% perf-profile.children.cycles-pp.cpuhp_invoke_callback_range
0.00 +0.4 0.44 ? 14% perf-profile.children.cycles-pp.cpu_down
0.00 +0.4 0.44 ? 14% perf-profile.children.cycles-pp._cpu_down
0.00 +0.4 0.45 ? 14% perf-profile.children.cycles-pp.kernfs_fop_write_iter
0.00 +0.4 0.45 ? 14% perf-profile.children.cycles-pp.online_store
0.00 +0.4 0.45 ? 14% perf-profile.children.cycles-pp.device_offline
0.00 +0.5 0.46 ? 13% perf-profile.children.cycles-pp.cpuhp_invoke_callback
0.00 +0.6 0.56 ? 10% perf-profile.children.cycles-pp.find_lock_entries
0.05 ? 8% +0.6 0.65 ? 84% perf-profile.children.cycles-pp.xfs_ialloc_read_agi
1.35 ? 25% +0.6 1.95 ? 14% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.80 ? 8% +0.6 1.40 ? 16% perf-profile.children.cycles-pp.__handle_mm_fault
0.00 +0.6 0.61 ? 98% perf-profile.children.cycles-pp.xfs_init_new_inode
0.00 +0.6 0.63 ?102% perf-profile.children.cycles-pp.xfs_dir_lookup
0.00 +0.6 0.64 ?100% perf-profile.children.cycles-pp.xfs_lookup
2.15 ? 16% +0.7 2.84 ? 10% perf-profile.children.cycles-pp._raw_spin_lock
0.89 ? 5% +0.7 1.59 ? 14% perf-profile.children.cycles-pp.handle_mm_fault
0.00 +0.7 0.70 ?101% perf-profile.children.cycles-pp.xfs_vn_lookup
1.11 ? 4% +0.7 1.83 ? 14% perf-profile.children.cycles-pp.do_user_addr_fault
1.18 ? 3% +0.7 1.90 ? 14% perf-profile.children.cycles-pp.exc_page_fault
1.26 +0.8 2.06 ? 15% perf-profile.children.cycles-pp.asm_exc_page_fault
0.00 +1.0 1.00 ?102% perf-profile.children.cycles-pp.xfs_dir_createname
0.00 +1.1 1.09 ? 12% perf-profile.children.cycles-pp.__invalidate_mapping_pages
0.00 +1.1 1.14 ? 34% perf-profile.children.cycles-pp.mnt_get_count
0.00 +1.2 1.19 ? 32% perf-profile.children.cycles-pp.mntput_no_expire
0.58 ? 18% +1.3 1.87 ? 9% perf-profile.children.cycles-pp.drm_fb_memcpy_dstclip
0.59 ? 17% +1.3 1.88 ? 9% perf-profile.children.cycles-pp.drm_atomic_helper_commit_planes
0.59 ? 17% +1.3 1.88 ? 9% perf-profile.children.cycles-pp.mgag200_simple_display_pipe_update
0.58 ? 19% +1.3 1.87 ? 9% perf-profile.children.cycles-pp.memcpy_toio
0.59 ? 17% +1.3 1.88 ? 9% perf-profile.children.cycles-pp.drm_atomic_helper_commit_tail
0.59 ? 17% +1.3 1.89 ? 9% perf-profile.children.cycles-pp.commit_tail
0.60 ? 15% +1.3 1.90 ? 9% perf-profile.children.cycles-pp.drm_atomic_helper_dirtyfb
0.59 ? 17% +1.3 1.89 ? 9% perf-profile.children.cycles-pp.drm_atomic_helper_commit
0.00 +1.3 1.31 ? 33% perf-profile.children.cycles-pp.simple_recursive_removal
0.00 +1.3 1.32 ? 33% perf-profile.children.cycles-pp.debugfs_remove
0.61 ? 15% +1.3 1.95 ? 9% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
0.00 +1.4 1.41 ? 13% perf-profile.children.cycles-pp.iterate_supers
0.00 +1.4 1.41 ? 13% perf-profile.children.cycles-pp.drop_pagecache_sb
0.00 +1.5 1.54 ? 33% perf-profile.children.cycles-pp.update_sched_domain_debugfs
0.00 +1.8 1.76 ? 14% perf-profile.children.cycles-pp.drop_caches_sysctl_handler
0.00 +1.8 1.78 ? 13% perf-profile.children.cycles-pp.proc_sys_call_handler
0.00 +2.0 1.96 ? 31% perf-profile.children.cycles-pp.cpuset_hotplug_workfn
0.00 +2.0 1.96 ? 31% perf-profile.children.cycles-pp.rebuild_sched_domains
0.00 +2.0 1.96 ? 31% perf-profile.children.cycles-pp.rebuild_sched_domains_locked
0.40 ? 10% +2.1 2.53 ? 9% perf-profile.children.cycles-pp.write
0.00 +2.2 2.17 ?106% perf-profile.children.cycles-pp.xfs_trans_alloc_icreate
1.06 ? 2% +2.3 3.36 ? 11% perf-profile.children.cycles-pp.new_sync_write
1.10 ? 2% +2.3 3.42 ? 11% perf-profile.children.cycles-pp.vfs_write
1.11 ? 3% +2.3 3.45 ? 12% perf-profile.children.cycles-pp.ksys_write
0.00 +2.5 2.49 ? 5% perf-profile.children.cycles-pp.rcu_momentary_dyntick_idle
0.00 +2.8 2.76 ? 8% perf-profile.children.cycles-pp.multi_cpu_stop
0.02 ?141% +2.8 2.79 ? 8% perf-profile.children.cycles-pp.cpu_stopper_thread
0.33 ? 38% +3.0 3.28 ? 9% perf-profile.children.cycles-pp.smpboot_thread_fn
2.33 ? 5% +3.5 5.81 ? 13% perf-profile.children.cycles-pp.worker_thread
1.99 ? 4% +3.5 5.47 ? 13% perf-profile.children.cycles-pp.process_one_work
0.00 +4.0 4.02 ?120% perf-profile.children.cycles-pp.xfs_dialloc
4.11 ? 8% +6.8 10.89 ? 5% perf-profile.children.cycles-pp.kthread
4.14 ? 8% +6.8 10.93 ? 5% perf-profile.children.cycles-pp.ret_from_fork
0.00 +9.8 9.84 ?112% perf-profile.children.cycles-pp.xfs_create
0.00 +10.0 10.00 ?112% perf-profile.children.cycles-pp.xfs_generic_create
0.47 ? 4% +11.9 12.40 ?103% perf-profile.children.cycles-pp.path_openat
0.47 ? 4% +12.0 12.42 ?103% perf-profile.children.cycles-pp.do_filp_open
0.22 ? 14% +12.1 12.30 ?107% perf-profile.children.cycles-pp.open64
0.54 +12.2 12.77 ?101% perf-profile.children.cycles-pp.do_sys_openat2
0.55 ? 2% +12.2 12.79 ?101% perf-profile.children.cycles-pp.do_sys_open
2.21 ? 8% -1.3 0.94 ? 26% perf-profile.self.cycles-pp.__xfs_btree_check_sblock
1.05 ? 8% -0.6 0.45 ? 42% perf-profile.self.cycles-pp.xfs_inobt_get_rec
3.21 ? 12% -0.6 2.63 ? 11% perf-profile.self.cycles-pp.menu_select
0.84 ? 7% -0.5 0.32 ? 18% perf-profile.self.cycles-pp.xfs_btree_get_rec
0.93 ? 5% -0.5 0.42 ? 19% perf-profile.self.cycles-pp.xfs_inobt_get_maxrecs
0.96 ? 11% -0.5 0.46 ? 47% perf-profile.self.cycles-pp.xfs_btree_increment
1.02 ? 10% -0.5 0.56 ? 25% perf-profile.self.cycles-pp.xfs_errortag_test
0.86 ? 10% -0.4 0.47 ? 20% perf-profile.self.cycles-pp.xfs_next_bit
0.70 ? 12% -0.4 0.32 ? 42% perf-profile.self.cycles-pp.xfs_btree_check_sblock
0.62 ? 11% -0.3 0.33 ? 25% perf-profile.self.cycles-pp.xfs_verify_agino
0.41 ? 10% -0.2 0.18 ? 33% perf-profile.self.cycles-pp.xfs_btree_get_block
0.43 ? 20% -0.2 0.22 ? 22% perf-profile.self.cycles-pp.xfs_inobt_irec_to_allocmask
0.48 ? 10% -0.2 0.27 ? 44% perf-profile.self.cycles-pp.xfs_btree_check_block
0.33 ? 11% -0.2 0.15 ? 19% perf-profile.self.cycles-pp.xfs_check_agi_freecount
0.36 ? 20% -0.2 0.19 ? 26% perf-profile.self.cycles-pp.xfs_inobt_btrec_to_irec
0.39 ? 18% -0.2 0.24 ? 8% perf-profile.self.cycles-pp.update_blocked_averages
0.67 ? 6% -0.1 0.52 ? 4% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.33 ? 11% -0.1 0.19 ? 29% perf-profile.self.cycles-pp.clockevents_program_event
0.33 ? 6% -0.1 0.20 ? 34% perf-profile.self.cycles-pp.xfs_btree_rec_offset
0.42 ? 11% -0.1 0.30 ? 19% perf-profile.self.cycles-pp.irqtime_account_irq
0.78 ? 3% -0.1 0.67 ? 11% perf-profile.self.cycles-pp.read_tsc
0.22 ? 5% -0.1 0.11 ? 18% perf-profile.self.cycles-pp.update_dl_rq_load_avg
0.26 ? 12% -0.1 0.15 ? 12% perf-profile.self.cycles-pp.__hrtimer_get_next_event
0.20 ? 18% -0.1 0.10 ? 37% perf-profile.self.cycles-pp.xfs_btree_readahead
0.29 ? 10% -0.1 0.20 ? 4% perf-profile.self.cycles-pp.xfs_log_commit_cil
0.41 ? 7% -0.1 0.32 ? 5% perf-profile.self.cycles-pp.__update_load_avg_se
0.20 ? 10% -0.1 0.11 ? 28% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.66 ? 3% -0.1 0.58 ? 5% perf-profile.self.cycles-pp.lapic_next_deadline
0.35 ? 6% -0.1 0.27 ? 9% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt
0.31 ? 10% -0.1 0.23 ? 17% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt
0.40 ? 2% -0.1 0.31 ? 18% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.19 ? 4% -0.1 0.12 ? 10% perf-profile.self.cycles-pp.tick_sched_timer
0.22 ? 2% -0.1 0.14 ? 8% perf-profile.self.cycles-pp.enqueue_entity
0.31 ? 3% -0.1 0.23 ? 19% perf-profile.self.cycles-pp.irq_exit_rcu
0.62 ? 5% -0.1 0.55 ? 2% perf-profile.self.cycles-pp.update_irq_load_avg
0.44 ? 8% -0.1 0.37 ? 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.21 ? 14% -0.1 0.14 ? 25% perf-profile.self.cycles-pp.rcu_eqs_enter
0.16 ? 15% -0.1 0.09 ? 26% perf-profile.self.cycles-pp.hrtimer_next_event_without
0.26 ? 15% -0.1 0.19 ? 10% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.22 ? 14% -0.1 0.16 ? 13% perf-profile.self.cycles-pp.get_cpu_device
0.17 ? 8% -0.1 0.12 ? 22% perf-profile.self.cycles-pp.get_perf_callchain
0.14 ? 11% -0.1 0.09 ? 15% perf-profile.self.cycles-pp.__switch_to_asm
0.32 ? 5% -0.1 0.26 ? 11% perf-profile.self.cycles-pp.scheduler_tick
0.31 ? 4% -0.1 0.25 ? 4% perf-profile.self.cycles-pp.rb_erase
0.14 ? 8% -0.1 0.09 ? 5% perf-profile.self.cycles-pp.send_call_function_single_ipi
0.17 ? 22% -0.0 0.12 ? 10% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.14 ? 20% -0.0 0.09 ? 27% perf-profile.self.cycles-pp.rcu_idle_exit
0.16 ? 10% -0.0 0.11 ? 12% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.12 ? 27% -0.0 0.07 ? 28% perf-profile.self.cycles-pp.sched_ttwu_pending
0.08 ? 14% -0.0 0.04 ? 70% perf-profile.self.cycles-pp.xfs_trans_reserve
0.14 ? 13% -0.0 0.09 ? 20% perf-profile.self.cycles-pp.xfs_trans_buf_item_match
0.20 ? 6% -0.0 0.16 ? 10% perf-profile.self.cycles-pp.rb_next
0.08 ? 14% -0.0 0.04 ? 71% perf-profile.self.cycles-pp.xfs_buf_item_size_segment
0.24 ? 11% -0.0 0.20 ? 2% perf-profile.self.cycles-pp.perf_output_copy
0.18 ? 2% -0.0 0.14 ? 10% perf-profile.self.cycles-pp.__remove_hrtimer
0.16 ? 2% -0.0 0.13 ? 6% perf-profile.self.cycles-pp.update_curr
0.10 ? 21% -0.0 0.07 ? 20% perf-profile.self.cycles-pp.pm_qos_read_value
0.17 ? 14% -0.0 0.14 ? 3% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.09 ? 14% -0.0 0.07 ? 14% perf-profile.self.cycles-pp.account_process_tick
0.11 ? 7% -0.0 0.09 ? 5% perf-profile.self.cycles-pp.dequeue_entity
0.07 ? 28% +0.0 0.10 ? 14% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.07 ? 18% +0.0 0.10 ? 12% perf-profile.self.cycles-pp.perf_tp_event_match
0.07 ? 23% +0.0 0.10 ? 12% perf-profile.self.cycles-pp.symbol__is_idle
0.06 ? 8% +0.0 0.09 perf-profile.self.cycles-pp.__cond_resched
0.07 ? 25% +0.0 0.10 perf-profile.self.cycles-pp.__slab_free
0.11 ? 17% +0.0 0.15 ? 11% perf-profile.self.cycles-pp.memset_erms
0.10 ? 9% +0.0 0.14 ? 28% perf-profile.self.cycles-pp.__handle_mm_fault
0.06 ? 13% +0.0 0.10 ? 29% perf-profile.self.cycles-pp.map_id_range_down
0.05 ? 70% +0.0 0.09 ? 15% perf-profile.self.cycles-pp.down_write
0.12 ? 7% +0.1 0.18 ? 10% perf-profile.self.cycles-pp.__might_sleep
0.09 ? 10% +0.1 0.15 ? 17% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.02 ?141% +0.1 0.07 ? 23% perf-profile.self.cycles-pp.vma_interval_tree_remove
0.06 ? 23% +0.1 0.12 ? 27% perf-profile.self.cycles-pp.zap_pte_range
0.00 +0.1 0.06 perf-profile.self.cycles-pp.vma_interval_tree_insert
0.00 +0.1 0.06 ? 23% perf-profile.self.cycles-pp.call_rcu
0.04 ? 71% +0.1 0.10 ? 21% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.04 ? 73% +0.1 0.10 ? 32% perf-profile.self.cycles-pp.sys_imageblit
0.00 +0.1 0.06 ? 7% perf-profile.self.cycles-pp.perf_output_put_handle
0.00 +0.1 0.07 ? 14% perf-profile.self.cycles-pp.try_charge
0.00 +0.1 0.07 ? 25% perf-profile.self.cycles-pp.pagecache_get_page
0.04 ? 71% +0.1 0.11 ? 23% perf-profile.self.cycles-pp.cpumask_next
0.17 ? 14% +0.1 0.24 ? 23% perf-profile.self.cycles-pp.evlist__parse_sample
0.00 +0.1 0.07 ? 12% perf-profile.self.cycles-pp.___d_drop
0.00 +0.1 0.07 ? 12% perf-profile.self.cycles-pp.clear_page_erms
0.00 +0.1 0.07 ? 28% perf-profile.self.cycles-pp.xas_load
0.00 +0.1 0.08 ? 16% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.03 ?141% +0.1 0.11 ? 34% perf-profile.self.cycles-pp.perf_session__process_events
0.00 +0.1 0.09 ? 30% perf-profile.self.cycles-pp._PyEval_EvalFrameDefault
0.33 ? 15% +0.1 0.42 ? 8% perf-profile.self.cycles-pp.kmem_cache_free
0.42 ? 11% +0.1 0.51 ? 6% perf-profile.self.cycles-pp.__module_address
0.00 +0.1 0.09 ? 18% perf-profile.self.cycles-pp.page_mapping
0.20 ? 16% +0.1 0.31 ? 26% perf-profile.self.cycles-pp.delay_tsc
0.00 +0.1 0.12 ? 30% perf-profile.self.cycles-pp.multi_cpu_stop
0.02 ?141% +0.1 0.15 ? 16% perf-profile.self.cycles-pp.path_init
0.48 ? 15% +0.2 0.63 ? 10% perf-profile.self.cycles-pp.io_serial_in
0.00 +0.2 0.15 ? 17% perf-profile.self.cycles-pp.release_pages
0.14 ? 45% +0.2 0.31 ? 27% perf-profile.self.cycles-pp.perf_mmap__read_head
0.00 +0.2 0.18 ? 9% perf-profile.self.cycles-pp.xas_find
0.00 +0.2 0.20 ? 12% perf-profile.self.cycles-pp.drop_pagecache_sb
0.00 +0.3 0.34 ? 10% perf-profile.self.cycles-pp.find_lock_entries
1.32 ? 26% +0.5 1.83 ? 13% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.00 +1.0 0.98 ? 37% perf-profile.self.cycles-pp.mnt_get_count
0.51 ? 13% +1.3 1.81 ? 7% perf-profile.self.cycles-pp.memcpy_toio
0.00 +2.5 2.49 ? 5% perf-profile.self.cycles-pp.rcu_momentary_dyntick_idle





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation

Thanks,
Oliver Sang

---

Attachments:

(No filename) (208.17 kB)
config-5.13.0-rc4-00092-geef983ffeae7 (176.88 kB)
job-script (8.46 kB)
job.yaml (5.82 kB)
reproduce (574.00 B)
Download all attachments