Greeting,
FYI, we noticed a 41.3% improvement of hackbench.throughput due to commit:
commit: 5387c90490f7f42df3209154ca955a453ee01b41 ("mm/memcg: improve refill_obj_stock() performance")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: hackbench
on test machine: 104 threads 2 sockets Skylake with 192G memory
with following parameters:
nr_threads: 100%
iterations: 4
mode: threads
ipc: socket
cpufreq_governor: performance
ucode: 0x2006a0a
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sched/cfs-scheduler/hackbench.c
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput 51.9% improvement |
| test machine | 96 threads 2 sockets Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=socket |
| | iterations=8 |
| | mode=process |
| | nr_threads=1600% |
| | ucode=0x4003006 |
+------------------+----------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
bin/lkp run generated-yaml-file
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-9/performance/socket/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/hackbench/0x2006a0a
commit:
68ac5b3c8d ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")
5387c90490 ("mm/memcg: improve refill_obj_stock() performance")
68ac5b3c8db2fda0 5387c90490f7f42df3209154ca9
---------------- ---------------------------
%stddev %change %stddev
\ | \
82459 +41.3% 116521 ? 2% hackbench.throughput
601.44 -29.1% 426.40 ? 2% hackbench.time.elapsed_time
601.44 -29.1% 426.40 ? 2% hackbench.time.elapsed_time.max
2.344e+09 ? 3% -85.5% 3.396e+08 ? 21% hackbench.time.involuntary_context_switches
53613 -32.8% 36018 ? 2% hackbench.time.system_time
3.493e+09 ? 2% -77.9% 7.702e+08 ? 14% hackbench.time.voluntary_context_switches
1374661 ? 22% -58.4% 572519 ? 13% numa-numastat.node0.local_node
1403760 ? 21% -55.5% 623973 ? 15% numa-numastat.node0.numa_hit
651.41 -26.7% 477.78 ? 2% uptime.boot
5074 ? 3% +9.5% 5557 ? 3% uptime.idle
65187074 ? 25% +336.1% 2.843e+08 ? 6% cpuidle.C1.time
5686636 ? 13% +402.4% 28566892 ? 5% cpuidle.C1.usage
5451564 ? 23% +189.6% 15789364 ? 5% cpuidle.POLL.time
1684249 ? 24% +103.1% 3421281 ? 11% cpuidle.POLL.usage
3.45 ? 4% +6.9 10.38 ? 36% mpstat.cpu.all.idle%
1.19 ? 4% -0.1 1.04 ? 3% mpstat.cpu.all.irq%
0.02 +0.0 0.04 ? 5% mpstat.cpu.all.soft%
12.20 +2.7 14.92 ? 5% mpstat.cpu.all.usr%
3295209 +80.7% 5954774 ? 20% vmstat.memory.cache
2143 -22.4% 1663 vmstat.procs.r
9678625 ? 2% -72.3% 2678610 ? 17% vmstat.system.cs
955036 ? 3% -41.1% 562152 ? 7% vmstat.system.in
568807 ? 3% +10.5% 628623 ? 5% numa-vmstat.node0.nr_file_pages
16548 ? 30% +239.6% 56197 ? 46% numa-vmstat.node0.nr_mapped
523.67 ? 35% +55.0% 811.67 ? 17% numa-vmstat.node0.nr_page_table_pages
36972 ? 26% +53.5% 56757 ? 5% numa-vmstat.node0.nr_slab_unreclaimable
193900 ? 8% +319.4% 813228 ? 37% numa-vmstat.node1.nr_shmem
5677672 ? 13% +403.0% 28559515 ? 5% turbostat.C1
0.10 ? 26% +0.5 0.64 ? 7% turbostat.C1%
0.68 ? 2% +97.8% 1.34 ? 3% turbostat.CPU%c1
5.956e+08 ? 3% -56.6% 2.585e+08 ? 5% turbostat.IRQ
118.32 +12.9% 133.57 turbostat.RAMWatt
3177467 +81.9% 5780159 ? 20% meminfo.Cached
1203614 +222.4% 3880986 ? 31% meminfo.Committed_AS
4677171 +64.7% 7701539 ? 17% meminfo.Memused
297339 +24.9% 371485 meminfo.SUnreclaim
806445 +323.2% 3412731 ? 35% meminfo.Shmem
420575 +19.0% 500317 meminfo.Slab
5215936 +51.2% 7884576 ? 14% meminfo.max_used_kB
2275408 ? 3% +10.5% 2514228 ? 5% numa-meminfo.node0.FilePages
64515 ? 31% +247.9% 224420 ? 46% numa-meminfo.node0.Mapped
3161940 ? 5% +18.3% 3741976 ? 3% numa-meminfo.node0.MemUsed
2096 ? 35% +55.0% 3249 ? 17% numa-meminfo.node0.PageTables
148152 ? 26% +53.3% 227063 ? 5% numa-meminfo.node0.SUnreclaim
224798 ? 18% +37.9% 309991 ? 5% numa-meminfo.node0.Slab
774062 ? 8% +320.0% 3251142 ? 37% numa-meminfo.node1.Shmem
4772460 -1.6% 4696954 proc-vmstat.nr_dirty_background_threshold
9556589 -1.6% 9405393 proc-vmstat.nr_dirty_threshold
794697 +81.8% 1444609 ? 20% proc-vmstat.nr_file_pages
48011598 -1.6% 47256315 proc-vmstat.nr_free_pages
201937 +322.3% 852751 ? 35% proc-vmstat.nr_shmem
30809 +4.5% 32203 proc-vmstat.nr_slab_reclaimable
74338 +24.9% 92861 proc-vmstat.nr_slab_unreclaimable
3223037 ? 2% -20.0% 2579246 ? 17% proc-vmstat.numa_hit
3128922 ? 2% -20.6% 2485141 ? 18% proc-vmstat.numa_local
6652818 ? 3% -50.8% 3274182 ? 18% proc-vmstat.pgalloc_normal
6372725 ? 3% -68.5% 2009197 ? 11% proc-vmstat.pgfree
78108 -13.4% 67635 ? 4% proc-vmstat.pgreuse
64709 ? 4% -8.6% 59163 ? 5% slabinfo.anon_vma_chain.active_objs
1012 ? 4% -8.5% 926.50 ? 5% slabinfo.anon_vma_chain.active_slabs
64825 ? 4% -8.5% 59329 ? 5% slabinfo.anon_vma_chain.num_objs
1012 ? 4% -8.5% 926.50 ? 5% slabinfo.anon_vma_chain.num_slabs
766.50 +17.5% 900.50 slabinfo.kmalloc-256.active_slabs
24539 +17.5% 28832 slabinfo.kmalloc-256.num_objs
766.50 +17.5% 900.50 slabinfo.kmalloc-256.num_slabs
39056 ? 3% +181.6% 109997 ? 3% slabinfo.kmalloc-512.active_objs
1397 ? 3% +217.3% 4434 ? 4% slabinfo.kmalloc-512.active_slabs
44743 ? 3% +217.2% 141914 ? 4% slabinfo.kmalloc-512.num_objs
1397 ? 3% +217.3% 4434 ? 4% slabinfo.kmalloc-512.num_slabs
31672 +31.2% 41555 ? 11% slabinfo.radix_tree_node.active_objs
565.17 +31.2% 741.67 ? 11% slabinfo.radix_tree_node.active_slabs
31677 +31.2% 41555 ? 11% slabinfo.radix_tree_node.num_objs
565.17 +31.2% 741.67 ? 11% slabinfo.radix_tree_node.num_slabs
30624 ? 3% +229.8% 101008 ? 3% slabinfo.skbuff_head_cache.active_objs
1117 ? 4% +267.9% 4111 ? 4% slabinfo.skbuff_head_cache.active_slabs
35782 ? 4% +267.8% 131591 ? 4% slabinfo.skbuff_head_cache.num_objs
1117 ? 4% +267.9% 4111 ? 4% slabinfo.skbuff_head_cache.num_slabs
12.17 ? 59% -76.6% 2.85 ?105% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg
107.59 ? 26% -86.3% 14.72 ? 87% perf-sched.sch_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
12.08 ? 10% -80.9% 2.30 ?127% perf-sched.sch_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
11216 ? 19% -53.8% 5178 ? 79% perf-sched.sch_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
12780 ? 13% -58.6% 5291 ? 82% perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
13649 ? 12% -49.4% 6904 ? 79% perf-sched.total_wait_time.max.ms
95.29 ? 22% -66.6% 31.87 ? 85% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
509.93 ? 20% -88.0% 61.07 ? 83% perf-sched.wait_and_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
43.59 ? 11% -80.4% 8.55 ?127% perf-sched.wait_and_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
23259 ? 15% -55.3% 10391 ? 79% perf-sched.wait_and_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
25565 ? 13% -57.7% 10809 ? 83% perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
177.56 ? 57% -70.7% 52.07 ? 81% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
35.59 ? 19% -74.3% 9.13 ? 94% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
36.21 ? 47% -82.1% 6.50 ? 99% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg
95.29 ? 22% -66.6% 31.87 ? 85% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
402.34 ? 20% -88.5% 46.35 ? 82% perf-sched.wait_time.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
31.51 ? 12% -80.2% 6.24 ?126% perf-sched.wait_time.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
13649 ? 12% -55.2% 6120 ? 83% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
12457 ? 15% -53.4% 5811 ? 84% perf-sched.wait_time.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
13018 ? 14% -57.3% 5565 ? 83% perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
11.68 -4.0% 11.22 ? 3% perf-stat.i.MPKI
1.91e+10 ? 2% -13.0% 1.662e+10 ? 4% perf-stat.i.branch-instructions
3.114e+08 -13.3% 2.7e+08 ? 4% perf-stat.i.branch-misses
6.20 +9.1 15.31 ? 4% perf-stat.i.cache-miss-rate%
63695030 ? 4% +112.1% 1.351e+08 ? 4% perf-stat.i.cache-misses
1.062e+09 ? 3% -13.9% 9.139e+08 ? 8% perf-stat.i.cache-references
9804802 ? 2% -72.2% 2721599 ? 18% perf-stat.i.context-switches
3.20 ? 2% +12.1% 3.59 ? 4% perf-stat.i.cpi
682632 ? 6% -64.6% 241483 ? 19% perf-stat.i.cpu-migrations
5079 ? 4% -57.8% 2145 ? 4% perf-stat.i.cycles-between-cache-misses
0.08 ? 2% +0.1 0.15 ? 4% perf-stat.i.dTLB-load-miss-rate%
19602583 +76.8% 34648829 ? 3% perf-stat.i.dTLB-load-misses
2.642e+10 ? 2% -10.9% 2.353e+10 ? 4% perf-stat.i.dTLB-loads
0.01 ? 8% +0.0 0.04 ? 10% perf-stat.i.dTLB-store-miss-rate%
886982 ? 8% +468.7% 5044093 ? 9% perf-stat.i.dTLB-store-misses
1.612e+10 -8.9% 1.468e+10 ? 4% perf-stat.i.dTLB-stores
71.40 +12.4 83.83 perf-stat.i.iTLB-load-miss-rate%
11045291 ? 6% -52.6% 5234559 ? 10% perf-stat.i.iTLB-loads
9.29e+10 ? 2% -11.1% 8.26e+10 ? 4% perf-stat.i.instructions
3720 -11.7% 3283 ? 2% perf-stat.i.instructions-per-iTLB-miss
0.32 ? 2% -9.9% 0.29 ? 4% perf-stat.i.ipc
532.69 ? 5% +33.9% 713.49 ? 5% perf-stat.i.metric.K/sec
602.85 ? 2% -11.1% 535.87 ? 4% perf-stat.i.metric.M/sec
2939 ? 3% +60.2% 4710 ? 8% perf-stat.i.minor-faults
87.86 -38.2 49.64 ? 6% perf-stat.i.node-load-miss-rate%
2480788 ? 7% +578.9% 16841829 ? 4% perf-stat.i.node-loads
91.60 -43.7 47.94 ? 6% perf-stat.i.node-store-miss-rate%
10368046 ? 6% +17.6% 12189032 ? 6% perf-stat.i.node-store-misses
965185 ? 7% +1354.2% 14035704 ? 4% perf-stat.i.node-stores
2941 ? 3% +60.2% 4712 ? 8% perf-stat.i.page-faults
6.05 +8.9 14.91 ? 5% perf-stat.overall.cache-miss-rate%
3.11 ? 2% +11.6% 3.47 ? 5% perf-stat.overall.cpi
4511 ? 3% -53.1% 2113 ? 4% perf-stat.overall.cycles-between-cache-misses
0.07 ? 2% +0.1 0.15 ? 4% perf-stat.overall.dTLB-load-miss-rate%
0.01 ? 9% +0.0 0.03 ? 10% perf-stat.overall.dTLB-store-miss-rate%
70.35 +13.2 83.55 perf-stat.overall.iTLB-load-miss-rate%
3571 ? 2% -12.3% 3131 ? 2% perf-stat.overall.instructions-per-iTLB-miss
0.32 ? 2% -10.2% 0.29 ? 4% perf-stat.overall.ipc
89.09 -38.4 50.67 ? 8% perf-stat.overall.node-load-miss-rate%
90.82 -44.9 45.92 ? 5% perf-stat.overall.node-store-miss-rate%
1.896e+10 ? 2% -12.8% 1.654e+10 ? 4% perf-stat.ps.branch-instructions
3.094e+08 -13.1% 2.688e+08 ? 4% perf-stat.ps.branch-misses
63654107 ? 3% +111.9% 1.349e+08 ? 4% perf-stat.ps.cache-misses
1.053e+09 ? 3% -13.8% 9.081e+08 ? 7% perf-stat.ps.cache-references
9699719 ? 2% -72.4% 2679869 ? 18% perf-stat.ps.context-switches
670636 ? 6% -65.0% 234729 ? 18% perf-stat.ps.cpu-migrations
19593701 +76.7% 34623370 ? 3% perf-stat.ps.dTLB-load-misses
2.624e+10 ? 2% -10.7% 2.342e+10 ? 4% perf-stat.ps.dTLB-loads
901124 ? 9% +459.7% 5043349 ? 9% perf-stat.ps.dTLB-store-misses
1.601e+10 -8.7% 1.462e+10 ? 4% perf-stat.ps.dTLB-stores
10905261 ? 6% -52.5% 5180101 ? 10% perf-stat.ps.iTLB-loads
9.225e+10 ? 2% -10.9% 8.219e+10 ? 4% perf-stat.ps.instructions
1.12 ? 44% +99.7% 2.24 ? 25% perf-stat.ps.major-faults
2910 ? 3% +56.2% 4547 ? 8% perf-stat.ps.minor-faults
2537682 ? 7% +568.1% 16953030 ? 4% perf-stat.ps.node-loads
10280989 ? 6% +16.7% 11999128 ? 6% perf-stat.ps.node-store-misses
1033899 ? 7% +1266.0% 14122866 ? 4% perf-stat.ps.node-stores
2912 ? 3% +56.2% 4550 ? 8% perf-stat.ps.page-faults
5.555e+13 ? 2% -36.9% 3.505e+13 ? 2% perf-stat.total.instructions
82383 -38.1% 51010 ? 2% softirqs.CPU0.RCU
11946 ? 8% +51.9% 18143 ? 6% softirqs.CPU0.SCHED
83901 -39.4% 50854 ? 2% softirqs.CPU1.RCU
10530 ? 9% +62.6% 17122 ? 5% softirqs.CPU1.SCHED
82690 -40.3% 49378 ? 2% softirqs.CPU10.RCU
8595 ? 5% +68.9% 14520 ? 4% softirqs.CPU10.SCHED
81004 -39.4% 49126 ? 3% softirqs.CPU100.RCU
10023 ? 14% +60.5% 16092 ? 5% softirqs.CPU100.SCHED
80721 -39.7% 48692 ? 3% softirqs.CPU101.RCU
9652 ? 12% +67.1% 16127 ? 3% softirqs.CPU101.SCHED
81743 -39.5% 49494 ? 4% softirqs.CPU102.RCU
9461 ? 12% +68.5% 15946 ? 4% softirqs.CPU102.SCHED
81567 -39.9% 49048 ? 4% softirqs.CPU103.RCU
10377 ? 13% +57.1% 16306 ? 3% softirqs.CPU103.SCHED
83547 -39.9% 50187 ? 3% softirqs.CPU11.RCU
8718 ? 7% +67.4% 14598 ? 4% softirqs.CPU11.SCHED
83780 -39.7% 50514 ? 3% softirqs.CPU12.RCU
8551 ? 7% +69.7% 14515 ? 5% softirqs.CPU12.SCHED
82874 ? 2% -37.9% 51480 ? 3% softirqs.CPU13.RCU
8707 ? 7% +74.3% 15179 ? 5% softirqs.CPU13.SCHED
79623 ? 9% -37.1% 50081 ? 4% softirqs.CPU14.RCU
8757 ? 8% +62.8% 14260 ? 5% softirqs.CPU14.SCHED
91367 -40.7% 54191 ? 5% softirqs.CPU15.RCU
8433 ? 6% +69.1% 14260 ? 5% softirqs.CPU15.SCHED
91569 -41.1% 53944 ? 3% softirqs.CPU16.RCU
8422 ? 5% +75.1% 14746 ? 5% softirqs.CPU16.SCHED
90429 ? 5% -40.0% 54232 ? 2% softirqs.CPU17.RCU
8511 ? 7% +69.8% 14452 ? 3% softirqs.CPU17.SCHED
92542 -41.6% 54049 ? 3% softirqs.CPU18.RCU
8609 ? 9% +64.8% 14191 ? 5% softirqs.CPU18.SCHED
93207 -42.0% 54033 ? 3% softirqs.CPU19.RCU
8866 ? 6% +59.6% 14153 ? 4% softirqs.CPU19.SCHED
83467 -39.0% 50951 ? 3% softirqs.CPU2.RCU
9144 ? 7% +63.6% 14964 ? 5% softirqs.CPU2.SCHED
90907 -41.1% 53543 ? 3% softirqs.CPU20.RCU
8915 ? 10% +63.0% 14530 ? 4% softirqs.CPU20.SCHED
91022 -40.9% 53798 ? 3% softirqs.CPU21.RCU
8302 ? 4% +73.1% 14372 ? 4% softirqs.CPU21.SCHED
90088 ? 4% -40.4% 53699 ? 3% softirqs.CPU22.RCU
8682 ? 5% +64.6% 14291 ? 4% softirqs.CPU22.SCHED
91532 -40.7% 54267 ? 3% softirqs.CPU23.RCU
8545 ? 6% +72.0% 14696 ? 5% softirqs.CPU23.SCHED
92532 -41.1% 54495 ? 3% softirqs.CPU24.RCU
8682 ? 7% +68.9% 14661 ? 4% softirqs.CPU24.SCHED
90745 ? 6% -40.1% 54374 ? 3% softirqs.CPU25.RCU
8656 ? 6% +67.3% 14479 ? 5% softirqs.CPU25.SCHED
80912 -41.1% 47680 ? 3% softirqs.CPU26.RCU
9647 ? 12% +63.0% 15726 ? 8% softirqs.CPU26.SCHED
81056 -40.1% 48515 ? 2% softirqs.CPU27.RCU
10080 ? 12% +70.4% 17182 ? 8% softirqs.CPU27.SCHED
80849 -40.3% 48288 ? 4% softirqs.CPU28.RCU
9318 ? 16% +73.8% 16195 ? 4% softirqs.CPU28.SCHED
81348 -40.3% 48602 ? 3% softirqs.CPU29.RCU
9837 ? 9% +61.9% 15923 ? 3% softirqs.CPU29.SCHED
82656 ? 2% -38.1% 51137 ? 4% softirqs.CPU3.RCU
8719 ? 5% +70.4% 14859 ? 4% softirqs.CPU3.SCHED
83915 -38.2% 51833 ? 5% softirqs.CPU30.RCU
9430 ? 9% +70.3% 16062 ? 5% softirqs.CPU30.SCHED
84800 ? 2% -40.3% 50624 ? 3% softirqs.CPU31.RCU
9622 ? 12% +67.9% 16158 ? 6% softirqs.CPU31.SCHED
81832 ? 8% -38.0% 50745 ? 3% softirqs.CPU32.RCU
9722 ? 10% +66.6% 16202 ? 5% softirqs.CPU32.SCHED
83675 -39.7% 50422 ? 3% softirqs.CPU33.RCU
9401 ? 11% +72.2% 16187 ? 5% softirqs.CPU33.SCHED
84672 -40.0% 50793 ? 3% softirqs.CPU34.RCU
9561 ? 17% +65.6% 15831 ? 5% softirqs.CPU34.SCHED
84570 -40.1% 50664 ? 3% softirqs.CPU35.RCU
9508 ? 12% +68.9% 16065 ? 3% softirqs.CPU35.SCHED
84118 -39.4% 50963 ? 3% softirqs.CPU36.RCU
9782 ? 16% +62.9% 15935 ? 6% softirqs.CPU36.SCHED
85110 -40.0% 51085 ? 3% softirqs.CPU37.RCU
9587 ? 15% +65.5% 15866 ? 5% softirqs.CPU37.SCHED
85245 ? 2% -40.1% 51046 ? 3% softirqs.CPU38.RCU
9434 ? 13% +69.1% 15955 ? 4% softirqs.CPU38.SCHED
84202 -39.8% 50729 ? 3% softirqs.CPU39.RCU
9671 ? 11% +63.8% 15841 ? 5% softirqs.CPU39.SCHED
83756 ? 3% -40.1% 50200 ? 3% softirqs.CPU4.RCU
8758 ? 6% +67.9% 14703 ? 5% softirqs.CPU4.SCHED
84595 -40.2% 50626 ? 3% softirqs.CPU40.RCU
9627 ? 15% +62.6% 15651 ? 7% softirqs.CPU40.SCHED
83591 -39.5% 50583 ? 3% softirqs.CPU41.RCU
8874 ? 8% +80.0% 15972 ? 6% softirqs.CPU41.SCHED
84667 ? 2% -39.9% 50851 ? 3% softirqs.CPU42.RCU
9534 ? 17% +66.2% 15850 ? 5% softirqs.CPU42.SCHED
85428 ? 2% -40.2% 51057 ? 3% softirqs.CPU43.RCU
9135 ? 9% +75.2% 16002 ? 4% softirqs.CPU43.SCHED
84516 ? 2% -40.1% 50629 ? 3% softirqs.CPU44.RCU
9459 ? 13% +68.8% 15971 ? 4% softirqs.CPU44.SCHED
85668 -40.8% 50691 ? 3% softirqs.CPU45.RCU
9758 ? 19% +62.6% 15862 ? 6% softirqs.CPU45.SCHED
84519 -40.4% 50333 ? 3% softirqs.CPU46.RCU
9430 ? 11% +69.3% 15970 ? 4% softirqs.CPU46.SCHED
84197 -40.3% 50268 ? 3% softirqs.CPU47.RCU
9879 ? 18% +61.5% 15951 ? 5% softirqs.CPU47.SCHED
84157 -40.3% 50225 ? 3% softirqs.CPU48.RCU
9486 ? 15% +70.4% 16166 ? 4% softirqs.CPU48.SCHED
84677 -40.9% 50082 ? 4% softirqs.CPU49.RCU
9683 ? 12% +63.1% 15789 ? 5% softirqs.CPU49.SCHED
83127 -39.5% 50295 ? 3% softirqs.CPU5.RCU
8662 ? 6% +68.8% 14625 ? 6% softirqs.CPU5.SCHED
84627 -40.4% 50461 ? 3% softirqs.CPU50.RCU
9172 ? 11% +72.1% 15788 ? 3% softirqs.CPU50.SCHED
85517 -41.2% 50256 ? 3% softirqs.CPU51.RCU
10235 ? 18% +57.5% 16117 ? 5% softirqs.CPU51.SCHED
89582 ? 2% -41.6% 52277 ? 4% softirqs.CPU52.RCU
8268 ? 5% +69.3% 13998 ? 5% softirqs.CPU52.SCHED
91542 ? 2% -42.7% 52468 ? 3% softirqs.CPU53.RCU
8564 ? 4% +67.9% 14382 ? 5% softirqs.CPU53.SCHED
91896 ? 2% -42.8% 52597 ? 4% softirqs.CPU54.RCU
7566 ? 4% +77.8% 13452 ? 8% softirqs.CPU54.SCHED
91657 ? 3% -41.9% 53221 ? 4% softirqs.CPU55.RCU
8339 ? 5% +72.7% 14402 ? 4% softirqs.CPU55.SCHED
92388 ? 2% -42.2% 53390 ? 2% softirqs.CPU56.RCU
8593 ? 5% +67.9% 14426 ? 4% softirqs.CPU56.SCHED
91852 -41.7% 53505 ? 2% softirqs.CPU57.RCU
8502 ? 5% +71.3% 14568 ? 4% softirqs.CPU57.SCHED
93510 ? 2% -42.8% 53479 ? 2% softirqs.CPU58.RCU
8769 ? 7% +66.6% 14611 ? 4% softirqs.CPU58.SCHED
87046 ? 10% -39.3% 52836 ? 3% softirqs.CPU59.RCU
8785 ? 9% +63.6% 14375 ? 5% softirqs.CPU59.SCHED
84834 ? 2% -40.3% 50626 ? 3% softirqs.CPU6.RCU
8858 ? 6% +65.3% 14639 ? 5% softirqs.CPU6.SCHED
83069 -39.6% 50157 ? 3% softirqs.CPU60.RCU
8628 ? 7% +69.9% 14661 ? 5% softirqs.CPU60.SCHED
83152 -40.1% 49776 ? 3% softirqs.CPU61.RCU
8854 ? 10% +63.8% 14501 ? 6% softirqs.CPU61.SCHED
82984 -40.1% 49686 ? 2% softirqs.CPU62.RCU
8622 ? 5% +71.5% 14785 ? 5% softirqs.CPU62.SCHED
84589 -40.4% 50403 ? 2% softirqs.CPU63.RCU
8722 ? 7% +67.0% 14565 ? 4% softirqs.CPU63.SCHED
84360 -40.3% 50355 ? 3% softirqs.CPU64.RCU
8807 ? 6% +61.3% 14203 ? 4% softirqs.CPU64.SCHED
82988 -39.5% 50218 ? 3% softirqs.CPU65.RCU
8580 ? 5% +69.1% 14506 ? 5% softirqs.CPU65.SCHED
80761 ? 7% -37.3% 50666 ? 3% softirqs.CPU66.RCU
8602 ? 5% +71.9% 14789 ? 4% softirqs.CPU66.SCHED
82949 -39.5% 50221 ? 3% softirqs.CPU67.RCU
8626 ? 6% +71.1% 14762 ? 5% softirqs.CPU67.SCHED
83358 -39.0% 50882 ? 3% softirqs.CPU68.RCU
8559 ? 6% +72.7% 14784 ? 3% softirqs.CPU68.SCHED
80823 ? 9% -37.2% 50754 ? 3% softirqs.CPU69.RCU
8981 ? 12% +63.6% 14693 ? 4% softirqs.CPU69.SCHED
81901 ? 7% -38.3% 50504 ? 4% softirqs.CPU7.RCU
9028 ? 6% +59.1% 14360 ? 6% softirqs.CPU7.SCHED
84749 ? 2% -39.9% 50898 ? 3% softirqs.CPU70.RCU
8761 ? 7% +67.9% 14707 ? 4% softirqs.CPU70.SCHED
84493 ? 2% -39.7% 50972 ? 3% softirqs.CPU71.RCU
8746 ? 6% +66.0% 14519 ? 6% softirqs.CPU71.SCHED
83345 -39.6% 50312 ? 3% softirqs.CPU72.RCU
8936 ? 12% +63.5% 14610 ? 5% softirqs.CPU72.SCHED
83229 -39.0% 50755 ? 3% softirqs.CPU73.RCU
8482 ? 5% +77.3% 15042 ? 5% softirqs.CPU73.SCHED
79942 ? 9% -37.1% 50285 ? 3% softirqs.CPU74.RCU
8981 ? 7% +64.1% 14740 ? 5% softirqs.CPU74.SCHED
83686 ? 2% -41.2% 49243 ? 3% softirqs.CPU75.RCU
8671 ? 5% +71.8% 14901 ? 5% softirqs.CPU75.SCHED
84048 ? 2% -41.8% 48892 ? 4% softirqs.CPU76.RCU
9352 ? 8% +57.0% 14684 ? 3% softirqs.CPU76.SCHED
80865 ? 7% -39.9% 48572 ? 3% softirqs.CPU77.RCU
8789 ? 4% +69.0% 14853 ? 4% softirqs.CPU77.SCHED
87858 -41.6% 51306 ? 3% softirqs.CPU78.RCU
9232 ? 9% +69.4% 15642 ? 8% softirqs.CPU78.SCHED
88237 -40.7% 52303 ? 3% softirqs.CPU79.RCU
10236 ? 12% +61.0% 16483 ? 9% softirqs.CPU79.SCHED
82490 ? 2% -39.1% 50211 ? 3% softirqs.CPU8.RCU
8488 ? 5% +69.0% 14344 ? 6% softirqs.CPU8.SCHED
88590 -41.2% 52132 ? 4% softirqs.CPU80.RCU
10327 ? 10% +53.8% 15886 ? 5% softirqs.CPU80.SCHED
87957 -40.4% 52465 ? 3% softirqs.CPU81.RCU
9534 ? 13% +73.0% 16494 ? 7% softirqs.CPU81.SCHED
87951 -40.6% 52263 ? 3% softirqs.CPU82.RCU
9557 ? 10% +71.7% 16409 ? 7% softirqs.CPU82.SCHED
87989 ? 2% -40.7% 52154 ? 3% softirqs.CPU83.RCU
10051 ? 23% +59.6% 16040 ? 5% softirqs.CPU83.SCHED
87013 ? 4% -40.0% 52210 ? 3% softirqs.CPU84.RCU
10147 ? 13% +55.9% 15817 ? 6% softirqs.CPU84.SCHED
87829 -40.7% 52106 ? 3% softirqs.CPU85.RCU
9448 ? 11% +69.9% 16054 ? 4% softirqs.CPU85.SCHED
88170 -40.3% 52637 ? 3% softirqs.CPU86.RCU
9309 ? 11% +73.6% 16158 ? 5% softirqs.CPU86.SCHED
88373 -40.9% 52219 ? 3% softirqs.CPU87.RCU
9834 ? 12% +63.0% 16033 ? 3% softirqs.CPU87.SCHED
87687 -40.3% 52361 ? 3% softirqs.CPU88.RCU
9885 ? 14% +60.9% 15907 ? 7% softirqs.CPU88.SCHED
88035 -40.9% 52034 ? 3% softirqs.CPU89.RCU
9647 ? 13% +65.9% 16003 ? 5% softirqs.CPU89.SCHED
82978 ? 2% -40.2% 49601 ? 3% softirqs.CPU9.RCU
8802 ? 7% +62.1% 14265 ? 5% softirqs.CPU9.SCHED
82405 -40.1% 49331 ? 2% softirqs.CPU90.RCU
9559 ? 14% +67.0% 15961 ? 5% softirqs.CPU90.SCHED
81438 -39.6% 49209 ? 3% softirqs.CPU91.RCU
9803 ? 11% +64.1% 16091 ? 7% softirqs.CPU91.SCHED
81721 -40.1% 48978 ? 3% softirqs.CPU92.RCU
9665 ? 13% +63.8% 15828 ? 6% softirqs.CPU92.SCHED
81370 -39.7% 49067 ? 3% softirqs.CPU93.RCU
9448 ? 10% +67.2% 15794 ? 6% softirqs.CPU93.SCHED
81624 -39.9% 49065 ? 3% softirqs.CPU94.RCU
9737 ? 16% +64.4% 16010 ? 6% softirqs.CPU94.SCHED
81770 -39.6% 49413 ? 3% softirqs.CPU95.RCU
9066 ? 10% +83.2% 16607 ? 9% softirqs.CPU95.SCHED
81782 -39.9% 49184 ? 3% softirqs.CPU96.RCU
9772 ? 14% +61.6% 15792 ? 4% softirqs.CPU96.SCHED
82155 -40.0% 49329 ? 3% softirqs.CPU97.RCU
9878 ? 18% +59.8% 15789 ? 6% softirqs.CPU97.SCHED
81298 -39.6% 49144 ? 3% softirqs.CPU98.RCU
9705 ? 14% +63.7% 15889 ? 5% softirqs.CPU98.SCHED
81242 -39.4% 49224 ? 3% softirqs.CPU99.RCU
9817 ? 14% +64.9% 16184 ? 5% softirqs.CPU99.SCHED
8852871 -40.1% 5301088 ? 3% softirqs.RCU
958083 ? 4% +66.6% 1596070 ? 3% softirqs.SCHED
92537 -30.2% 64615 ? 2% softirqs.TIMER
1205 -29.0% 856.00 ? 2% interrupts.9:IO-APIC.9-fasteoi.acpi
89891349 ? 18% -60.7% 35361947 ? 25% interrupts.CAL:Function_call_interrupts
1623974 ? 57% -71.7% 459021 ? 84% interrupts.CPU0.CAL:Function_call_interrupts
1205651 -29.1% 855249 ? 2% interrupts.CPU0.LOC:Local_timer_interrupts
3196764 ? 7% -66.4% 1072530 ? 6% interrupts.CPU0.RES:Rescheduling_interrupts
1205 -29.0% 856.00 ? 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
1205641 -29.1% 855146 ? 2% interrupts.CPU1.LOC:Local_timer_interrupts
3425943 ? 6% -69.8% 1033843 ? 10% interrupts.CPU1.RES:Rescheduling_interrupts
707756 ? 71% -59.9% 283504 ? 29% interrupts.CPU10.CAL:Function_call_interrupts
1205637 -29.1% 855186 ? 2% interrupts.CPU10.LOC:Local_timer_interrupts
3586704 ? 2% -70.0% 1076909 ? 6% interrupts.CPU10.RES:Rescheduling_interrupts
1205570 -29.1% 855284 ? 2% interrupts.CPU100.LOC:Local_timer_interrupts
3642188 ? 10% -68.5% 1146714 ? 16% interrupts.CPU100.RES:Rescheduling_interrupts
1220194 ? 53% -66.3% 411230 ? 58% interrupts.CPU101.CAL:Function_call_interrupts
1205569 -29.1% 855310 ? 2% interrupts.CPU101.LOC:Local_timer_interrupts
3686422 ? 9% -69.4% 1127680 ? 8% interrupts.CPU101.RES:Rescheduling_interrupts
1205539 -29.1% 855303 ? 2% interrupts.CPU102.LOC:Local_timer_interrupts
3466868 ? 9% -66.0% 1178496 ? 9% interrupts.CPU102.RES:Rescheduling_interrupts
753033 ? 38% -51.9% 362540 ? 39% interrupts.CPU103.CAL:Function_call_interrupts
1205534 -29.1% 855323 ? 2% interrupts.CPU103.LOC:Local_timer_interrupts
3450153 ? 8% -68.4% 1088970 ? 6% interrupts.CPU103.RES:Rescheduling_interrupts
724554 ? 60% -62.5% 271669 ? 33% interrupts.CPU11.CAL:Function_call_interrupts
1205604 -29.1% 855122 ? 2% interrupts.CPU11.LOC:Local_timer_interrupts
3400360 ? 6% -68.8% 1061657 ? 15% interrupts.CPU11.RES:Rescheduling_interrupts
641236 ? 59% -61.2% 248758 ? 32% interrupts.CPU12.CAL:Function_call_interrupts
1205609 -29.1% 855116 ? 2% interrupts.CPU12.LOC:Local_timer_interrupts
3561488 ? 5% -70.5% 1050780 ? 8% interrupts.CPU12.RES:Rescheduling_interrupts
734040 ? 70% -63.5% 268243 ? 31% interrupts.CPU13.CAL:Function_call_interrupts
1205642 -29.1% 855148 ? 2% interrupts.CPU13.LOC:Local_timer_interrupts
3308551 ? 8% -66.5% 1109093 ? 8% interrupts.CPU13.RES:Rescheduling_interrupts
785202 ? 64% -67.1% 258319 ? 29% interrupts.CPU14.CAL:Function_call_interrupts
1205608 -29.1% 855174 ? 2% interrupts.CPU14.LOC:Local_timer_interrupts
3615978 ? 7% -70.6% 1063276 ? 8% interrupts.CPU14.RES:Rescheduling_interrupts
757597 ? 64% -67.7% 244389 ? 16% interrupts.CPU15.CAL:Function_call_interrupts
1205614 -29.1% 855152 ? 2% interrupts.CPU15.LOC:Local_timer_interrupts
3343478 ? 4% -66.8% 1110832 ? 11% interrupts.CPU15.RES:Rescheduling_interrupts
505614 ? 44% -54.3% 230896 ? 17% interrupts.CPU16.CAL:Function_call_interrupts
1205612 -29.1% 855161 ? 2% interrupts.CPU16.LOC:Local_timer_interrupts
3677516 ? 9% -74.0% 957228 ? 4% interrupts.CPU16.RES:Rescheduling_interrupts
756708 ? 52% -70.5% 223318 ? 17% interrupts.CPU17.CAL:Function_call_interrupts
1205608 -29.1% 855129 ? 2% interrupts.CPU17.LOC:Local_timer_interrupts
3411368 ? 3% -70.5% 1005739 ? 9% interrupts.CPU17.RES:Rescheduling_interrupts
650745 ? 48% -64.0% 234382 ? 15% interrupts.CPU18.CAL:Function_call_interrupts
1205588 -29.1% 855144 ? 2% interrupts.CPU18.LOC:Local_timer_interrupts
3489665 ? 6% -67.4% 1137268 ? 20% interrupts.CPU18.RES:Rescheduling_interrupts
1205594 -29.1% 855172 ? 2% interrupts.CPU19.LOC:Local_timer_interrupts
3184389 ? 4% -67.2% 1044787 ? 12% interrupts.CPU19.RES:Rescheduling_interrupts
1205645 -29.1% 855163 ? 2% interrupts.CPU2.LOC:Local_timer_interrupts
3344801 ? 5% -67.3% 1094443 ? 6% interrupts.CPU2.RES:Rescheduling_interrupts
718386 ? 67% -63.5% 262499 ? 19% interrupts.CPU20.CAL:Function_call_interrupts
1205599 -29.1% 855188 ? 2% interrupts.CPU20.LOC:Local_timer_interrupts
3403935 ? 6% -67.7% 1100497 ? 8% interrupts.CPU20.RES:Rescheduling_interrupts
1205604 -29.1% 855136 ? 2% interrupts.CPU21.LOC:Local_timer_interrupts
3497519 ? 8% -69.7% 1060157 ? 11% interrupts.CPU21.RES:Rescheduling_interrupts
775676 ? 52% -55.6% 344109 ? 51% interrupts.CPU22.CAL:Function_call_interrupts
1205655 -29.1% 855162 ? 2% interrupts.CPU22.LOC:Local_timer_interrupts
3455529 ? 6% -67.9% 1108198 ? 10% interrupts.CPU22.RES:Rescheduling_interrupts
705272 ? 56% -62.7% 262796 ? 21% interrupts.CPU23.CAL:Function_call_interrupts
1205628 -29.1% 855179 ? 2% interrupts.CPU23.LOC:Local_timer_interrupts
3677020 ? 9% -71.0% 1066870 ? 6% interrupts.CPU23.RES:Rescheduling_interrupts
1205621 -29.1% 855116 ? 2% interrupts.CPU24.LOC:Local_timer_interrupts
3311684 ? 6% -69.2% 1020747 ? 3% interrupts.CPU24.RES:Rescheduling_interrupts
656460 ? 67% -65.4% 227021 ? 16% interrupts.CPU25.CAL:Function_call_interrupts
1205631 -29.1% 855144 ? 2% interrupts.CPU25.LOC:Local_timer_interrupts
3232461 ? 7% -68.0% 1034332 ? 13% interrupts.CPU25.RES:Rescheduling_interrupts
1205541 -29.1% 855326 ? 2% interrupts.CPU26.LOC:Local_timer_interrupts
2889950 ? 15% -61.4% 1115390 ? 6% interrupts.CPU26.RES:Rescheduling_interrupts
1026174 ? 48% -69.4% 314389 ? 32% interrupts.CPU27.CAL:Function_call_interrupts
1205558 -29.1% 855282 ? 2% interrupts.CPU27.LOC:Local_timer_interrupts
3593896 ? 7% -69.9% 1081988 ? 7% interrupts.CPU27.RES:Rescheduling_interrupts
984752 ? 53% -64.3% 351367 ? 48% interrupts.CPU28.CAL:Function_call_interrupts
1205555 -29.1% 855306 ? 2% interrupts.CPU28.LOC:Local_timer_interrupts
3424262 ? 10% -67.1% 1127593 ? 9% interrupts.CPU28.RES:Rescheduling_interrupts
1205576 -29.1% 855297 ? 2% interrupts.CPU29.LOC:Local_timer_interrupts
3512273 ? 8% -68.6% 1104147 ? 12% interrupts.CPU29.RES:Rescheduling_interrupts
669897 ? 54% -58.9% 275239 ? 27% interrupts.CPU3.CAL:Function_call_interrupts
1205627 -29.1% 855178 ? 2% interrupts.CPU3.LOC:Local_timer_interrupts
3595156 ? 5% -69.9% 1082314 ? 8% interrupts.CPU3.RES:Rescheduling_interrupts
1004398 ? 38% -58.3% 418341 ? 60% interrupts.CPU30.CAL:Function_call_interrupts
1205681 -29.1% 855320 ? 2% interrupts.CPU30.LOC:Local_timer_interrupts
3499045 ? 2% -68.5% 1101860 ? 9% interrupts.CPU30.RES:Rescheduling_interrupts
1112178 ? 39% -67.4% 362178 ? 36% interrupts.CPU31.CAL:Function_call_interrupts
1205626 -29.1% 855320 ? 2% interrupts.CPU31.LOC:Local_timer_interrupts
3329087 ? 4% -65.4% 1153373 ? 7% interrupts.CPU31.RES:Rescheduling_interrupts
1205545 -29.0% 855336 ? 2% interrupts.CPU32.LOC:Local_timer_interrupts
3613423 ? 7% -67.6% 1169236 ? 11% interrupts.CPU32.RES:Rescheduling_interrupts
1205644 -29.1% 855326 ? 2% interrupts.CPU33.LOC:Local_timer_interrupts
3581469 ? 8% -68.2% 1140463 ? 10% interrupts.CPU33.RES:Rescheduling_interrupts
1205560 -29.1% 855270 ? 2% interrupts.CPU34.LOC:Local_timer_interrupts
3470665 ? 4% -65.9% 1182794 ? 12% interrupts.CPU34.RES:Rescheduling_interrupts
1205608 -29.1% 855330 ? 2% interrupts.CPU35.LOC:Local_timer_interrupts
3591355 ? 6% -67.4% 1169240 ? 13% interrupts.CPU35.RES:Rescheduling_interrupts
1205561 -29.1% 855312 ? 2% interrupts.CPU36.LOC:Local_timer_interrupts
3565397 ? 7% -68.0% 1141008 ? 8% interrupts.CPU36.RES:Rescheduling_interrupts
1205566 -29.1% 855277 ? 2% interrupts.CPU37.LOC:Local_timer_interrupts
3496519 ? 4% -69.1% 1079423 ? 7% interrupts.CPU37.RES:Rescheduling_interrupts
1205562 -29.1% 855258 ? 2% interrupts.CPU38.LOC:Local_timer_interrupts
3564781 ? 7% -69.5% 1088902 ? 12% interrupts.CPU38.RES:Rescheduling_interrupts
1205555 -29.0% 855356 ? 2% interrupts.CPU39.LOC:Local_timer_interrupts
3286445 ? 9% -64.2% 1177572 ? 13% interrupts.CPU39.RES:Rescheduling_interrupts
487100 ? 43% -47.5% 255865 ? 23% interrupts.CPU4.CAL:Function_call_interrupts
1205646 -29.1% 855198 ? 2% interrupts.CPU4.LOC:Local_timer_interrupts
3273624 ? 5% -66.2% 1106857 ? 15% interrupts.CPU4.RES:Rescheduling_interrupts
1205633 -29.1% 855273 ? 2% interrupts.CPU40.LOC:Local_timer_interrupts
3479005 -64.1% 1248042 ? 12% interrupts.CPU40.RES:Rescheduling_interrupts
1205609 -29.1% 855264 ? 2% interrupts.CPU41.LOC:Local_timer_interrupts
3647883 ? 10% -68.3% 1157486 ? 7% interrupts.CPU41.RES:Rescheduling_interrupts
1205645 -29.1% 855251 ? 2% interrupts.CPU42.LOC:Local_timer_interrupts
3676170 ? 7% -69.2% 1133012 ? 10% interrupts.CPU42.RES:Rescheduling_interrupts
1205540 -29.1% 855299 ? 2% interrupts.CPU43.LOC:Local_timer_interrupts
3629110 ? 8% -69.7% 1100768 ? 5% interrupts.CPU43.RES:Rescheduling_interrupts
1205622 -29.1% 855335 ? 2% interrupts.CPU44.LOC:Local_timer_interrupts
3463203 ? 5% -64.9% 1215805 ? 13% interrupts.CPU44.RES:Rescheduling_interrupts
1079682 ? 54% -71.2% 310597 ? 37% interrupts.CPU45.CAL:Function_call_interrupts
1205568 -29.1% 855297 ? 2% interrupts.CPU45.LOC:Local_timer_interrupts
3679792 ? 19% -69.9% 1108076 ? 9% interrupts.CPU45.RES:Rescheduling_interrupts
1205556 -29.1% 855281 ? 2% interrupts.CPU46.LOC:Local_timer_interrupts
3498927 ? 6% -67.1% 1151118 ? 10% interrupts.CPU46.RES:Rescheduling_interrupts
1205560 -29.1% 855311 ? 2% interrupts.CPU47.LOC:Local_timer_interrupts
3719213 ? 6% -68.3% 1178088 ? 14% interrupts.CPU47.RES:Rescheduling_interrupts
1205608 -29.1% 855310 ? 2% interrupts.CPU48.LOC:Local_timer_interrupts
3651264 ? 8% -69.0% 1132056 ? 14% interrupts.CPU48.RES:Rescheduling_interrupts
1235025 ? 51% -66.8% 409515 ? 54% interrupts.CPU49.CAL:Function_call_interrupts
1205544 -29.0% 855344 ? 2% interrupts.CPU49.LOC:Local_timer_interrupts
3487785 ? 6% -68.7% 1091874 ? 6% interrupts.CPU49.RES:Rescheduling_interrupts
649411 ? 60% -56.1% 284866 ? 37% interrupts.CPU5.CAL:Function_call_interrupts
1205621 -29.1% 855152 ? 2% interrupts.CPU5.LOC:Local_timer_interrupts
3481679 ? 5% -68.2% 1107732 ? 17% interrupts.CPU5.RES:Rescheduling_interrupts
1205609 -29.1% 855312 ? 2% interrupts.CPU50.LOC:Local_timer_interrupts
3656124 ? 5% -67.7% 1180412 ? 10% interrupts.CPU50.RES:Rescheduling_interrupts
1205628 -29.1% 855379 ? 2% interrupts.CPU51.LOC:Local_timer_interrupts
3642072 ? 9% -70.0% 1093120 ? 8% interrupts.CPU51.RES:Rescheduling_interrupts
1535288 ? 61% -73.9% 400663 ? 64% interrupts.CPU52.CAL:Function_call_interrupts
1205570 -29.1% 855149 ? 2% interrupts.CPU52.LOC:Local_timer_interrupts
3181327 ? 8% -66.1% 1079937 ? 4% interrupts.CPU52.RES:Rescheduling_interrupts
1205565 -29.1% 855109 ? 2% interrupts.CPU53.LOC:Local_timer_interrupts
3287450 ? 9% -68.9% 1022387 ? 7% interrupts.CPU53.RES:Rescheduling_interrupts
1205586 -29.1% 855161 ? 2% interrupts.CPU54.LOC:Local_timer_interrupts
3447162 ? 8% -68.2% 1095360 ? 9% interrupts.CPU54.RES:Rescheduling_interrupts
673858 ? 62% -57.6% 285884 ? 29% interrupts.CPU55.CAL:Function_call_interrupts
1205592 -29.1% 855167 ? 2% interrupts.CPU55.LOC:Local_timer_interrupts
3681265 ? 4% -69.9% 1107408 ? 9% interrupts.CPU55.RES:Rescheduling_interrupts
1205653 -29.1% 855132 ? 2% interrupts.CPU56.LOC:Local_timer_interrupts
3492355 ? 5% -69.2% 1074433 ? 17% interrupts.CPU56.RES:Rescheduling_interrupts
609594 ? 74% -57.4% 259486 ? 30% interrupts.CPU57.CAL:Function_call_interrupts
1205600 -29.1% 855174 ? 2% interrupts.CPU57.LOC:Local_timer_interrupts
3250090 ? 8% -68.5% 1023666 ? 13% interrupts.CPU57.RES:Rescheduling_interrupts
1205585 -29.1% 855155 ? 2% interrupts.CPU58.LOC:Local_timer_interrupts
3292319 ? 6% -67.8% 1059167 ? 12% interrupts.CPU58.RES:Rescheduling_interrupts
589279 ? 38% -54.7% 266987 ? 21% interrupts.CPU59.CAL:Function_call_interrupts
1205592 -29.1% 855127 ? 2% interrupts.CPU59.LOC:Local_timer_interrupts
2995999 ? 12% -64.6% 1060886 ? 11% interrupts.CPU59.RES:Rescheduling_interrupts
604415 ? 67% -60.6% 237906 ? 18% interrupts.CPU6.CAL:Function_call_interrupts
1205626 -29.1% 855145 ? 2% interrupts.CPU6.LOC:Local_timer_interrupts
3320534 ? 7% -68.0% 1062145 ? 11% interrupts.CPU6.RES:Rescheduling_interrupts
681116 ? 56% -57.6% 288687 ? 31% interrupts.CPU60.CAL:Function_call_interrupts
1205605 -29.1% 855108 ? 2% interrupts.CPU60.LOC:Local_timer_interrupts
3447925 ? 11% -68.3% 1092859 ? 16% interrupts.CPU60.RES:Rescheduling_interrupts
1205600 -29.1% 855136 ? 2% interrupts.CPU61.LOC:Local_timer_interrupts
3688455 ? 10% -68.8% 1149466 ? 12% interrupts.CPU61.RES:Rescheduling_interrupts
1205607 -29.1% 855183 ? 2% interrupts.CPU62.LOC:Local_timer_interrupts
3384029 ? 7% -68.4% 1070519 ? 8% interrupts.CPU62.RES:Rescheduling_interrupts
549499 ? 52% -56.7% 237951 ? 27% interrupts.CPU63.CAL:Function_call_interrupts
1205597 -29.1% 855069 ? 2% interrupts.CPU63.LOC:Local_timer_interrupts
3329589 ? 6% -69.4% 1017592 ? 10% interrupts.CPU63.RES:Rescheduling_interrupts
612129 ? 48% -59.2% 249650 ? 31% interrupts.CPU64.CAL:Function_call_interrupts
1205619 -29.1% 855146 ? 2% interrupts.CPU64.LOC:Local_timer_interrupts
7832 -23.7% 5977 ? 27% interrupts.CPU64.NMI:Non-maskable_interrupts
7832 -23.7% 5977 ? 27% interrupts.CPU64.PMI:Performance_monitoring_interrupts
3357501 ? 11% -69.3% 1029447 ? 8% interrupts.CPU64.RES:Rescheduling_interrupts
659883 ? 70% -61.9% 251209 ? 26% interrupts.CPU65.CAL:Function_call_interrupts
1205621 -29.1% 855152 ? 2% interrupts.CPU65.LOC:Local_timer_interrupts
3421441 ? 12% -68.0% 1093274 ? 11% interrupts.CPU65.RES:Rescheduling_interrupts
569279 ? 75% -52.3% 271317 ? 26% interrupts.CPU66.CAL:Function_call_interrupts
1205711 -29.1% 855164 ? 2% interrupts.CPU66.LOC:Local_timer_interrupts
3597977 ? 10% -70.5% 1059957 ? 8% interrupts.CPU66.RES:Rescheduling_interrupts
772167 ? 57% -69.0% 239751 ? 14% interrupts.CPU67.CAL:Function_call_interrupts
1205625 -29.1% 855124 ? 2% interrupts.CPU67.LOC:Local_timer_interrupts
3415111 ? 5% -67.5% 1110632 ? 8% interrupts.CPU67.RES:Rescheduling_interrupts
454837 ? 40% -50.0% 227206 ? 15% interrupts.CPU68.CAL:Function_call_interrupts
1205569 -29.1% 855170 ? 2% interrupts.CPU68.LOC:Local_timer_interrupts
7836 -38.8% 4795 ? 35% interrupts.CPU68.NMI:Non-maskable_interrupts
7836 -38.8% 4795 ? 35% interrupts.CPU68.PMI:Performance_monitoring_interrupts
3416078 ? 8% -72.0% 956119 ? 5% interrupts.CPU68.RES:Rescheduling_interrupts
627063 ? 46% -60.8% 246039 ? 27% interrupts.CPU69.CAL:Function_call_interrupts
1205639 -29.1% 855170 ? 2% interrupts.CPU69.LOC:Local_timer_interrupts
3327270 ? 6% -69.3% 1020049 ? 11% interrupts.CPU69.RES:Rescheduling_interrupts
593490 ? 42% -52.4% 282559 ? 29% interrupts.CPU7.CAL:Function_call_interrupts
1205657 -29.1% 855174 ? 2% interrupts.CPU7.LOC:Local_timer_interrupts
3303249 ? 5% -67.7% 1067112 ? 10% interrupts.CPU7.RES:Rescheduling_interrupts
652681 ? 58% -65.0% 228460 ? 15% interrupts.CPU70.CAL:Function_call_interrupts
1205689 -29.1% 855186 ? 2% interrupts.CPU70.LOC:Local_timer_interrupts
7831 -38.9% 4784 ? 35% interrupts.CPU70.NMI:Non-maskable_interrupts
7831 -38.9% 4784 ? 35% interrupts.CPU70.PMI:Performance_monitoring_interrupts
3225378 ? 9% -67.3% 1055573 ? 9% interrupts.CPU70.RES:Rescheduling_interrupts
1205544 -29.1% 855158 ? 2% interrupts.CPU71.LOC:Local_timer_interrupts
3349906 ? 4% -69.4% 1026051 ? 13% interrupts.CPU71.RES:Rescheduling_interrupts
667025 ? 62% -58.9% 274338 ? 23% interrupts.CPU72.CAL:Function_call_interrupts
1205643 -29.1% 855196 ? 2% interrupts.CPU72.LOC:Local_timer_interrupts
3452935 ? 10% -68.3% 1096091 ? 8% interrupts.CPU72.RES:Rescheduling_interrupts
1205653 -29.1% 855183 ? 2% interrupts.CPU73.LOC:Local_timer_interrupts
3347362 ? 14% -66.9% 1107626 ? 12% interrupts.CPU73.RES:Rescheduling_interrupts
1205609 -29.1% 855130 ? 2% interrupts.CPU74.LOC:Local_timer_interrupts
3510382 ? 9% -67.8% 1128850 ? 12% interrupts.CPU74.RES:Rescheduling_interrupts
1205610 -29.1% 855095 ? 2% interrupts.CPU75.LOC:Local_timer_interrupts
3684382 ? 5% -69.9% 1108322 ? 8% interrupts.CPU75.RES:Rescheduling_interrupts
1205626 -29.1% 855135 ? 2% interrupts.CPU76.LOC:Local_timer_interrupts
3424662 ? 8% -70.0% 1026323 ? 5% interrupts.CPU76.RES:Rescheduling_interrupts
679078 ? 73% -65.9% 231660 ? 18% interrupts.CPU77.CAL:Function_call_interrupts
1205618 -29.1% 855181 ? 2% interrupts.CPU77.LOC:Local_timer_interrupts
3145940 ? 8% -66.5% 1055195 ? 12% interrupts.CPU77.RES:Rescheduling_interrupts
1205573 -29.1% 855322 ? 2% interrupts.CPU78.LOC:Local_timer_interrupts
2959807 ? 11% -62.0% 1126121 ? 8% interrupts.CPU78.RES:Rescheduling_interrupts
1036604 ? 45% -68.8% 323128 ? 30% interrupts.CPU79.CAL:Function_call_interrupts
1205615 -29.1% 855298 ? 2% interrupts.CPU79.LOC:Local_timer_interrupts
3462919 ? 7% -69.0% 1071952 ? 7% interrupts.CPU79.RES:Rescheduling_interrupts
1205598 -29.1% 855153 ? 2% interrupts.CPU8.LOC:Local_timer_interrupts
3589350 ? 7% -69.3% 1101811 ? 16% interrupts.CPU8.RES:Rescheduling_interrupts
1205586 -29.1% 855353 ? 2% interrupts.CPU80.LOC:Local_timer_interrupts
3502569 ? 9% -68.1% 1118950 ? 9% interrupts.CPU80.RES:Rescheduling_interrupts
1205553 -29.1% 855317 ? 2% interrupts.CPU81.LOC:Local_timer_interrupts
3453668 ? 2% -67.5% 1124145 ? 11% interrupts.CPU81.RES:Rescheduling_interrupts
1205574 -29.0% 855366 ? 2% interrupts.CPU82.LOC:Local_timer_interrupts
3443920 ? 9% -68.3% 1092950 ? 5% interrupts.CPU82.RES:Rescheduling_interrupts
1205620 -29.1% 855340 ? 2% interrupts.CPU83.LOC:Local_timer_interrupts
3332776 ? 5% -65.4% 1152593 ? 6% interrupts.CPU83.RES:Rescheduling_interrupts
1205595 -29.0% 855383 ? 2% interrupts.CPU84.LOC:Local_timer_interrupts
3298021 ? 11% -63.9% 1191118 ? 13% interrupts.CPU84.RES:Rescheduling_interrupts
1205568 -29.1% 855327 ? 2% interrupts.CPU85.LOC:Local_timer_interrupts
3403587 ? 7% -65.2% 1185507 ? 9% interrupts.CPU85.RES:Rescheduling_interrupts
1205556 -29.0% 855358 ? 2% interrupts.CPU86.LOC:Local_timer_interrupts
3413766 ? 6% -65.8% 1168950 ? 11% interrupts.CPU86.RES:Rescheduling_interrupts
1205588 -29.1% 855254 ? 2% interrupts.CPU87.LOC:Local_timer_interrupts
3819022 ? 3% -69.0% 1182764 ? 11% interrupts.CPU87.RES:Rescheduling_interrupts
1205634 -29.0% 855403 ? 2% interrupts.CPU88.LOC:Local_timer_interrupts
3554878 ? 4% -68.0% 1135894 ? 9% interrupts.CPU88.RES:Rescheduling_interrupts
1205618 -29.1% 855273 ? 2% interrupts.CPU89.LOC:Local_timer_interrupts
3506043 ? 4% -69.3% 1075922 ? 3% interrupts.CPU89.RES:Rescheduling_interrupts
637361 ? 38% -54.9% 287418 ? 24% interrupts.CPU9.CAL:Function_call_interrupts
1205624 -29.1% 855173 ? 2% interrupts.CPU9.LOC:Local_timer_interrupts
3575063 ? 4% -66.4% 1200524 ? 15% interrupts.CPU9.RES:Rescheduling_interrupts
1205711 -29.1% 855277 ? 2% interrupts.CPU90.LOC:Local_timer_interrupts
3540800 ? 7% -68.4% 1118482 ? 9% interrupts.CPU90.RES:Rescheduling_interrupts
1205569 -29.1% 855324 ? 2% interrupts.CPU91.LOC:Local_timer_interrupts
3631748 ? 6% -67.7% 1172587 ? 14% interrupts.CPU91.RES:Rescheduling_interrupts
1205564 -29.1% 855323 ? 2% interrupts.CPU92.LOC:Local_timer_interrupts
3573401 ? 6% -66.0% 1216552 ? 13% interrupts.CPU92.RES:Rescheduling_interrupts
1205583 -29.1% 855337 ? 2% interrupts.CPU93.LOC:Local_timer_interrupts
3802054 ? 9% -69.1% 1173720 ? 10% interrupts.CPU93.RES:Rescheduling_interrupts
1205561 -29.1% 855271 ? 2% interrupts.CPU94.LOC:Local_timer_interrupts
3658583 ? 16% -68.8% 1140697 ? 7% interrupts.CPU94.RES:Rescheduling_interrupts
1205530 -29.1% 855297 ? 2% interrupts.CPU95.LOC:Local_timer_interrupts
3497532 ? 8% -68.3% 1109998 ? 7% interrupts.CPU95.RES:Rescheduling_interrupts
1205578 -29.1% 855338 ? 2% interrupts.CPU96.LOC:Local_timer_interrupts
3462753 ? 11% -66.0% 1177706 ? 11% interrupts.CPU96.RES:Rescheduling_interrupts
1098954 ? 63% -69.3% 337065 ? 44% interrupts.CPU97.CAL:Function_call_interrupts
1205573 -29.1% 855301 ? 2% interrupts.CPU97.LOC:Local_timer_interrupts
3430508 ? 7% -67.5% 1114780 ? 8% interrupts.CPU97.RES:Rescheduling_interrupts
1205586 -29.1% 855315 ? 2% interrupts.CPU98.LOC:Local_timer_interrupts
3582330 ? 6% -68.1% 1142146 ? 9% interrupts.CPU98.RES:Rescheduling_interrupts
1082601 ? 43% -65.9% 369132 ? 51% interrupts.CPU99.CAL:Function_call_interrupts
1205570 -29.0% 855354 ? 2% interrupts.CPU99.LOC:Local_timer_interrupts
3408496 ? 9% -65.4% 1179828 ? 15% interrupts.CPU99.RES:Rescheduling_interrupts
506.50 ? 2% +358.4% 2321 ? 35% interrupts.IWI:IRQ_work_interrupts
1.254e+08 -29.1% 88944386 ? 2% interrupts.LOC:Local_timer_interrupts
3.601e+08 -68.1% 1.15e+08 ? 5% interrupts.RES:Rescheduling_interrupts
23.71 ? 6% -12.3 11.38 ? 15% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
23.78 ? 6% -12.3 11.46 ? 15% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
24.29 ? 6% -10.1 14.15 ? 18% perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
44.65 -8.8 35.82 ? 44% perf-profile.calltrace.cycles-pp.__libc_read
12.80 ? 6% -8.8 4.03 ? 15% perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
41.47 -8.1 33.38 ? 44% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
41.23 -8.0 33.26 ? 44% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
7.15 ? 5% -7.1 0.00 perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
10.39 ? 5% -5.7 4.71 ? 16% perf-profile.calltrace.cycles-pp.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
12.04 ? 4% -4.3 7.71 ? 15% perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
10.12 ? 7% -3.8 6.31 ? 16% perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
10.20 ? 7% -3.8 6.41 ? 16% perf-profile.calltrace.cycles-pp.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
7.54 ? 7% -3.5 4.06 ? 15% perf-profile.calltrace.cycles-pp.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
4.26 ? 6% -1.8 2.49 ? 17% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.50 ? 2% -1.8 2.72 ? 45% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
3.31 ? 7% -1.3 1.98 ? 21% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.24 ? 3% -0.9 0.32 ?102% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
2.40 ? 7% -0.9 1.49 ? 27% perf-profile.calltrace.cycles-pp.select_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
41.59 -0.7 40.88 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
42.10 -0.6 41.50 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
1.01 ? 3% -0.5 0.47 ? 45% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
0.46 ? 44% +0.2 0.65 ? 6% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.63 ? 4% +0.2 0.84 ? 8% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.44 ? 44% +0.3 0.73 ? 10% perf-profile.calltrace.cycles-pp.__fget_files.__fget_light.__fdget_pos.ksys_write.do_syscall_64
0.62 ? 2% +0.3 0.94 ? 9% perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
0.64 ? 2% +0.3 0.97 ? 9% perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
0.80 ? 2% +0.4 1.16 ? 10% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
0.55 ? 5% +0.4 0.92 ? 11% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
0.60 ? 7% +0.5 1.08 ? 29% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.schedule_timeout.unix_stream_read_generic
0.46 ? 45% +0.5 0.95 ? 15% perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.08 ?223% +0.5 0.58 ? 5% perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
1.03 ? 2% +0.5 1.53 ? 12% perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.97 ? 2% +0.5 1.47 ? 13% perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
1.01 ? 2% +0.5 1.52 ? 12% perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
0.76 ? 4% +0.5 1.30 ? 14% perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
1.58 +0.7 2.25 ? 9% perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
1.54 +0.7 2.23 ? 9% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
1.50 +0.7 2.19 ? 9% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
0.18 ?141% +0.7 0.88 ? 11% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
0.00 +0.7 0.70 ? 12% perf-profile.calltrace.cycles-pp.__slab_free.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +0.8 0.77 ? 12% perf-profile.calltrace.cycles-pp.__slab_free.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +1.0 0.96 ? 31% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.schedule_timeout
1.10 ? 9% +1.0 2.14 ? 29% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.64 ? 16% +1.3 1.93 ? 30% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.3 1.33 ? 20% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic
0.11 ?223% +1.7 1.80 ? 21% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_prepare
0.00 +1.7 1.69 ? 19% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg
0.12 ?223% +1.7 1.87 ? 22% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.00 +1.8 1.82 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +1.9 1.90 ? 34% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
0.26 ?100% +1.9 2.21 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
0.00 +2.0 2.00 ? 20% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb
0.00 +2.1 2.06 ? 32% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
0.00 +2.1 2.14 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
0.00 +2.7 2.66 ? 19% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.consume_skb
4.48 ? 12% +2.9 7.40 ? 20% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.38 ? 13% +3.0 7.36 ? 20% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.67 ? 10% +3.3 3.94 ? 20% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
0.91 ? 10% +3.3 4.22 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
0.98 ? 9% +3.3 4.32 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
0.00 +3.4 3.38 ? 19% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.consume_skb.unix_stream_read_generic
0.00 +3.7 3.66 ? 19% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
3.65 ? 16% +3.8 7.41 ? 16% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
2.40 ? 22% +4.0 6.39 ? 20% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.20 ?223% +4.4 4.61 ? 16% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout
0.22 ?223% +4.6 4.82 ? 16% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.unix_stream_read_generic
1.75 ? 32% +4.6 6.38 ? 16% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
0.94 ? 56% +4.7 5.68 ? 20% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.84 ? 58% +4.8 5.61 ? 15% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.22 ?223% +4.8 5.03 ? 20% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity
0.22 ?223% +4.9 5.14 ? 20% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair
0.21 ?223% +4.9 5.13 ? 15% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
0.21 ?223% +5.0 5.16 ? 15% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
0.21 ?223% +5.0 5.17 ? 15% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
0.91 ? 59% +5.0 5.95 ? 16% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.24 ?223% +5.1 5.33 ? 20% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate
0.23 ?223% +5.1 5.32 ? 16% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
0.26 ?223% +5.2 5.43 ? 15% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.23 ?223% +5.2 5.45 ? 16% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
0.26 ?223% +5.4 5.61 ? 16% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
0.30 ?223% +5.9 6.18 ? 18% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
0.30 ?223% +5.9 6.23 ? 18% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
0.30 ?223% +5.9 6.24 ? 18% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
0.00 +6.0 5.96 ? 20% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
13.17 ? 10% +6.5 19.67 ? 16% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
12.98 ? 10% +6.6 19.54 ? 16% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
0.00 +6.6 6.57 ? 21% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
12.81 ? 10% +6.6 19.44 ? 16% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
0.00 +7.0 6.95 ? 18% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
8.23 ? 14% +7.2 15.44 ? 17% perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
8.02 ? 14% +7.3 15.34 ? 17% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
7.82 ? 14% +7.4 15.25 ? 17% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
13.78 ? 10% +8.3 22.12 ? 15% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
14.12 ? 10% +8.6 22.67 ? 14% perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.44 ?223% +9.8 10.26 ? 18% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
0.44 ?223% +9.9 10.32 ? 18% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
0.77 ?223% +16.9 17.66 ? 19% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
0.84 ?223% +18.4 19.29 ? 18% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
0.85 ?223% +18.6 19.40 ? 18% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
0.92 ?223% +20.2 21.07 ? 18% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
23.72 ? 6% -12.3 11.40 ? 15% perf-profile.children.cycles-pp.__alloc_skb
23.78 ? 6% -12.3 11.46 ? 15% perf-profile.children.cycles-pp.alloc_skb_with_frags
24.29 ? 6% -10.1 14.16 ? 18% perf-profile.children.cycles-pp.sock_alloc_send_pskb
10.46 ? 5% -9.9 0.55 ? 8% perf-profile.children.cycles-pp.get_obj_cgroup_from_current
44.80 -8.8 35.98 ? 44% perf-profile.children.cycles-pp.__libc_read
12.84 ? 6% -8.8 4.07 ? 15% perf-profile.children.cycles-pp.kmem_cache_alloc_node
10.40 ? 5% -5.7 4.72 ? 16% perf-profile.children.cycles-pp.kfree
12.04 ? 4% -4.3 7.71 ? 15% perf-profile.children.cycles-pp.consume_skb
10.17 ? 7% -3.8 6.36 ? 16% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
10.21 ? 7% -3.8 6.42 ? 16% perf-profile.children.cycles-pp.kmalloc_reserve
3.80 ? 7% -3.7 0.13 ? 11% perf-profile.children.cycles-pp.refill_obj_stock
7.56 ? 7% -3.5 4.08 ? 15% perf-profile.children.cycles-pp.kmem_cache_free
3.32 ? 10% -2.3 1.03 ? 11% perf-profile.children.cycles-pp.mod_objcg_state
4.28 ? 6% -1.7 2.53 ? 17% perf-profile.children.cycles-pp.select_task_rq_fair
11.30 ? 3% -1.7 9.58 ? 3% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
2.54 ? 6% -1.4 1.12 ? 19% perf-profile.children.cycles-pp.pick_next_task_fair
3.33 ? 7% -1.3 2.01 ? 21% perf-profile.children.cycles-pp.select_idle_sibling
2.54 ? 4% -1.2 1.36 ? 17% perf-profile.children.cycles-pp.update_load_avg
2.42 ? 7% -0.9 1.53 ? 26% perf-profile.children.cycles-pp.select_idle_cpu
1.13 ? 10% -0.8 0.30 ? 22% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg
41.62 -0.7 40.92 perf-profile.children.cycles-pp.unix_stream_sendmsg
1.27 ? 8% -0.7 0.58 ? 13% perf-profile.children.cycles-pp.switch_fpu_return
1.24 ? 5% -0.7 0.56 ? 21% perf-profile.children.cycles-pp.reweight_entity
42.10 -0.6 41.52 perf-profile.children.cycles-pp.sock_sendmsg
1.52 ? 8% -0.6 0.97 ? 21% perf-profile.children.cycles-pp.available_idle_cpu
0.82 ? 4% -0.4 0.38 ? 20% perf-profile.children.cycles-pp.set_next_entity
0.84 ? 3% -0.4 0.44 ? 16% perf-profile.children.cycles-pp.update_rq_clock
0.78 ? 9% -0.4 0.38 ? 18% perf-profile.children.cycles-pp.__switch_to_asm
0.62 ? 10% -0.4 0.24 ? 23% perf-profile.children.cycles-pp.pick_next_entity
0.66 ? 4% -0.4 0.29 ? 22% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.72 ? 4% -0.4 0.36 ? 18% perf-profile.children.cycles-pp.__update_load_avg_se
0.63 ? 3% -0.4 0.28 ? 23% perf-profile.children.cycles-pp.check_preempt_curr
0.55 ? 3% -0.3 0.21 ? 19% perf-profile.children.cycles-pp.__enqueue_entity
0.60 ? 9% -0.3 0.28 ? 13% perf-profile.children.cycles-pp.__switch_to
0.50 ? 11% -0.3 0.19 ? 20% perf-profile.children.cycles-pp.put_prev_entity
0.65 ? 4% -0.3 0.35 ? 16% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.47 ? 6% -0.3 0.21 ? 23% perf-profile.children.cycles-pp.check_preempt_wakeup
0.40 ? 10% -0.2 0.18 ? 21% perf-profile.children.cycles-pp.copy_fpregs_to_fpstate
0.35 ? 9% -0.2 0.13 ? 18% perf-profile.children.cycles-pp.___perf_sw_event
0.26 ? 13% -0.2 0.10 ? 23% perf-profile.children.cycles-pp.clear_buddies
0.27 ? 7% -0.1 0.13 ? 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.26 ? 9% -0.1 0.13 ? 19% perf-profile.children.cycles-pp.__calc_delta
0.30 ? 3% -0.1 0.16 ? 19% perf-profile.children.cycles-pp.update_min_vruntime
0.41 ? 3% -0.1 0.28 ? 45% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.27 ? 4% -0.1 0.14 ? 22% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.20 ? 3% -0.1 0.08 ? 16% perf-profile.children.cycles-pp.rb_erase
0.17 ? 8% -0.1 0.08 ? 18% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.18 ? 11% -0.1 0.10 ? 11% perf-profile.children.cycles-pp.cpus_share_cache
0.12 ? 6% -0.1 0.04 ? 71% perf-profile.children.cycles-pp.clockevents_program_event
0.23 ? 4% -0.1 0.14 ? 7% perf-profile.children.cycles-pp.preempt_schedule_common
0.24 ? 6% -0.1 0.16 ? 23% perf-profile.children.cycles-pp._find_next_bit
0.15 ? 9% -0.1 0.07 ? 35% perf-profile.children.cycles-pp.set_next_buddy
0.14 ? 10% -0.1 0.06 ? 23% perf-profile.children.cycles-pp.resched_curr
0.10 ? 11% -0.1 0.03 ?100% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.13 ? 7% -0.1 0.06 ? 20% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.25 ? 4% -0.1 0.18 ? 10% perf-profile.children.cycles-pp.__list_del_entry_valid
0.14 ? 5% -0.1 0.07 ? 20% perf-profile.children.cycles-pp.rb_insert_color
0.14 ? 14% -0.1 0.07 ? 20% perf-profile.children.cycles-pp.migrate_task_rq_fair
0.41 -0.1 0.35 ? 3% perf-profile.children.cycles-pp.__cond_resched
0.08 ? 4% -0.1 0.03 ?100% perf-profile.children.cycles-pp.rb_next
0.11 ? 4% -0.0 0.07 ? 18% perf-profile.children.cycles-pp.__list_add_valid
0.10 ? 6% -0.0 0.06 ? 17% perf-profile.children.cycles-pp.cpumask_next
0.10 ? 10% -0.0 0.07 ? 9% perf-profile.children.cycles-pp.native_irq_return_iret
0.08 ? 16% -0.0 0.05 ? 8% perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
0.09 ? 4% -0.0 0.07 ? 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.05 +0.0 0.07 ? 9% perf-profile.children.cycles-pp.kmalloc_slab
0.18 ? 3% +0.0 0.21 ? 5% perf-profile.children.cycles-pp.__might_fault
0.10 ? 3% +0.0 0.13 ? 7% perf-profile.children.cycles-pp.apparmor_socket_getpeersec_dgram
0.14 ? 3% +0.0 0.18 ? 8% perf-profile.children.cycles-pp.apparmor_file_permission
0.02 ?142% +0.0 0.06 ? 14% perf-profile.children.cycles-pp.irq_exit_rcu
0.15 ? 4% +0.0 0.20 ? 6% perf-profile.children.cycles-pp.security_socket_getpeersec_dgram
0.15 ? 6% +0.1 0.20 perf-profile.children.cycles-pp.rcu_read_unlock_strict
0.10 ? 3% +0.1 0.16 ? 12% perf-profile.children.cycles-pp.aa_file_perm
0.00 +0.1 0.06 ? 18% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.rcu_is_watching
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context
0.36 ? 4% +0.1 0.42 ? 4% perf-profile.children.cycles-pp.fsnotify
0.01 ?223% +0.1 0.07 ? 23% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test
0.08 ? 20% +0.1 0.14 ? 19% perf-profile.children.cycles-pp.perf_trace_buf_update
0.00 +0.1 0.07 ? 27% perf-profile.children.cycles-pp.__module_text_address
0.00 +0.1 0.07 ? 14% perf-profile.children.cycles-pp.ftrace_ops_trampoline
0.25 ? 5% +0.1 0.33 ? 10% perf-profile.children.cycles-pp.__virt_addr_valid
0.00 +0.1 0.08 ? 13% perf-profile.children.cycles-pp.is_ftrace_trampoline
0.08 ? 21% +0.1 0.16 ? 11% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.06 ? 14% +0.1 0.15 ? 18% perf-profile.children.cycles-pp.refill_stock
0.57 ? 4% +0.1 0.66 ? 5% perf-profile.children.cycles-pp.common_file_perm
0.28 ? 11% +0.1 0.37 ? 15% perf-profile.children.cycles-pp.native_sched_clock
0.00 +0.1 0.09 ? 23% perf-profile.children.cycles-pp.is_module_text_address
0.02 ?142% +0.1 0.12 ? 22% perf-profile.children.cycles-pp.drain_stock
0.40 ? 5% +0.1 0.51 ? 6% perf-profile.children.cycles-pp.security_socket_sendmsg
0.41 ? 3% +0.1 0.52 ? 4% perf-profile.children.cycles-pp.security_socket_recvmsg
0.29 +0.1 0.40 ? 11% perf-profile.children.cycles-pp.fput_many
0.48 ? 3% +0.1 0.59 ? 5% perf-profile.children.cycles-pp.sock_recvmsg
0.00 +0.1 0.12 ? 16% perf-profile.children.cycles-pp.get_stack_info_noinstr
0.31 ? 10% +0.1 0.43 ? 15% perf-profile.children.cycles-pp.sched_clock_cpu
0.25 ? 9% +0.1 0.37 ? 11% perf-profile.children.cycles-pp.__build_skb_around
0.02 ?141% +0.1 0.16 ? 18% perf-profile.children.cycles-pp.try_charge
0.00 +0.2 0.15 ? 16% perf-profile.children.cycles-pp.local_clock
0.01 ?223% +0.2 0.16 ? 15% perf-profile.children.cycles-pp.__is_insn_slot_addr
0.32 ? 9% +0.2 0.48 ? 10% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.00 +0.2 0.16 ? 14% perf-profile.children.cycles-pp.get_callchain_entry
0.50 ? 4% +0.2 0.67 ? 5% perf-profile.children.cycles-pp.aa_sk_perm
0.11 ? 24% +0.2 0.28 ? 30% perf-profile.children.cycles-pp.__mod_memcg_state
0.00 +0.2 0.17 ? 14% perf-profile.children.cycles-pp.perf_tp_event_match
0.01 ?223% +0.2 0.19 ? 11% perf-profile.children.cycles-pp.bpf_ksym_find
0.18 ? 10% +0.2 0.36 ? 15% perf-profile.children.cycles-pp.skb_release_data
0.25 ? 5% +0.2 0.44 ? 14% perf-profile.children.cycles-pp.skb_unlink
0.00 +0.2 0.19 ? 15% perf-profile.children.cycles-pp.get_stack_info
0.34 ? 7% +0.2 0.54 ? 5% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.01 ?223% +0.2 0.21 ? 63% perf-profile.children.cycles-pp.generic_file_write_iter
0.01 ?223% +0.2 0.21 ? 63% perf-profile.children.cycles-pp.__generic_file_write_iter
0.01 ?223% +0.2 0.21 ? 63% perf-profile.children.cycles-pp.generic_perform_write
0.91 ? 4% +0.2 1.11 ? 6% perf-profile.children.cycles-pp.security_file_permission
0.00 +0.2 0.21 ? 57% perf-profile.children.cycles-pp.perf_output_copy
0.16 ? 16% +0.2 0.37 ? 24% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.43 ? 7% +0.2 0.64 ? 11% perf-profile.children.cycles-pp.skb_set_owner_w
0.33 ? 13% +0.2 0.55 ? 10% perf-profile.children.cycles-pp.finish_task_switch
0.01 ?223% +0.2 0.25 ? 17% perf-profile.children.cycles-pp.perf_instruction_pointer
0.46 ? 3% +0.3 0.71 ? 2% perf-profile.children.cycles-pp.hrtimer_interrupt
0.01 ?223% +0.3 0.27 ? 12% perf-profile.children.cycles-pp.is_bpf_text_address
0.46 ? 3% +0.3 0.72 ? 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
1.04 ? 2% +0.3 1.31 ? 8% perf-profile.children.cycles-pp.__fget_light
0.51 ? 4% +0.3 0.79 ? 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.78 ? 4% +0.3 1.06 ? 10% perf-profile.children.cycles-pp.__fget_files
1.06 ? 2% +0.3 1.34 ? 8% perf-profile.children.cycles-pp.__fdget_pos
0.57 ? 3% +0.3 0.86 ? 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.24 ? 2% +0.3 0.54 ? 18% perf-profile.children.cycles-pp.unix_write_space
0.01 ?223% +0.3 0.31 ? 12% perf-profile.children.cycles-pp.perf_misc_flags
0.11 ? 12% +0.3 0.42 ? 6% perf-profile.children.cycles-pp.task_tick_fair
0.23 ? 7% +0.3 0.55 ? 4% perf-profile.children.cycles-pp.tick_sched_timer
0.21 ? 7% +0.3 0.53 ? 4% perf-profile.children.cycles-pp.update_process_times
0.21 ? 6% +0.3 0.54 ? 4% perf-profile.children.cycles-pp.tick_sched_handle
0.65 +0.3 0.97 ? 9% perf-profile.children.cycles-pp.simple_copy_to_iter
0.14 ? 8% +0.3 0.47 ? 4% perf-profile.children.cycles-pp.scheduler_tick
0.27 ? 5% +0.3 0.60 ? 4% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.00 +0.3 0.34 ? 58% perf-profile.children.cycles-pp.perf_output_sample
0.02 ?223% +0.4 0.37 ? 11% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.80 ? 2% +0.4 1.16 ? 9% perf-profile.children.cycles-pp._copy_to_iter
0.02 ?223% +0.4 0.38 ? 15% perf-profile.children.cycles-pp.perf_output_begin_forward
0.19 ? 96% +0.4 0.56 ? 27% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.55 ? 5% +0.4 0.92 ? 11% perf-profile.children.cycles-pp.copyout
0.20 ? 95% +0.4 0.57 ? 27% perf-profile.children.cycles-pp.sysvec_call_function_single
0.14 ? 94% +0.4 0.53 ? 28% perf-profile.children.cycles-pp.sched_ttwu_pending
1.05 +0.4 1.44 ? 8% perf-profile.children.cycles-pp.__check_object_size
0.22 ? 95% +0.4 0.62 ? 27% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.01 ?223% +0.4 0.40 ? 26% perf-profile.children.cycles-pp.unfreeze_partials
0.02 ?144% +0.4 0.43 ? 26% perf-profile.children.cycles-pp.put_cpu_partial
0.02 ?223% +0.4 0.44 ? 15% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.02 ?223% +0.4 0.45 ? 12% perf-profile.children.cycles-pp.perf_event_pid_type
0.02 ?223% +0.5 0.51 ? 15% perf-profile.children.cycles-pp.__get_user_nocheck_8
1.03 +0.5 1.53 ? 12% perf-profile.children.cycles-pp.skb_release_all
0.86 ? 3% +0.5 1.35 ? 11% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.00 ? 2% +0.5 1.50 ? 13% perf-profile.children.cycles-pp.unix_destruct_scm
1.02 ? 2% +0.5 1.52 ? 12% perf-profile.children.cycles-pp.skb_release_head_state
0.76 ? 4% +0.5 1.30 ? 14% perf-profile.children.cycles-pp.sock_wfree
0.22 ? 40% +0.6 0.78 ? 18% perf-profile.children.cycles-pp.set_task_cpu
0.04 ?223% +0.6 0.66 ? 17% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
1.58 +0.7 2.26 ? 9% perf-profile.children.cycles-pp.unix_stream_read_actor
1.55 +0.7 2.23 ? 9% perf-profile.children.cycles-pp.skb_copy_datagram_iter
1.51 +0.7 2.20 ? 9% perf-profile.children.cycles-pp.__skb_datagram_iter
0.03 ?223% +0.7 0.73 ? 14% perf-profile.children.cycles-pp.kvm_is_in_guest
0.06 ? 54% +0.8 0.83 ? 22% perf-profile.children.cycles-pp.get_partial_node
0.04 ?223% +0.9 0.98 ? 14% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.04 ?223% +1.0 1.01 ? 15% perf-profile.children.cycles-pp.perf_callchain_user
0.24 ? 24% +1.0 1.22 ? 20% perf-profile.children.cycles-pp.___slab_alloc
0.25 ? 24% +1.0 1.25 ? 20% perf-profile.children.cycles-pp.__slab_alloc
0.46 ? 11% +1.0 1.47 ? 12% perf-profile.children.cycles-pp.__slab_free
0.00 +1.2 1.21 ? 53% perf-profile.children.cycles-pp.intel_idle
0.06 ?223% +1.3 1.32 ? 16% perf-profile.children.cycles-pp.kernel_text_address
0.00 +1.3 1.28 ? 53% perf-profile.children.cycles-pp.cpuidle_enter
0.00 +1.3 1.28 ? 53% perf-profile.children.cycles-pp.cpuidle_enter_state
0.06 ?223% +1.4 1.45 ? 18% perf-profile.children.cycles-pp.stack_access_ok
0.01 ?223% +1.6 1.60 ? 54% perf-profile.children.cycles-pp.start_secondary
0.01 ?223% +1.6 1.61 ? 54% perf-profile.children.cycles-pp.do_idle
0.01 ?223% +1.6 1.61 ? 54% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
0.01 ?223% +1.6 1.61 ? 54% perf-profile.children.cycles-pp.cpu_startup_entry
0.08 ?223% +1.6 1.70 ? 16% perf-profile.children.cycles-pp.__kernel_text_address
0.10 ?223% +2.1 2.17 ? 17% perf-profile.children.cycles-pp.unwind_get_return_address
0.34 ? 9% +2.2 2.49 ? 20% perf-profile.children.cycles-pp.propagate_protected_usage
0.10 ?223% +2.2 2.30 ? 15% perf-profile.children.cycles-pp.orc_find
3.20 ? 4% +3.0 6.16 ? 9% perf-profile.children.cycles-pp._raw_spin_lock
0.76 ? 7% +3.3 4.03 ? 19% perf-profile.children.cycles-pp.page_counter_cancel
4.58 ? 12% +3.4 7.95 ? 17% perf-profile.children.cycles-pp.ttwu_do_activate
1.91 ? 6% +3.4 5.33 ? 21% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
4.48 ? 12% +3.4 7.90 ? 17% perf-profile.children.cycles-pp.enqueue_task_fair
1.86 ? 8% +3.6 5.49 ? 19% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
0.16 ?223% +3.7 3.82 ? 16% perf-profile.children.cycles-pp.__orc_find
3.66 ? 16% +3.9 7.55 ? 16% perf-profile.children.cycles-pp.dequeue_task_fair
0.94 ? 7% +4.2 5.19 ? 19% perf-profile.children.cycles-pp.page_counter_uncharge
2.51 ? 21% +4.4 6.86 ? 17% perf-profile.children.cycles-pp.enqueue_entity
1.78 ? 31% +4.7 6.51 ? 15% perf-profile.children.cycles-pp.dequeue_entity
0.84 ? 58% +4.9 5.72 ? 14% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
1.36 ? 9% +5.0 6.37 ? 19% perf-profile.children.cycles-pp.obj_cgroup_charge_pages
1.06 ? 9% +5.0 6.08 ? 20% perf-profile.children.cycles-pp.page_counter_try_charge
1.48 ? 8% +5.0 6.53 ? 19% perf-profile.children.cycles-pp.obj_cgroup_charge
1.13 ? 16% +6.5 7.64 ? 8% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.42 ?173% +6.6 7.06 ? 16% perf-profile.children.cycles-pp.perf_trace_sched_switch
13.18 ? 10% +6.8 19.97 ? 16% perf-profile.children.cycles-pp.__wake_up_common
13.00 ? 10% +6.8 19.84 ? 16% perf-profile.children.cycles-pp.autoremove_wake_function
12.83 ? 10% +6.9 19.75 ? 16% perf-profile.children.cycles-pp.try_to_wake_up
0.33 ?223% +7.2 7.56 ? 17% perf-profile.children.cycles-pp.__unwind_start
8.25 ? 14% +7.5 15.73 ? 16% perf-profile.children.cycles-pp.schedule_timeout
10.33 ? 15% +8.4 18.70 ? 17% perf-profile.children.cycles-pp.schedule
14.13 ? 10% +8.5 22.68 ? 14% perf-profile.children.cycles-pp.sock_def_readable
10.39 ? 15% +8.6 18.98 ? 16% perf-profile.children.cycles-pp.__schedule
13.80 ? 10% +8.6 22.42 ? 14% perf-profile.children.cycles-pp.__wake_up_common_lock
2.88 ? 38% +10.0 12.88 ? 16% perf-profile.children.cycles-pp.update_curr
0.65 ?165% +11.1 11.71 ? 16% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
0.62 ?223% +13.8 14.40 ? 16% perf-profile.children.cycles-pp.unwind_next_frame
0.82 ?223% +18.1 18.92 ? 16% perf-profile.children.cycles-pp.perf_callchain_kernel
0.89 ?223% +19.5 20.42 ? 16% perf-profile.children.cycles-pp.get_perf_callchain
0.90 ?223% +19.6 20.54 ? 16% perf-profile.children.cycles-pp.perf_callchain
0.97 ?223% +21.4 22.35 ? 16% perf-profile.children.cycles-pp.perf_prepare_sample
1.00 ?223% +22.4 23.36 ? 15% perf-profile.children.cycles-pp.perf_event_output_forward
1.01 ?223% +22.5 23.48 ? 15% perf-profile.children.cycles-pp.__perf_event_overflow
1.01 ?223% +22.5 23.54 ? 15% perf-profile.children.cycles-pp.perf_swevent_overflow
1.31 ?176% +23.0 24.27 ? 15% perf-profile.children.cycles-pp.perf_tp_event
10.27 ? 5% -9.8 0.44 ? 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current
6.50 ? 5% -5.9 0.64 ? 6% perf-profile.self.cycles-pp.kfree
5.18 ? 8% -4.4 0.76 ? 5% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
4.63 ? 7% -4.1 0.53 ? 6% perf-profile.self.cycles-pp.kmem_cache_alloc_node
4.44 ? 7% -3.5 0.92 ? 10% perf-profile.self.cycles-pp.kmem_cache_free
3.14 ? 10% -2.5 0.64 ? 8% perf-profile.self.cycles-pp.mod_objcg_state
7.51 ? 4% -1.8 5.75 ? 11% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
1.12 ? 10% -0.8 0.29 ? 22% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg
1.26 ? 9% -0.7 0.58 ? 14% perf-profile.self.cycles-pp.switch_fpu_return
1.19 ? 2% -0.6 0.54 ? 25% perf-profile.self.cycles-pp.update_curr
1.40 ? 4% -0.6 0.77 ? 15% perf-profile.self.cycles-pp.__schedule
1.50 ? 8% -0.5 0.96 ? 21% perf-profile.self.cycles-pp.available_idle_cpu
1.07 ? 5% -0.4 0.63 ? 19% perf-profile.self.cycles-pp.update_load_avg
0.78 ? 9% -0.4 0.38 ? 17% perf-profile.self.cycles-pp.__switch_to_asm
0.71 ? 4% -0.3 0.36 ? 18% perf-profile.self.cycles-pp.__update_load_avg_se
0.55 ? 3% -0.3 0.21 ? 20% perf-profile.self.cycles-pp.__enqueue_entity
0.82 ? 7% -0.3 0.49 ? 27% perf-profile.self.cycles-pp.select_idle_cpu
0.62 ? 4% -0.3 0.32 ? 16% perf-profile.self.cycles-pp.update_rq_clock
0.62 ? 5% -0.3 0.33 ? 16% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.52 ? 9% -0.3 0.24 ? 13% perf-profile.self.cycles-pp.__switch_to
0.47 ? 7% -0.3 0.20 ? 13% perf-profile.self.cycles-pp.reweight_entity
0.53 ? 5% -0.3 0.27 ? 18% perf-profile.self.cycles-pp.enqueue_task_fair
0.56 ? 3% -0.2 0.31 ? 12% perf-profile.self.cycles-pp.select_task_rq_fair
0.42 ? 3% -0.2 0.18 ? 22% perf-profile.self.cycles-pp.pick_next_task_fair
0.40 ? 11% -0.2 0.18 ? 19% perf-profile.self.cycles-pp.copy_fpregs_to_fpstate
0.45 ? 5% -0.2 0.26 ? 16% perf-profile.self.cycles-pp.dequeue_task_fair
0.30 ? 9% -0.2 0.11 ? 19% perf-profile.self.cycles-pp.___perf_sw_event
0.42 ? 5% -0.2 0.22 ? 18% perf-profile.self.cycles-pp.enqueue_entity
0.42 ? 2% -0.2 0.23 ? 15% perf-profile.self.cycles-pp.prepare_to_wait
0.39 ? 6% -0.2 0.21 ? 15% perf-profile.self.cycles-pp.select_idle_sibling
0.46 ? 3% -0.2 0.30 ? 15% perf-profile.self.cycles-pp.try_to_wake_up
0.28 ? 8% -0.2 0.12 ? 22% perf-profile.self.cycles-pp.pick_next_entity
0.23 ? 12% -0.1 0.09 ? 23% perf-profile.self.cycles-pp.clear_buddies
0.40 ? 8% -0.1 0.26 ? 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.26 ? 9% -0.1 0.12 ? 20% perf-profile.self.cycles-pp.__calc_delta
0.30 ? 2% -0.1 0.16 ? 18% perf-profile.self.cycles-pp.update_min_vruntime
0.27 ? 8% -0.1 0.13 ? 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.39 ? 3% -0.1 0.26 ? 45% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.24 ? 5% -0.1 0.11 ? 18% perf-profile.self.cycles-pp.dequeue_entity
0.22 ? 6% -0.1 0.10 ? 13% perf-profile.self.cycles-pp.schedule_timeout
0.26 ? 6% -0.1 0.15 ? 11% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.19 ? 4% -0.1 0.08 ? 16% perf-profile.self.cycles-pp.rb_erase
0.21 ? 7% -0.1 0.10 ? 22% perf-profile.self.cycles-pp.check_preempt_wakeup
0.15 ? 6% -0.1 0.05 ? 50% perf-profile.self.cycles-pp.check_preempt_curr
0.17 ? 4% -0.1 0.08 ? 22% perf-profile.self.cycles-pp.set_next_entity
0.17 ? 8% -0.1 0.08 ? 18% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.42 ? 4% -0.1 0.33 ? 5% perf-profile.self.cycles-pp.vfs_read
0.18 ? 11% -0.1 0.10 ? 11% perf-profile.self.cycles-pp.cpus_share_cache
0.17 ? 8% -0.1 0.08 ? 20% perf-profile.self.cycles-pp.schedule
0.38 ? 5% -0.1 0.29 ? 12% perf-profile.self.cycles-pp.cpuacct_charge
0.14 ? 8% -0.1 0.06 ? 33% perf-profile.self.cycles-pp.set_next_buddy
0.14 ? 9% -0.1 0.06 ? 21% perf-profile.self.cycles-pp.resched_curr
0.19 ? 6% -0.1 0.12 ? 16% perf-profile.self.cycles-pp.finish_task_switch
0.14 ? 5% -0.1 0.07 ? 23% perf-profile.self.cycles-pp.rb_insert_color
0.22 ? 6% -0.1 0.15 ? 22% perf-profile.self.cycles-pp._find_next_bit
0.20 ? 6% -0.1 0.13 ? 30% perf-profile.self.cycles-pp.cpumask_next_wrap
0.31 ? 6% -0.1 0.25 ? 4% perf-profile.self.cycles-pp.new_sync_read
0.22 ? 5% -0.1 0.16 ? 8% perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ? 10% -0.1 0.12 ? 7% perf-profile.self.cycles-pp.__wake_up_common
0.21 ? 6% -0.0 0.16 ? 4% perf-profile.self.cycles-pp.unix_destruct_scm
0.10 ? 4% -0.0 0.06 ? 16% perf-profile.self.cycles-pp.__list_add_valid
0.18 ? 4% -0.0 0.15 ? 6% perf-profile.self.cycles-pp.do_syscall_64
0.18 ? 7% -0.0 0.15 ? 5% perf-profile.self.cycles-pp._copy_to_iter
0.10 ? 11% -0.0 0.07 ? 9% perf-profile.self.cycles-pp.native_irq_return_iret
0.13 ? 3% +0.0 0.15 ? 5% perf-profile.self.cycles-pp._copy_from_iter
0.05 +0.0 0.07 ? 10% perf-profile.self.cycles-pp.kmalloc_slab
0.08 ? 5% +0.0 0.10 ? 7% perf-profile.self.cycles-pp.__cond_resched
0.09 ? 10% +0.0 0.12 ? 3% perf-profile.self.cycles-pp.rcu_read_unlock_strict
0.08 ? 4% +0.0 0.11 ? 10% perf-profile.self.cycles-pp.apparmor_socket_getpeersec_dgram
0.11 ? 3% +0.0 0.15 ? 10% perf-profile.self.cycles-pp.obj_cgroup_charge
0.02 ?141% +0.0 0.06 ? 8% perf-profile.self.cycles-pp.security_socket_recvmsg
0.09 ? 7% +0.0 0.13 ? 12% perf-profile.self.cycles-pp.security_file_permission
0.04 ? 47% +0.0 0.09 ? 9% perf-profile.self.cycles-pp.perf_trace_buf_alloc
0.09 +0.0 0.14 ? 11% perf-profile.self.cycles-pp.aa_file_perm
0.00 +0.1 0.06 ? 20% perf-profile.self.cycles-pp.get_stack_info_noinstr
0.00 +0.1 0.06 ? 13% perf-profile.self.cycles-pp.perf_trace_run_bpf_submit
0.01 ?223% +0.1 0.06 ? 19% perf-profile.self.cycles-pp.perf_trace_buf_update
0.00 +0.1 0.06 ? 19% perf-profile.self.cycles-pp.sched_clock_cpu
0.00 +0.1 0.06 ? 18% perf-profile.self.cycles-pp.rcu_is_watching
0.00 +0.1 0.06 ? 18% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.00 +0.1 0.06 ? 9% perf-profile.self.cycles-pp.perf_swevent_overflow
0.00 +0.1 0.06 ? 23% perf-profile.self.cycles-pp.is_bpf_text_address
0.34 ? 5% +0.1 0.40 ? 4% perf-profile.self.cycles-pp.fsnotify
0.27 ? 7% +0.1 0.34 ? 7% perf-profile.self.cycles-pp.aa_sk_perm
0.00 +0.1 0.06 ? 21% perf-profile.self.cycles-pp.perf_instruction_pointer
0.01 ?223% +0.1 0.07 ? 18% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test
0.00 +0.1 0.06 ? 14% perf-profile.self.cycles-pp.ftrace_ops_trampoline
0.24 ? 5% +0.1 0.31 ? 10% perf-profile.self.cycles-pp.__virt_addr_valid
0.00 +0.1 0.08 ? 9% perf-profile.self.cycles-pp.perf_misc_flags
0.00 +0.1 0.08 ? 13% perf-profile.self.cycles-pp.get_stack_info
0.00 +0.1 0.08 ? 16% perf-profile.self.cycles-pp.perf_event_pid_type
0.00 +0.1 0.08 ? 14% perf-profile.self.cycles-pp.__perf_event_overflow
0.27 ? 10% +0.1 0.36 ? 15% perf-profile.self.cycles-pp.native_sched_clock
0.30 ? 3% +0.1 0.39 ? 8% perf-profile.self.cycles-pp.sock_write_iter
0.28 ? 2% +0.1 0.38 ? 7% perf-profile.self.cycles-pp.vfs_write
0.28 +0.1 0.39 ? 11% perf-profile.self.cycles-pp.fput_many
0.00 +0.1 0.10 ? 16% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.00 +0.1 0.11 ? 19% perf-profile.self.cycles-pp.perf_callchain
0.15 ? 17% +0.1 0.26 ? 13% perf-profile.self.cycles-pp.__build_skb_around
0.00 +0.1 0.11 ? 23% perf-profile.self.cycles-pp.unfreeze_partials
0.32 ? 2% +0.1 0.43 ? 7% perf-profile.self.cycles-pp.__alloc_skb
0.16 ? 15% +0.1 0.28 ? 23% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.00 +0.1 0.12 ? 27% perf-profile.self.cycles-pp.obj_cgroup_charge_pages
0.00 +0.1 0.12 ? 24% perf-profile.self.cycles-pp.obj_cgroup_uncharge_pages
0.00 +0.1 0.13 ? 17% perf-profile.self.cycles-pp.__is_insn_slot_addr
0.00 +0.1 0.14 ? 15% perf-profile.self.cycles-pp.local_clock
0.10 ? 25% +0.1 0.24 ? 18% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.00 +0.2 0.16 ? 15% perf-profile.self.cycles-pp.get_callchain_entry
0.01 ?223% +0.2 0.17 ? 20% perf-profile.self.cycles-pp.get_partial_node
0.11 ? 24% +0.2 0.28 ? 31% perf-profile.self.cycles-pp.__mod_memcg_state
0.00 +0.2 0.17 ? 14% perf-profile.self.cycles-pp.perf_tp_event_match
0.30 ? 7% +0.2 0.47 ? 5% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.17 ? 9% +0.2 0.35 ? 14% perf-profile.self.cycles-pp.skb_release_data
0.01 ?223% +0.2 0.19 ? 11% perf-profile.self.cycles-pp.bpf_ksym_find
0.22 ? 15% +0.2 0.40 ? 13% perf-profile.self.cycles-pp.perf_tp_event
0.42 ? 7% +0.2 0.63 ? 11% perf-profile.self.cycles-pp.skb_set_owner_w
0.33 ? 3% +0.2 0.54 ? 12% perf-profile.self.cycles-pp.sock_def_readable
0.16 ? 22% +0.2 0.38 ? 17% perf-profile.self.cycles-pp.___slab_alloc
0.01 ?223% +0.2 0.24 ? 13% perf-profile.self.cycles-pp.perf_event_output_forward
0.50 ? 7% +0.2 0.74 ? 12% perf-profile.self.cycles-pp.sock_wfree
0.76 ? 4% +0.3 1.04 ? 10% perf-profile.self.cycles-pp.__fget_files
0.40 ? 4% +0.3 0.68 ? 10% perf-profile.self.cycles-pp.__check_object_size
0.01 ?223% +0.3 0.29 ? 16% perf-profile.self.cycles-pp.perf_callchain_user
0.01 ?223% +0.3 0.30 ? 17% perf-profile.self.cycles-pp.perf_prepare_sample
0.01 ?223% +0.3 0.33 ? 18% perf-profile.self.cycles-pp.get_perf_callchain
0.02 ?223% +0.3 0.34 ? 17% perf-profile.self.cycles-pp.__kernel_text_address
0.02 ?223% +0.3 0.35 ? 15% perf-profile.self.cycles-pp.perf_output_begin_forward
0.02 ?223% +0.3 0.35 ? 11% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.02 ?223% +0.3 0.36 ? 16% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.02 ?223% +0.5 0.48 ? 17% perf-profile.self.cycles-pp.unwind_get_return_address
0.02 ?223% +0.5 0.49 ? 15% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.84 ? 3% +0.5 1.32 ? 11% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.03 ?223% +0.6 0.63 ? 17% perf-profile.self.cycles-pp.kernel_text_address
0.03 ?223% +0.6 0.64 ? 13% perf-profile.self.cycles-pp.kvm_is_in_guest
0.03 ?223% +0.7 0.72 ? 17% perf-profile.self.cycles-pp.__unwind_start
0.46 ? 11% +1.0 1.45 ? 11% perf-profile.self.cycles-pp.__slab_free
0.00 +1.2 1.21 ? 53% perf-profile.self.cycles-pp.intel_idle
0.06 ?223% +1.2 1.27 ? 17% perf-profile.self.cycles-pp.stack_access_ok
0.07 ?223% +1.5 1.55 ? 16% perf-profile.self.cycles-pp.perf_callchain_kernel
0.09 ?223% +2.1 2.18 ? 15% perf-profile.self.cycles-pp.orc_find
0.34 ? 9% +2.1 2.46 ? 20% perf-profile.self.cycles-pp.propagate_protected_usage
0.76 ? 8% +3.2 3.97 ? 19% perf-profile.self.cycles-pp.page_counter_cancel
0.16 ?223% +3.6 3.81 ? 16% perf-profile.self.cycles-pp.__orc_find
0.89 ? 8% +3.8 4.66 ? 19% perf-profile.self.cycles-pp.page_counter_try_charge
0.29 ?223% +6.4 6.64 ? 17% perf-profile.self.cycles-pp.unwind_next_frame
1.12 ? 17% +6.5 7.63 ? 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
hackbench.throughput
120000 +------------------------------------------------------------------+
|OOOOOOOOOOOOOOOO O OO OOO OOOOOOOOO OOOOO O O O O O O|
100000 |-+ O O |
| |
| ++++++ |
80000 |-+ : |
| +++++ : |
60000 |+++ +++++++++++ ++++ ++++ :+ +++ |
| + + +++++++ + + |
40000 |-+ |
| |
| |
20000 |-+ |
| |
0 +------------------------------------------------------------------+
hackbench.time.voluntary_context_switches
4e+09 +-----------------------------------------------------------------+
| + |
3.5e+09 |++ + + + + + ++++ ++ +++ |
3e+09 |-++++ ++++++ ++++++++++ +++ +++++ ++ |
| |
2.5e+09 |-+ |
| |
2e+09 |-+ |
| |
1.5e+09 |-+ O |
1e+09 |-+ O O OO OO |
| O OOO O OO OO O O O O OO O O O OO OOOO |
5e+08 |OOOOOOOOOOOOOO OOO OO OO O O OO O OOO O O O O|
| |
0 +-----------------------------------------------------------------+
hackbench.time.involuntary_context_switches
2.5e+09 +-----------------------------------------------------------------+
| + + +++ |
|+ + + + + ++ + + + ++ + |
2e+09 |-++ + ++++ + ++ :+++ + + + ++ ++ ++ |
| + + + + + |
| |
1.5e+09 |-+ |
| |
1e+09 |-+ |
| |
| O O |
5e+08 |-+ O O OO O |
| O O OOO O OO OO O O O O OO O O O OO OOOO |
|OOOOOOOOOOOOOO O O OO OO O O OO O OOO O O O O|
0 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2sp4: 96 threads 2 sockets Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-9/performance/socket/8/x86_64-rhel-8.3/process/1600%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp4/hackbench/0x4003006
commit:
68ac5b3c8d ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")
5387c90490 ("mm/memcg: improve refill_obj_stock() performance")
68ac5b3c8db2fda0 5387c90490f7f42df3209154ca9
---------------- ---------------------------
%stddev %change %stddev
\ | \
88731 ? 2% +51.9% 134791 hackbench.throughput
1198 ? 2% -29.4% 846.30 hackbench.time.elapsed_time
1198 ? 2% -29.4% 846.30 hackbench.time.elapsed_time.max
3.018e+08 ? 19% -80.7% 58143527 ? 4% hackbench.time.involuntary_context_switches
8720 -3.9% 8384 hackbench.time.percent_of_cpu_this_job_got
103040 ? 2% -32.3% 69790 hackbench.time.system_time
1444 ? 3% -19.0% 1169 hackbench.time.user_time
6.95e+08 ? 9% -47.3% 3.664e+08 hackbench.time.voluntary_context_switches
1235 ? 2% -28.5% 883.79 uptime.boot
3.53e+08 ? 3% +5.9% 3.739e+08 ? 4% cpuidle.C1.time
11398249 ? 7% +20.0% 13679143 ? 13% cpuidle.C1.usage
4054497 ? 7% +31.0% 5312525 ? 21% numa-numastat.node1.local_node
4091772 ? 7% +30.1% 5321980 ? 21% numa-numastat.node1.numa_hit
2127053 +14.6% 2438337 ? 12% meminfo.Inactive
2126906 +14.6% 2438190 ? 12% meminfo.Inactive(anon)
48105 ? 13% +517.3% 296955 ? 81% meminfo.Mapped
9.59 +5.6 15.23 ? 12% mpstat.cpu.all.idle%
0.00 ? 21% +0.0 0.00 ? 14% mpstat.cpu.all.iowait%
1.25 +0.2 1.45 mpstat.cpu.all.usr%
6676 ? 3% -27.5% 4838 ? 5% vmstat.procs.r
876931 ? 10% -35.4% 566878 vmstat.system.cs
198694 ? 15% +8.1% 214875 vmstat.system.in
7497 ? 35% +100.0% 14992 ? 22% numa-vmstat.node0.nr_mapped
22560 ? 44% +449.2% 123907 ? 51% numa-vmstat.node1.nr_inactive_anon
363.67 ? 9% +43.3% 521.00 ? 7% numa-vmstat.node1.nr_page_table_pages
93398 ? 6% -30.7% 64683 ? 7% numa-vmstat.node1.nr_slab_unreclaimable
22560 ? 44% +449.2% 123907 ? 51% numa-vmstat.node1.nr_zone_inactive_anon
527933 +15.7% 610634 ? 12% proc-vmstat.nr_inactive_anon
12053 ? 8% +533.6% 76375 ? 81% proc-vmstat.nr_mapped
712630 -7.3% 660684 proc-vmstat.nr_slab_unreclaimable
527933 +15.7% 610634 ? 12% proc-vmstat.nr_zone_inactive_anon
185313 ? 21% +79.2% 332164 ? 41% proc-vmstat.numa_pte_updates
961071 -3.3% 929685 proc-vmstat.pgreuse
11391736 ? 7% +20.1% 13679615 ? 13% turbostat.C1
0.31 ? 4% +0.2 0.46 ? 4% turbostat.C1%
8.10 ? 3% +39.8% 11.33 ? 2% turbostat.CPU%c1
75.83 ? 2% +4.2% 79.00 turbostat.PkgTmp
274.06 +4.4% 286.20 turbostat.PkgWatt
71.28 +14.8% 81.85 turbostat.RAMWatt
29763 ? 35% +94.1% 57779 ? 22% numa-meminfo.node0.Mapped
213202 ? 25% +86.4% 397378 ? 18% numa-meminfo.node1.AnonPages.max
90610 ? 42% +434.9% 484697 ? 50% numa-meminfo.node1.Inactive
90610 ? 42% +434.9% 484697 ? 50% numa-meminfo.node1.Inactive(anon)
1462 ? 10% +41.8% 2073 ? 8% numa-meminfo.node1.PageTables
424993 ? 5% -32.9% 285287 ? 6% numa-meminfo.node1.SUnreclaim
503312 ? 7% -28.1% 361751 ? 5% numa-meminfo.node1.Slab
2069 ? 7% +19.2% 2467 ? 10% slabinfo.dmaengine-unmap-16.active_objs
2069 ? 7% +19.2% 2467 ? 10% slabinfo.dmaengine-unmap-16.num_objs
67777 ? 3% -19.8% 54363 ? 2% slabinfo.kmalloc-256.active_objs
4029 ? 3% -20.4% 3206 ? 8% slabinfo.kmalloc-256.active_slabs
128973 ? 3% -20.4% 102611 ? 8% slabinfo.kmalloc-256.num_objs
4029 ? 3% -20.4% 3206 ? 8% slabinfo.kmalloc-256.num_slabs
305469 ? 3% -18.8% 248152 ? 2% slabinfo.kmalloc-512.active_objs
23018 ? 4% -27.0% 16815 ? 4% slabinfo.kmalloc-512.active_slabs
736603 ? 4% -26.9% 538104 ? 4% slabinfo.kmalloc-512.num_objs
23018 ? 4% -27.0% 16815 ? 4% slabinfo.kmalloc-512.num_slabs
57170 ? 6% +9.9% 62827 ? 6% slabinfo.proc_inode_cache.active_objs
213837 ? 4% -30.0% 149771 ? 4% slabinfo.skbuff_head_cache.active_objs
20252 ? 4% -31.3% 13913 ? 5% slabinfo.skbuff_head_cache.active_slabs
648092 ? 4% -31.3% 445250 ? 5% slabinfo.skbuff_head_cache.num_objs
20252 ? 4% -31.3% 13913 ? 5% slabinfo.skbuff_head_cache.num_slabs
9.09 ? 2% +5.1% 9.55 ? 4% perf-stat.i.MPKI
7.058e+09 ? 2% +14.1% 8.056e+09 ? 2% perf-stat.i.branch-instructions
42700294 ? 2% +9.6% 46778564 ? 5% perf-stat.i.branch-misses
1.42e+08 ? 3% +27.3% 1.806e+08 perf-stat.i.cache-misses
3.069e+08 ? 3% +21.1% 3.716e+08 ? 4% perf-stat.i.cache-references
790961 ? 7% -15.5% 668663 ? 3% perf-stat.i.context-switches
6.83 -6.3% 6.40 perf-stat.i.cpi
2.103e+11 -7.0% 1.957e+11 perf-stat.i.cpu-cycles
43342 ? 3% +16.5% 50479 ? 5% perf-stat.i.cpu-migrations
9.593e+09 ? 2% +15.9% 1.112e+10 ? 3% perf-stat.i.dTLB-loads
5.023e+09 ? 2% +17.0% 5.875e+09 ? 3% perf-stat.i.dTLB-stores
60.11 -2.0 58.09 perf-stat.i.iTLB-load-miss-rate%
22838344 ? 3% +12.7% 25735632 ? 5% perf-stat.i.iTLB-load-misses
6214874 ? 5% -14.4% 5317074 ? 3% perf-stat.i.iTLB-loads
3.441e+10 ? 2% +15.0% 3.957e+10 ? 3% perf-stat.i.instructions
3456 ? 3% +9.2% 3776 perf-stat.i.instructions-per-iTLB-miss
0.19 +14.6% 0.22 perf-stat.i.ipc
826.53 ? 11% +25.0% 1033 ? 11% perf-stat.i.major-faults
2.15 -7.4% 1.99 perf-stat.i.metric.GHz
932.12 ? 2% +16.5% 1086 perf-stat.i.metric.K/sec
226.32 ? 2% +14.6% 259.45 ? 3% perf-stat.i.metric.M/sec
45074 ? 2% +16.6% 52551 ? 2% perf-stat.i.minor-faults
17095614 ? 2% +16.8% 19974228 ? 2% perf-stat.i.node-load-misses
25241388 ? 4% +25.9% 31770755 ? 2% perf-stat.i.node-loads
8040076 ? 3% +5.4% 8478188 perf-stat.i.node-store-misses
25780743 ? 3% +23.9% 31948210 perf-stat.i.node-stores
45900 ? 2% +16.7% 53584 perf-stat.i.page-faults
8.96 ? 2% +4.2% 9.34 perf-stat.overall.MPKI
0.63 -0.0 0.59 perf-stat.overall.branch-miss-rate%
51.11 ? 2% +4.3 55.39 perf-stat.overall.cache-miss-rate%
5.55 -26.5% 4.08 perf-stat.overall.cpi
1212 ? 3% -34.9% 789.00 perf-stat.overall.cycles-between-cache-misses
84.57 +6.9 91.44 perf-stat.overall.iTLB-load-miss-rate%
1308 +1.8% 1332 perf-stat.overall.instructions-per-iTLB-miss
0.18 +36.0% 0.25 perf-stat.overall.ipc
32.99 -1.6 31.38 perf-stat.overall.node-load-miss-rate%
18.38 -3.7 14.73 ? 2% perf-stat.overall.node-store-miss-rate%
9.668e+09 +29.8% 1.255e+10 perf-stat.ps.branch-instructions
61121486 ? 2% +20.2% 73498533 perf-stat.ps.branch-misses
2.236e+08 ? 3% +47.8% 3.304e+08 perf-stat.ps.cache-misses
4.376e+08 ? 3% +36.3% 5.967e+08 perf-stat.ps.cache-references
868230 ? 10% -35.4% 560685 perf-stat.ps.context-switches
2.708e+11 -3.7% 2.607e+11 perf-stat.ps.cpu-cycles
22081 ? 3% +19.4% 26354 ? 6% perf-stat.ps.cpu-migrations
1.389e+10 +31.7% 1.829e+10 perf-stat.ps.dTLB-loads
8.253e+09 +32.3% 1.092e+10 perf-stat.ps.dTLB-stores
37312739 +28.5% 47963236 perf-stat.ps.iTLB-load-misses
6820471 ? 10% -34.2% 4490884 perf-stat.ps.iTLB-loads
4.881e+10 +30.9% 6.388e+10 perf-stat.ps.instructions
238.96 ? 9% +51.7% 362.46 ? 9% perf-stat.ps.major-faults
16479 ? 2% +40.7% 23188 perf-stat.ps.minor-faults
19674114 ? 2% +33.9% 26336370 perf-stat.ps.node-load-misses
39977675 ? 3% +44.1% 57598395 perf-stat.ps.node-loads
10261782 ? 3% +10.2% 11306322 ? 2% perf-stat.ps.node-store-misses
45563654 ? 2% +43.6% 65439023 perf-stat.ps.node-stores
16718 ? 2% +40.9% 23550 perf-stat.ps.page-faults
5.849e+13 -7.5% 5.41e+13 perf-stat.total.instructions
74.65 ?174% -95.2% 3.62 ? 10% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
6.38 ? 37% -90.1% 0.63 ? 10% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.82 ?205% +700.9% 22.58 ? 99% perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
61.66 ? 55% -94.4% 3.45 ? 9% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
55.16 ?190% -94.3% 3.14 ? 6% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
61.35 ?130% -97.5% 1.51 ? 18% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
0.00 ?223% +1.7e+05% 4.82 ? 48% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.switch_task_namespaces.do_exit.do_group_exit
0.05 ?182% +5386.3% 2.80 ? 94% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.03 ?223% +655.6% 0.24 ? 82% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap
642.60 ?104% -100.0% 0.01 perf-sched.sch_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
81.46 ? 26% -78.6% 17.43 ? 2% perf-sched.sch_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
30.24 ? 14% -90.8% 2.78 ? 34% perf-sched.sch_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 ?223% +800.0% 0.00 ? 33% perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
3.97 ?159% +5369.6% 216.94 ? 96% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
51249 ? 10% -87.2% 6550 ? 19% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
48.63 ?215% +6430.7% 3175 ? 99% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
47502 ? 9% -95.6% 2108 ? 97% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
16.80 ? 58% +112.8% 35.76 ? 38% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
336.46 ?141% +532.8% 2129 ? 85% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.shrink_dentry_list.shrink_dcache_parent.d_invalidate
0.00 ?223% +2.6e+05% 7.32 ? 2% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.switch_task_namespaces.do_exit.do_group_exit
0.44 ?207% +2924.7% 13.35 ? 74% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.03 ?223% +12398.4% 3.94 ? 95% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap
10700 ?146% -100.0% 0.01 ? 10% perf-sched.sch_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
50088 ? 11% -87.1% 6447 ? 18% perf-sched.sch_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
51831 ? 10% -86.9% 6804 ? 23% perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.03 ? 74% +1.2e+06% 351.38 ? 99% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.00 ?223% +2750.0% 0.01 ? 78% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
28.77 ? 9% -88.0% 3.44 ? 33% perf-sched.total_sch_delay.average.ms
51831 ? 10% -84.6% 7979 ? 5% perf-sched.total_sch_delay.max.ms
500.44 ? 30% -91.7% 41.45 ? 10% perf-sched.total_wait_and_delay.average.ms
103802 ? 10% -86.6% 13886 ? 24% perf-sched.total_wait_and_delay.max.ms
471.67 ? 31% -91.9% 38.01 ? 8% perf-sched.total_wait_time.average.ms
52524 ? 9% -82.2% 9330 ? 6% perf-sched.total_wait_time.max.ms
7243 ? 40% -97.3% 193.02 ? 10% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
2463 ? 45% -95.0% 122.45 ?100% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
151.19 ? 35% -91.5% 12.91 ? 7% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
2134 ? 40% -96.1% 82.24 ? 36% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
2717 ? 24% -95.6% 120.86 ? 27% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
1830 ? 34% -91.8% 150.13 ? 59% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
2670 ? 45% -95.3% 126.44 ? 32% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
1984 ? 34% -90.7% 184.87 ? 44% perf-sched.wait_and_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
427.53 ? 25% -91.7% 35.57 ? 21% perf-sched.wait_and_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
6459 ? 47% -88.3% 756.04 ? 75% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
2610 ? 30% -94.7% 137.44 ? 67% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
958.17 ? 53% +977.9% 10328 ? 77% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
3.50 ?147% +5057.1% 180.50 ? 90% perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
41.50 ?148% +1483.1% 657.00 ? 82% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg
192.83 ? 32% +612.3% 1373 ? 82% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
83.00 ? 20% +336.7% 362.50 ? 74% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
0.83 ?223% +980.0% 9.00 ? 33% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
1076 ? 47% +884.3% 10591 ? 77% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
76448 ? 32% -87.9% 9266 ? 2% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
52597 ? 37% -93.8% 3243 ?100% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
101251 ? 11% -86.7% 13467 ? 21% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
94618 ? 10% -91.5% 8053 ? 2% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
51519 ? 10% -86.1% 7183 ? 13% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
53886 ? 30% -86.8% 7106 ? 21% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
49146 ? 11% -87.2% 6274 ? 22% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
99599 ? 10% -87.2% 12761 ? 20% perf-sched.wait_and_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
103736 ? 10% -86.7% 13844 ? 24% perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
51474 ? 11% -81.9% 9330 ? 6% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
69804 ? 26% -87.4% 8767 ? 5% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
7024 ? 40% -97.3% 188.98 ? 11% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
2388 ? 42% -94.1% 140.98 ? 70% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
144.81 ? 35% -91.5% 12.28 ? 7% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.89 ?152% +2800.1% 112.68 ? 99% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.shmem_alloc_page
2072 ? 41% -96.2% 78.80 ? 37% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
2661 ? 27% -95.6% 117.73 ? 27% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
0.05 ? 94% +586.5% 0.37 ? 89% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run
0.04 ?150% +2.9e+05% 122.78 ? 93% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.path_put.free_fs_struct
0.00 ?223% +1.7e+05% 4.66 ? 59% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.exit_signals.do_exit.do_group_exit
1768 ? 34% -91.6% 148.62 ? 60% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
0.00 ?223% +8.1e+06% 162.30 ? 99% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mmput.do_exit.do_group_exit
0.21 ?186% +5716.7% 12.26 ? 77% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap
431.11 ? 37% -96.3% 15.87 ? 56% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
2662 ? 45% -95.3% 124.57 ? 32% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
917.87 ? 80% -99.7% 2.66 ? 12% perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
612.52 ? 78% -94.2% 35.51 ? 73% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
1902 ? 35% -91.2% 167.44 ? 48% perf-sched.wait_time.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
397.28 ? 27% -91.7% 32.78 ? 20% perf-sched.wait_time.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
6459 ? 47% -88.3% 755.56 ? 75% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
2571 ? 31% -94.7% 137.15 ? 67% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
51872 ? 10% -85.0% 7779 ? 16% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
44962 ? 16% -92.0% 3613 ? 79% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
52368 ? 9% -84.8% 7967 ? 15% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.17 ?141% +4384.5% 232.00 ? 93% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.shmem_alloc_page
52181 ? 9% -86.2% 7196 ? 14% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
51516 ? 10% -86.1% 7179 ? 13% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
0.24 ?121% +7956.8% 18.95 ? 99% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run
0.10 ?173% +2.3e+06% 2173 ? 98% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.dput.path_put.free_fs_struct
0.00 ?223% +5.2e+05% 13.99 ? 5% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.exit_signals.do_exit.do_group_exit
49218 ? 14% -85.6% 7106 ? 21% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
0.00 ?223% +2.7e+07% 530.74 ? 99% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mmput.do_exit.do_group_exit
0.23 ?171% +1.9e+05% 438.66 ? 96% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap
29872 ? 46% -91.4% 2563 ? 93% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
49135 ? 11% -87.2% 6274 ? 22% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
14166 ?110% -100.0% 3.50 ? 14% perf-sched.wait_time.max.ms.rcu_gp_kthread.kthread.ret_from_fork
52040 ? 10% -84.8% 7888 ? 16% perf-sched.wait_time.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
52228 ? 9% -84.9% 7898 ? 16% perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
51474 ? 11% -81.9% 9330 ? 6% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
51558 ? 10% -86.2% 7136 ? 30% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
90666 ? 10% -42.1% 52520 softirqs.CPU0.RCU
51463 ? 3% -7.9% 47422 softirqs.CPU0.SCHED
90265 ? 7% -40.6% 53641 softirqs.CPU1.RCU
89635 ? 14% -41.2% 52698 softirqs.CPU10.RCU
48032 ? 4% -8.3% 44041 softirqs.CPU10.SCHED
91634 ? 11% -42.0% 53161 softirqs.CPU11.RCU
47798 ? 4% -7.7% 44097 softirqs.CPU11.SCHED
90853 ? 11% -42.3% 52394 softirqs.CPU12.RCU
90341 ? 11% -42.4% 52071 softirqs.CPU13.RCU
90927 ? 11% -46.8% 48391 ? 9% softirqs.CPU14.RCU
90667 ? 11% -42.9% 51795 softirqs.CPU15.RCU
99573 ? 12% -42.6% 57185 ? 3% softirqs.CPU16.RCU
97966 ? 13% -41.4% 57371 ? 3% softirqs.CPU17.RCU
101014 ? 12% -42.0% 58586 ? 4% softirqs.CPU18.RCU
100440 ? 11% -42.3% 57965 ? 3% softirqs.CPU19.RCU
90949 ? 11% -41.8% 52957 softirqs.CPU2.RCU
100584 ? 11% -42.0% 58364 ? 3% softirqs.CPU20.RCU
100354 ? 11% -40.1% 60121 softirqs.CPU21.RCU
99481 ? 11% -42.4% 57252 ? 3% softirqs.CPU22.RCU
99254 ? 11% -41.9% 57620 ? 3% softirqs.CPU23.RCU
97263 ? 11% -40.7% 57647 ? 2% softirqs.CPU24.RCU
96476 ? 12% -41.0% 56898 ? 2% softirqs.CPU25.RCU
96395 ? 11% -41.1% 56742 softirqs.CPU26.RCU
94372 ? 13% -39.9% 56724 ? 2% softirqs.CPU27.RCU
95926 ? 12% -39.6% 57921 softirqs.CPU28.RCU
95279 ? 12% -39.8% 57381 softirqs.CPU29.RCU
90706 ? 11% -41.9% 52660 softirqs.CPU3.RCU
48176 ? 4% -7.3% 44642 softirqs.CPU3.SCHED
94335 ? 10% -39.6% 57009 ? 2% softirqs.CPU30.RCU
96239 ? 12% -40.8% 56966 ? 2% softirqs.CPU31.RCU
90870 ? 12% -41.4% 53255 ? 5% softirqs.CPU32.RCU
90844 ? 12% -41.5% 53125 ? 5% softirqs.CPU33.RCU
90742 ? 12% -41.6% 52956 ? 6% softirqs.CPU34.RCU
44647 ? 3% -8.2% 40988 softirqs.CPU34.SCHED
90796 ? 12% -37.3% 56933 ? 11% softirqs.CPU35.RCU
44182 ? 3% -8.9% 40263 softirqs.CPU35.SCHED
91525 ? 12% -45.6% 49821 ? 13% softirqs.CPU36.RCU
91537 ? 12% -41.4% 53607 ? 5% softirqs.CPU37.RCU
90960 ? 11% -41.2% 53490 ? 5% softirqs.CPU38.RCU
90620 ? 12% -41.5% 53040 ? 5% softirqs.CPU39.RCU
91567 ? 10% -42.2% 52904 softirqs.CPU4.RCU
90523 ? 11% -40.3% 54022 ? 5% softirqs.CPU40.RCU
90743 ? 12% -40.8% 53759 ? 5% softirqs.CPU41.RCU
91862 ? 12% -40.0% 55081 ? 5% softirqs.CPU42.RCU
91010 ? 12% -41.5% 53199 ? 5% softirqs.CPU43.RCU
91228 ? 11% -41.0% 53796 ? 5% softirqs.CPU44.RCU
90426 ? 12% -41.3% 53111 ? 6% softirqs.CPU45.RCU
90685 ? 11% -41.6% 52997 ? 5% softirqs.CPU46.RCU
90912 ? 12% -41.6% 53098 ? 6% softirqs.CPU47.RCU
91490 ? 12% -41.9% 53170 softirqs.CPU48.RCU
89350 ? 8% -40.1% 53536 softirqs.CPU49.RCU
92261 ? 11% -42.9% 52639 softirqs.CPU5.RCU
48355 ? 4% -8.7% 44172 softirqs.CPU5.SCHED
90939 ? 11% -42.8% 51994 softirqs.CPU50.RCU
48293 ? 4% -8.5% 44180 softirqs.CPU50.SCHED
91370 ? 11% -41.9% 53087 softirqs.CPU51.RCU
92263 ? 12% -42.2% 53351 ? 2% softirqs.CPU52.RCU
48457 ? 4% -9.3% 43929 softirqs.CPU52.SCHED
91577 ? 11% -42.3% 52806 ? 2% softirqs.CPU53.RCU
92103 ? 11% -42.4% 53037 ? 2% softirqs.CPU54.RCU
91884 ? 11% -42.2% 53116 ? 2% softirqs.CPU55.RCU
90404 ? 11% -42.8% 51732 ? 2% softirqs.CPU56.RCU
48444 ? 4% -7.1% 44985 softirqs.CPU56.SCHED
91686 ? 13% -41.9% 53236 softirqs.CPU57.RCU
48647 ? 4% -7.6% 44927 softirqs.CPU57.SCHED
90029 ? 14% -41.0% 53128 softirqs.CPU58.RCU
92244 ? 11% -42.4% 53153 ? 2% softirqs.CPU59.RCU
90959 ? 12% -42.6% 52202 softirqs.CPU6.RCU
92044 ? 11% -42.6% 52791 ? 2% softirqs.CPU60.RCU
91630 ? 11% -42.0% 53171 softirqs.CPU61.RCU
48620 ? 4% -7.2% 45114 softirqs.CPU61.SCHED
90861 ? 11% -46.0% 49068 ? 4% softirqs.CPU62.RCU
48682 ? 4% -7.6% 44962 softirqs.CPU62.SCHED
91707 ? 11% -42.4% 52788 softirqs.CPU63.RCU
100669 ? 11% -41.9% 58458 ? 3% softirqs.CPU64.RCU
97372 ? 14% -40.2% 58204 ? 4% softirqs.CPU65.RCU
102174 ? 10% -39.2% 62079 ? 8% softirqs.CPU66.RCU
101076 ? 11% -41.3% 59348 ? 4% softirqs.CPU67.RCU
49027 ? 5% -8.4% 44896 softirqs.CPU67.SCHED
101268 ? 10% -41.7% 59004 ? 3% softirqs.CPU68.RCU
48450 ? 4% -7.5% 44815 softirqs.CPU68.SCHED
102974 ? 12% -40.9% 60814 softirqs.CPU69.RCU
92197 ? 10% -42.8% 52778 softirqs.CPU7.RCU
48087 ? 5% -9.5% 43505 softirqs.CPU7.SCHED
99972 ? 11% -42.1% 57915 ? 3% softirqs.CPU70.RCU
100098 ? 11% -41.4% 58704 ? 3% softirqs.CPU71.RCU
97160 ? 11% -40.6% 57754 ? 2% softirqs.CPU72.RCU
96268 ? 12% -40.2% 57598 softirqs.CPU73.RCU
96164 ? 11% -40.6% 57137 softirqs.CPU74.RCU
94295 ? 13% -39.2% 57325 ? 2% softirqs.CPU75.RCU
95673 ? 11% -40.3% 57103 ? 2% softirqs.CPU76.RCU
94985 ? 11% -38.7% 58209 ? 4% softirqs.CPU77.RCU
94691 ? 9% -39.5% 57304 ? 2% softirqs.CPU78.RCU
96298 ? 11% -40.6% 57235 ? 2% softirqs.CPU79.RCU
91480 ? 11% -42.8% 52334 softirqs.CPU8.RCU
48241 ? 3% -8.0% 44373 softirqs.CPU8.SCHED
90984 ? 11% -41.0% 53650 ? 5% softirqs.CPU80.RCU
91361 ? 11% -41.2% 53727 ? 5% softirqs.CPU81.RCU
91808 ? 12% -40.8% 54316 ? 5% softirqs.CPU82.RCU
92117 ? 12% -40.9% 54474 ? 5% softirqs.CPU83.RCU
90851 ? 12% -42.7% 52046 ? 7% softirqs.CPU84.RCU
90966 ? 12% -41.1% 53568 ? 4% softirqs.CPU85.RCU
91375 ? 11% -41.2% 53737 ? 5% softirqs.CPU86.RCU
91211 ? 12% -40.7% 54047 ? 3% softirqs.CPU87.RCU
92204 ? 12% -40.7% 54685 ? 5% softirqs.CPU88.RCU
44711 ? 3% -8.8% 40770 softirqs.CPU88.SCHED
92197 ? 12% -41.1% 54310 ? 4% softirqs.CPU89.RCU
91370 ? 11% -42.2% 52772 softirqs.CPU9.RCU
47961 ? 4% -7.7% 44262 softirqs.CPU9.SCHED
91228 ? 12% -40.5% 54251 ? 5% softirqs.CPU90.RCU
91678 ? 12% -41.3% 53774 ? 4% softirqs.CPU91.RCU
91366 ? 12% -40.9% 53966 ? 4% softirqs.CPU92.RCU
91670 ? 12% -41.4% 53740 ? 5% softirqs.CPU93.RCU
91846 ? 11% -41.4% 53849 ? 5% softirqs.CPU94.RCU
91251 ? 11% -41.2% 53680 ? 4% softirqs.CPU95.RCU
8973612 ? 11% -41.4% 5254181 ? 3% softirqs.RCU
142298 ? 6% -31.4% 97607 softirqs.TIMER
1171 ? 2% -29.3% 828.00 interrupts.293:PCI-MSI.327680-edge.xhci_hcd
0.67 ?141% +90125.0% 601.50 ? 96% interrupts.36:PCI-MSI.31981569-edge.i40e-eth0-TxRx-0
0.67 ?223% +2.6e+05% 1702 ? 89% interrupts.42:PCI-MSI.31981575-edge.i40e-eth0-TxRx-6
14.17 ?211% +11984.7% 1712 ? 80% interrupts.55:PCI-MSI.31981588-edge.i40e-eth0-TxRx-19
0.33 ?223% +1.9e+05% 617.50 ? 99% interrupts.85:PCI-MSI.31981618-edge.i40e-eth0-TxRx-49
0.00 +9.1e+104% 908.00 ? 98% interrupts.89:PCI-MSI.31981622-edge.i40e-eth0-TxRx-53
9672064 ? 11% -16.1% 8112763 ? 6% interrupts.CAL:Function_call_interrupts
0.33 ?141% +1.8e+05% 600.50 ? 96% interrupts.CPU0.36:PCI-MSI.31981569-edge.i40e-eth0-TxRx-0
108653 ? 35% -51.2% 53039 ? 13% interrupts.CPU0.CAL:Function_call_interrupts
153129 ? 24% -40.9% 90491 interrupts.CPU0.RES:Rescheduling_interrupts
137010 ? 24% -33.8% 90658 interrupts.CPU1.RES:Rescheduling_interrupts
1171 ? 2% -29.3% 828.00 interrupts.CPU10.293:PCI-MSI.327680-edge.xhci_hcd
98261 ? 28% -59.2% 40075 ? 36% interrupts.CPU10.CAL:Function_call_interrupts
8709 -27.4% 6321 ? 8% interrupts.CPU10.NMI:Non-maskable_interrupts
8709 -27.4% 6321 ? 8% interrupts.CPU10.PMI:Performance_monitoring_interrupts
83384 ? 22% -48.0% 43324 ? 19% interrupts.CPU11.CAL:Function_call_interrupts
8679 -27.1% 6330 ? 8% interrupts.CPU11.NMI:Non-maskable_interrupts
8679 -27.1% 6330 ? 8% interrupts.CPU11.PMI:Performance_monitoring_interrupts
126665 ? 17% -31.6% 86592 interrupts.CPU11.RES:Rescheduling_interrupts
137653 ? 22% -36.9% 86888 ? 5% interrupts.CPU12.RES:Rescheduling_interrupts
94196 ? 21% -46.8% 50088 ? 23% interrupts.CPU13.CAL:Function_call_interrupts
8679 -27.4% 6301 ? 9% interrupts.CPU13.NMI:Non-maskable_interrupts
8679 -27.4% 6301 ? 9% interrupts.CPU13.PMI:Performance_monitoring_interrupts
150899 ? 13% -39.7% 90941 interrupts.CPU13.RES:Rescheduling_interrupts
88472 ? 17% -50.4% 43838 ? 15% interrupts.CPU14.CAL:Function_call_interrupts
145702 ? 17% -39.2% 88557 ? 9% interrupts.CPU15.RES:Rescheduling_interrupts
107349 ? 18% -57.2% 45983 ? 17% interrupts.CPU16.CAL:Function_call_interrupts
162249 ? 27% -42.7% 92892 ? 7% interrupts.CPU16.RES:Rescheduling_interrupts
94151 ? 21% -53.2% 44062 ? 38% interrupts.CPU17.CAL:Function_call_interrupts
154164 ? 14% -44.7% 85208 interrupts.CPU17.RES:Rescheduling_interrupts
93957 ? 18% -51.4% 45677 ? 24% interrupts.CPU18.CAL:Function_call_interrupts
156643 ? 17% -42.9% 89440 ? 2% interrupts.CPU18.RES:Rescheduling_interrupts
14.00 ?214% +12128.6% 1712 ? 80% interrupts.CPU19.55:PCI-MSI.31981588-edge.i40e-eth0-TxRx-19
97845 ? 10% -51.5% 47434 ? 7% interrupts.CPU2.CAL:Function_call_interrupts
154321 ? 22% -41.3% 90564 ? 2% interrupts.CPU2.RES:Rescheduling_interrupts
97332 ? 33% -65.1% 33960 ? 16% interrupts.CPU20.CAL:Function_call_interrupts
161508 ? 26% -48.3% 83533 ? 2% interrupts.CPU20.RES:Rescheduling_interrupts
101433 ? 24% -49.8% 50929 ? 39% interrupts.CPU21.CAL:Function_call_interrupts
8678 -27.3% 6309 ? 8% interrupts.CPU21.NMI:Non-maskable_interrupts
8678 -27.3% 6309 ? 8% interrupts.CPU21.PMI:Performance_monitoring_interrupts
146323 ? 23% -40.0% 87782 interrupts.CPU21.RES:Rescheduling_interrupts
89347 ? 16% -50.4% 44339 ? 9% interrupts.CPU22.CAL:Function_call_interrupts
8680 -27.5% 6292 ? 8% interrupts.CPU22.NMI:Non-maskable_interrupts
8680 -27.5% 6292 ? 8% interrupts.CPU22.PMI:Performance_monitoring_interrupts
92971 ? 8% -44.2% 51920 ? 32% interrupts.CPU23.CAL:Function_call_interrupts
8681 -26.8% 6353 ? 10% interrupts.CPU24.NMI:Non-maskable_interrupts
8681 -26.8% 6353 ? 10% interrupts.CPU24.PMI:Performance_monitoring_interrupts
107963 ? 26% +40.5% 151674 ? 30% interrupts.CPU25.CAL:Function_call_interrupts
95907 ? 22% +80.8% 173388 ? 34% interrupts.CPU26.CAL:Function_call_interrupts
8678 -27.0% 6337 ? 10% interrupts.CPU29.NMI:Non-maskable_interrupts
8678 -27.0% 6337 ? 10% interrupts.CPU29.PMI:Performance_monitoring_interrupts
146737 ? 19% -39.4% 88973 ? 2% interrupts.CPU3.RES:Rescheduling_interrupts
82742 ? 23% +42.9% 118237 ? 12% interrupts.CPU30.CAL:Function_call_interrupts
8680 -47.1% 4590 ? 23% interrupts.CPU30.NMI:Non-maskable_interrupts
8680 -47.1% 4590 ? 23% interrupts.CPU30.PMI:Performance_monitoring_interrupts
8678 -26.7% 6361 ? 10% interrupts.CPU32.NMI:Non-maskable_interrupts
8678 -26.7% 6361 ? 10% interrupts.CPU32.PMI:Performance_monitoring_interrupts
101076 ? 19% +21.6% 122921 ? 14% interrupts.CPU33.CAL:Function_call_interrupts
8681 -43.4% 4917 ? 42% interrupts.CPU34.NMI:Non-maskable_interrupts
8681 -43.4% 4917 ? 42% interrupts.CPU34.PMI:Performance_monitoring_interrupts
88529 ? 15% +71.6% 151900 ? 21% interrupts.CPU37.CAL:Function_call_interrupts
91637 ? 22% +42.0% 130151 ? 4% interrupts.CPU39.CAL:Function_call_interrupts
98424 ? 23% -52.7% 46561 ? 18% interrupts.CPU4.CAL:Function_call_interrupts
135853 ? 8% -31.8% 92647 ? 7% interrupts.CPU4.RES:Rescheduling_interrupts
8681 -26.4% 6392 ? 11% interrupts.CPU41.NMI:Non-maskable_interrupts
8681 -26.4% 6392 ? 11% interrupts.CPU41.PMI:Performance_monitoring_interrupts
8691 -27.1% 6334 ? 10% interrupts.CPU42.NMI:Non-maskable_interrupts
8691 -27.1% 6334 ? 10% interrupts.CPU42.PMI:Performance_monitoring_interrupts
85664 ? 16% +70.1% 145682 ? 24% interrupts.CPU44.CAL:Function_call_interrupts
96126 ? 17% +30.4% 125315 ? 18% interrupts.CPU46.CAL:Function_call_interrupts
129280 ? 33% -65.7% 44346 ? 23% interrupts.CPU48.CAL:Function_call_interrupts
0.17 ?223% +3.7e+05% 617.50 ? 99% interrupts.CPU49.85:PCI-MSI.31981618-edge.i40e-eth0-TxRx-49
87848 ? 13% -45.6% 47768 ? 32% interrupts.CPU49.CAL:Function_call_interrupts
8677 -26.9% 6343 ? 9% interrupts.CPU49.NMI:Non-maskable_interrupts
8677 -26.9% 6343 ? 9% interrupts.CPU49.PMI:Performance_monitoring_interrupts
157134 ? 32% -43.1% 89471 ? 5% interrupts.CPU49.RES:Rescheduling_interrupts
94157 ? 9% -55.7% 41675 ? 30% interrupts.CPU5.CAL:Function_call_interrupts
137144 ? 21% -35.0% 89077 ? 2% interrupts.CPU5.RES:Rescheduling_interrupts
96730 ? 30% -45.9% 52337 ? 35% interrupts.CPU50.CAL:Function_call_interrupts
169729 ? 33% -49.1% 86377 ? 2% interrupts.CPU50.RES:Rescheduling_interrupts
85659 ? 17% -44.8% 47268 ? 23% interrupts.CPU51.CAL:Function_call_interrupts
8679 -63.6% 3162 ? 9% interrupts.CPU51.NMI:Non-maskable_interrupts
8679 -63.6% 3162 ? 9% interrupts.CPU51.PMI:Performance_monitoring_interrupts
150936 ? 21% -44.4% 83937 ? 9% interrupts.CPU51.RES:Rescheduling_interrupts
89499 ? 11% -50.0% 44788 ? 33% interrupts.CPU52.CAL:Function_call_interrupts
136534 ? 13% -38.5% 84029 ? 5% interrupts.CPU52.RES:Rescheduling_interrupts
0.00 +9.1e+104% 907.00 ? 98% interrupts.CPU53.89:PCI-MSI.31981622-edge.i40e-eth0-TxRx-53
95165 ? 19% -60.2% 37920 ? 24% interrupts.CPU54.CAL:Function_call_interrupts
8677 -25.4% 6475 ? 5% interrupts.CPU54.NMI:Non-maskable_interrupts
8677 -25.4% 6475 ? 5% interrupts.CPU54.PMI:Performance_monitoring_interrupts
144257 ? 24% -46.1% 77713 interrupts.CPU54.RES:Rescheduling_interrupts
86110 ? 23% -48.8% 44102 ? 28% interrupts.CPU55.CAL:Function_call_interrupts
165553 ? 39% -50.9% 81250 ? 2% interrupts.CPU55.RES:Rescheduling_interrupts
88875 ? 11% -43.7% 49999 ? 23% interrupts.CPU56.CAL:Function_call_interrupts
8678 -26.8% 6348 ? 9% interrupts.CPU56.NMI:Non-maskable_interrupts
8678 -26.8% 6348 ? 9% interrupts.CPU56.PMI:Performance_monitoring_interrupts
152559 ? 29% -41.3% 89528 ? 2% interrupts.CPU56.RES:Rescheduling_interrupts
104697 ? 24% -65.9% 35666 ? 27% interrupts.CPU57.CAL:Function_call_interrupts
93088 ? 27% -49.5% 47014 ? 38% interrupts.CPU58.CAL:Function_call_interrupts
136961 ? 30% -30.7% 94848 ? 2% interrupts.CPU58.RES:Rescheduling_interrupts
89163 ? 25% -55.2% 39970 ? 2% interrupts.CPU59.CAL:Function_call_interrupts
137811 ? 16% -41.7% 80338 ? 3% interrupts.CPU59.RES:Rescheduling_interrupts
0.67 ?223% +2.6e+05% 1702 ? 89% interrupts.CPU6.42:PCI-MSI.31981575-edge.i40e-eth0-TxRx-6
95706 ? 21% -50.2% 47643 interrupts.CPU6.CAL:Function_call_interrupts
93374 ? 20% -53.4% 43499 ? 40% interrupts.CPU60.CAL:Function_call_interrupts
8677 -47.3% 4573 ? 25% interrupts.CPU60.NMI:Non-maskable_interrupts
8677 -47.3% 4573 ? 25% interrupts.CPU60.PMI:Performance_monitoring_interrupts
144391 ? 29% -45.4% 78879 interrupts.CPU60.RES:Rescheduling_interrupts
110191 ? 24% -58.2% 46066 ? 33% interrupts.CPU61.CAL:Function_call_interrupts
161787 ? 23% -52.2% 77254 interrupts.CPU61.RES:Rescheduling_interrupts
92640 ? 21% -48.6% 47661 ? 13% interrupts.CPU62.CAL:Function_call_interrupts
8676 -27.4% 6297 ? 9% interrupts.CPU62.NMI:Non-maskable_interrupts
8676 -27.4% 6297 ? 9% interrupts.CPU62.PMI:Performance_monitoring_interrupts
132790 ? 19% -44.1% 74282 interrupts.CPU62.RES:Rescheduling_interrupts
89631 ? 30% -58.2% 37443 ? 5% interrupts.CPU63.CAL:Function_call_interrupts
156124 ? 26% -44.4% 86878 ? 5% interrupts.CPU63.RES:Rescheduling_interrupts
103552 ? 21% -62.3% 39058 ? 20% interrupts.CPU64.CAL:Function_call_interrupts
166368 ? 28% -51.7% 80391 ? 3% interrupts.CPU64.RES:Rescheduling_interrupts
91047 ? 16% -47.1% 48199 ? 41% interrupts.CPU65.CAL:Function_call_interrupts
153443 ? 22% -52.2% 73270 interrupts.CPU65.RES:Rescheduling_interrupts
96228 ? 15% -53.2% 44992 ? 4% interrupts.CPU66.CAL:Function_call_interrupts
8679 -27.4% 6305 ? 9% interrupts.CPU66.NMI:Non-maskable_interrupts
8679 -27.4% 6305 ? 9% interrupts.CPU66.PMI:Performance_monitoring_interrupts
180396 ? 28% -53.5% 83869 interrupts.CPU66.RES:Rescheduling_interrupts
95624 ? 24% -59.0% 39222 ? 30% interrupts.CPU67.CAL:Function_call_interrupts
103497 ? 27% -55.8% 45774 ? 24% interrupts.CPU68.CAL:Function_call_interrupts
166658 ? 29% -51.0% 81734 interrupts.CPU68.RES:Rescheduling_interrupts
105198 ? 12% -49.1% 53522 ? 11% interrupts.CPU69.CAL:Function_call_interrupts
8676 -27.3% 6308 ? 9% interrupts.CPU69.NMI:Non-maskable_interrupts
8676 -27.3% 6308 ? 9% interrupts.CPU69.PMI:Performance_monitoring_interrupts
139760 ? 23% -44.2% 77968 interrupts.CPU69.RES:Rescheduling_interrupts
92717 ? 30% -51.5% 44928 ? 6% interrupts.CPU7.CAL:Function_call_interrupts
98184 ? 28% -57.0% 42241 ? 38% interrupts.CPU70.CAL:Function_call_interrupts
93861 ? 24% -57.0% 40372 ? 26% interrupts.CPU71.CAL:Function_call_interrupts
136432 ? 18% -40.9% 80625 ? 3% interrupts.CPU71.RES:Rescheduling_interrupts
8677 -27.0% 6335 ? 10% interrupts.CPU73.NMI:Non-maskable_interrupts
8677 -27.0% 6335 ? 10% interrupts.CPU73.PMI:Performance_monitoring_interrupts
8677 -47.0% 4596 ? 24% interrupts.CPU75.NMI:Non-maskable_interrupts
8677 -47.0% 4596 ? 24% interrupts.CPU75.PMI:Performance_monitoring_interrupts
8679 -27.1% 6326 ? 10% interrupts.CPU76.NMI:Non-maskable_interrupts
8679 -27.1% 6326 ? 10% interrupts.CPU76.PMI:Performance_monitoring_interrupts
86491 ? 27% -42.9% 49374 ? 9% interrupts.CPU8.CAL:Function_call_interrupts
8682 -63.3% 3184 ? 9% interrupts.CPU8.NMI:Non-maskable_interrupts
8682 -63.3% 3184 ? 9% interrupts.CPU8.PMI:Performance_monitoring_interrupts
101544 ? 23% +26.6% 128589 ? 15% interrupts.CPU82.CAL:Function_call_interrupts
8680 -47.0% 4603 ? 24% interrupts.CPU85.NMI:Non-maskable_interrupts
8680 -47.0% 4603 ? 24% interrupts.CPU85.PMI:Performance_monitoring_interrupts
8675 -47.1% 4585 ? 23% interrupts.CPU86.NMI:Non-maskable_interrupts
8675 -47.1% 4585 ? 23% interrupts.CPU86.PMI:Performance_monitoring_interrupts
8677 -47.1% 4587 ? 23% interrupts.CPU88.NMI:Non-maskable_interrupts
8677 -47.1% 4587 ? 23% interrupts.CPU88.PMI:Performance_monitoring_interrupts
8679 -26.4% 6384 ? 11% interrupts.CPU89.NMI:Non-maskable_interrupts
8679 -26.4% 6384 ? 11% interrupts.CPU89.PMI:Performance_monitoring_interrupts
86292 ? 14% -39.4% 52293 ? 3% interrupts.CPU9.CAL:Function_call_interrupts
8679 -27.4% 6301 ? 8% interrupts.CPU9.NMI:Non-maskable_interrupts
8679 -27.4% 6301 ? 8% interrupts.CPU9.PMI:Performance_monitoring_interrupts
7232 ? 28% -56.0% 3181 ? 10% interrupts.CPU94.NMI:Non-maskable_interrupts
7232 ? 28% -56.0% 3181 ? 10% interrupts.CPU94.PMI:Performance_monitoring_interrupts
288.00 -33.3% 192.00 interrupts.MCP:Machine_check_polls
763672 -26.8% 558809 ? 8% interrupts.NMI:Non-maskable_interrupts
763672 -26.8% 558809 ? 8% interrupts.PMI:Performance_monitoring_interrupts
12807267 ? 11% -25.5% 9535680 ? 15% interrupts.RES:Rescheduling_interrupts
39.16 -14.8 24.36 ? 4% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
39.22 -14.8 24.45 ? 4% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
40.41 -14.1 26.34 ? 4% perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
12.52 ? 2% -12.5 0.00 perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
20.73 -12.5 8.28 ? 4% perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
19.57 -7.5 12.12 ? 2% perf-profile.calltrace.cycles-pp.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
14.54 -6.2 8.34 ? 2% perf-profile.calltrace.cycles-pp.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
23.30 -5.9 17.39 ? 2% perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
5.72 ? 2% -5.7 0.00 perf-profile.calltrace.cycles-pp.refill_obj_stock.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
5.62 ? 2% -5.6 0.00 perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kfree.consume_skb.unix_stream_read_generic
5.57 -5.6 0.00 perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
47.67 -4.8 42.88 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
47.98 -4.7 43.34 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
48.20 -4.5 43.66 perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
48.38 -4.2 44.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
46.61 -3.9 42.70 perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
46.71 -3.9 42.83 perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
47.21 -3.6 43.60 perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
47.37 -3.5 43.86 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
17.37 -2.7 14.64 ? 4% perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
17.44 -2.7 14.74 ? 4% perf-profile.calltrace.cycles-pp.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
8.71 ? 3% -2.3 6.42 ? 16% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
8.77 ? 3% -2.3 6.48 ? 16% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
8.96 ? 3% -2.2 6.72 ? 15% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
8.94 ? 3% -2.2 6.70 ? 15% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
9.06 ? 3% -2.2 6.82 ? 15% perf-profile.calltrace.cycles-pp.__libc_write
8.52 ? 4% -2.1 6.38 ? 13% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
8.57 ? 4% -2.1 6.43 ? 13% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
8.65 ? 4% -2.1 6.52 ? 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
8.63 ? 4% -2.1 6.50 ? 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
8.76 ? 4% -2.1 6.64 ? 13% perf-profile.calltrace.cycles-pp.__libc_read
0.62 ? 2% +0.0 0.66 ? 3% perf-profile.calltrace.cycles-pp.skb_release_data.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.54 ? 4% +0.2 0.70 ? 4% perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all
1.23 ? 3% +0.2 1.41 ? 4% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
0.61 ? 2% +0.2 0.80 ? 2% perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
0.62 +0.2 0.82 ? 3% perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
1.26 ? 3% +0.2 1.46 ? 4% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
1.41 ? 3% +0.3 1.66 ? 3% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
0.77 ? 2% +0.3 1.09 ? 4% perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
2.08 ? 2% +0.5 2.55 ? 3% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
2.09 ? 2% +0.5 2.58 ? 3% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
2.10 ? 2% +0.5 2.60 ? 3% perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.00 +0.5 0.53 ? 3% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
0.00 +0.5 0.54 ? 4% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
0.00 +0.6 0.56 ? 5% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
0.00 +0.6 0.58 ? 4% perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.00 +0.6 0.62 ? 4% perf-profile.calltrace.cycles-pp.__build_skb_around.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
0.00 +0.6 0.63 ? 7% perf-profile.calltrace.cycles-pp.mutex_unlock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.00 +0.7 0.70 perf-profile.calltrace.cycles-pp.__slab_free.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +0.7 0.70 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_prepare
0.00 +0.7 0.72 perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode
1.86 ? 2% +0.7 2.60 ? 4% perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
0.00 +0.7 0.74 ? 16% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic
0.80 +0.8 1.58 ? 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.00 +0.8 0.81 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.00 +0.8 0.81 ? 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +0.8 0.84 perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.08 ?223% +0.9 0.94 ? 6% perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
1.14 ? 3% +0.9 2.00 ? 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.00 +0.9 0.89 ? 31% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
0.00 +0.9 0.90 ? 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.72 ? 15% +0.9 1.66 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.0 0.95 ? 29% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
0.81 ? 14% +1.0 1.79 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.91 ? 2% +1.0 2.92 ? 4% perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
1.94 ? 2% +1.0 2.96 ? 4% perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
1.93 ? 2% +1.0 2.96 ? 4% perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
0.00 +1.0 1.04 ? 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +1.1 1.06 ? 18% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node
0.00 +1.1 1.06 ? 2% perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.00 +1.1 1.07 perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
0.00 +1.1 1.10 ? 2% perf-profile.calltrace.cycles-pp.__slab_free.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +1.1 1.12 ? 26% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.11 ?223% +1.1 1.25 perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.11 ?223% +1.1 1.26 perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.73 ? 8% +1.2 1.90 perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity
0.74 ? 8% +1.2 1.94 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair
0.69 ? 8% +1.2 1.92 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout
0.75 ? 7% +1.2 2.00 perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.unix_stream_read_generic
0.75 ? 8% +1.2 2.00 perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate
0.73 ? 8% +1.3 2.04 ? 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
0.74 ? 8% +1.3 2.05 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
0.74 ? 8% +1.3 2.06 ? 3% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
0.79 ? 8% +1.3 2.11 perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.89 ? 7% +1.3 2.24 ? 2% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
0.00 +1.4 1.35 ? 27% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
0.00 +1.4 1.36 ? 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
0.92 ? 6% +1.4 2.28 ? 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
0.00 +1.4 1.36 ? 4% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.00 +1.4 1.37 ? 27% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.77 ? 7% +1.4 2.14 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.96 ? 6% +1.4 2.34 ? 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
0.80 ? 7% +1.4 2.20 ? 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.00 ? 6% +1.5 2.46 ? 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.90 ? 8% +1.5 2.38 perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.00 +1.5 1.48 ? 15% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.consume_skb
1.04 ? 6% +1.6 2.62 ? 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
1.01 ? 7% +1.7 2.67 perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.7 1.66 ? 28% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +1.7 1.66 ? 28% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +1.7 1.66 ? 28% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
1.03 ? 7% +1.7 2.70 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +1.7 1.72 ? 29% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
1.14 ? 6% +1.8 2.90 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
0.00 +1.9 1.86 ? 11% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.9 1.86 ? 11% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.9 1.86 ? 11% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.66 ? 8% +1.9 2.52 perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
0.68 ? 8% +1.9 2.56 perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
0.68 ? 8% +1.9 2.56 perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
0.08 ?223% +2.0 2.08 ? 18% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller
0.00 +2.0 2.04 ? 13% perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
1.57 ? 7% +2.5 4.09 perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
1.60 ? 7% +2.5 4.12 perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
0.00 +4.2 4.23 ? 3% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic
2.78 ? 6% +4.5 7.28 perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
2.80 ? 6% +4.5 7.32 perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
2.83 ? 6% +4.5 7.37 perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.17 ?223% +4.6 4.74 perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
1.18 ? 6% +4.7 5.89 ? 7% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb
3.29 ? 15% +4.9 8.22 ? 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
3.31 ? 15% +5.0 8.26 ? 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
3.35 ? 15% +5.0 8.34 ? 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
1.32 ? 5% +5.0 6.36 ? 6% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
1.37 ? 5% +5.1 6.45 ? 6% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
0.00 +5.1 5.14 ? 5% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg
2.66 ? 8% +5.1 7.80 perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
2.68 ? 8% +5.2 7.84 perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
2.88 ? 7% +5.4 8.24 perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
0.00 +5.6 5.58 ? 4% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
3.49 ? 11% +6.0 9.52 ? 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
4.02 ? 10% +7.0 10.97 ? 5% perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.00 +8.3 8.28 ? 2% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.consume_skb
2.63 ? 13% +9.0 11.59 ? 6% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
3.38 ? 5% +9.3 12.64 ? 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
3.22 ? 11% +9.3 12.51 ? 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
0.00 +10.1 10.07 ? 4% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.consume_skb.unix_stream_read_generic
0.00 +11.0 10.96 ? 3% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
18.12 ? 2% -17.5 0.60 ? 8% perf-profile.children.cycles-pp.get_obj_cgroup_from_current
39.18 -14.8 24.38 ? 4% perf-profile.children.cycles-pp.__alloc_skb
39.22 -14.8 24.46 ? 4% perf-profile.children.cycles-pp.alloc_skb_with_frags
40.41 -14.1 26.35 ? 4% perf-profile.children.cycles-pp.sock_alloc_send_pskb
20.76 -12.4 8.32 ? 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node
9.95 -9.8 0.17 perf-profile.children.cycles-pp.refill_obj_stock
9.77 -9.8 0.00 perf-profile.children.cycles-pp.drain_obj_stock
19.57 -7.4 12.14 ? 2% perf-profile.children.cycles-pp.kfree
23.30 -5.9 17.39 ? 2% perf-profile.children.cycles-pp.consume_skb
14.56 -5.9 8.68 ? 2% perf-profile.children.cycles-pp.kmem_cache_free
47.69 -4.8 42.92 perf-profile.children.cycles-pp.unix_stream_sendmsg
48.00 -4.6 43.35 perf-profile.children.cycles-pp.sock_sendmsg
48.20 -4.5 43.66 perf-profile.children.cycles-pp.sock_write_iter
48.39 -4.2 44.16 perf-profile.children.cycles-pp.new_sync_write
48.96 -4.0 44.98 perf-profile.children.cycles-pp.vfs_write
46.63 -3.9 42.72 perf-profile.children.cycles-pp.unix_stream_read_generic
49.27 -3.9 45.40 perf-profile.children.cycles-pp.ksys_write
46.71 -3.9 42.84 perf-profile.children.cycles-pp.unix_stream_recvmsg
47.21 -3.6 43.60 perf-profile.children.cycles-pp.sock_read_iter
47.38 -3.5 43.87 perf-profile.children.cycles-pp.new_sync_read
5.10 -3.4 1.72 ? 4% perf-profile.children.cycles-pp.mod_objcg_state
48.06 -3.2 44.86 perf-profile.children.cycles-pp.vfs_read
98.81 -3.1 95.68 perf-profile.children.cycles-pp.do_syscall_64
48.31 -3.1 45.20 perf-profile.children.cycles-pp.ksys_read
98.97 -3.0 95.92 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
17.41 -2.7 14.70 ? 4% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
17.45 -2.7 14.76 ? 4% perf-profile.children.cycles-pp.kmalloc_reserve
9.08 ? 3% -2.2 6.85 ? 15% perf-profile.children.cycles-pp.__libc_write
8.79 ? 4% -2.1 6.67 ? 13% perf-profile.children.cycles-pp.__libc_read
2.45 ? 3% -1.4 1.04 ? 11% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg
0.08 ? 6% +0.0 0.09 perf-profile.children.cycles-pp.check_stack_object
0.42 +0.0 0.44 perf-profile.children.cycles-pp.task_tick_fair
0.07 ? 5% +0.0 0.09 perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.06 ? 7% +0.0 0.08 ? 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.06 ? 8% +0.0 0.08 ? 5% perf-profile.children.cycles-pp.get_stack_info
0.06 ? 8% +0.0 0.09 perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.04 ? 72% +0.0 0.08 ? 12% perf-profile.children.cycles-pp.mem_cgroup_from_task
0.13 ? 3% +0.0 0.16 ? 3% perf-profile.children.cycles-pp.aa_file_perm
0.06 ? 6% +0.0 0.10 perf-profile.children.cycles-pp.security_socket_getpeersec_dgram
0.10 ? 4% +0.0 0.14 perf-profile.children.cycles-pp.__task_pid_nr_ns
0.07 ? 7% +0.0 0.11 perf-profile.children.cycles-pp.rcu_all_qs
0.00 +0.1 0.05 perf-profile.children.cycles-pp.kmalloc_slab
0.00 +0.1 0.05 perf-profile.children.cycles-pp.build_id__mark_dso_hit
0.00 +0.1 0.05 perf-profile.children.cycles-pp.__calc_delta
0.00 +0.1 0.05 perf-profile.children.cycles-pp.__mmdrop
0.00 +0.1 0.05 perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.05 perf-profile.children.cycles-pp.__dentry_kill
0.00 +0.1 0.05 perf-profile.children.cycles-pp.d_invalidate
0.45 +0.1 0.50 ? 2% perf-profile.children.cycles-pp.scheduler_tick
0.00 +0.1 0.06 ? 9% perf-profile.children.cycles-pp.process_one_work
0.00 +0.1 0.06 ? 9% perf-profile.children.cycles-pp.update_min_vruntime
0.00 +0.1 0.06 ? 9% perf-profile.children.cycles-pp.machines__deliver_event
0.00 +0.1 0.06 ? 9% perf-profile.children.cycles-pp.___perf_sw_event
0.00 +0.1 0.06 ? 9% perf-profile.children.cycles-pp.copy_fpregs_to_fpstate
0.01 ?223% +0.1 0.06 ? 7% perf-profile.children.cycles-pp.perf_tp_event_match
0.00 +0.1 0.06 perf-profile.children.cycles-pp.get_callchain_entry
0.00 +0.1 0.06 perf-profile.children.cycles-pp.__is_insn_slot_addr
0.00 +0.1 0.06 perf-profile.children.cycles-pp.perf_trace_buf_update
0.00 +0.1 0.06 perf-profile.children.cycles-pp.get_stack_info_noinstr
0.00 +0.1 0.06 ? 16% perf-profile.children.cycles-pp.ktime_get
0.11 +0.1 0.17 perf-profile.children.cycles-pp.perf_event_pid_type
0.08 +0.1 0.14 perf-profile.children.cycles-pp.__update_load_avg_se
0.01 ?223% +0.1 0.07 perf-profile.children.cycles-pp.perf_misc_flags
0.15 ? 4% +0.1 0.22 ? 2% perf-profile.children.cycles-pp.__might_fault
0.10 ? 4% +0.1 0.16 perf-profile.children.cycles-pp.rcu_read_unlock_strict
0.00 +0.1 0.06 ? 23% perf-profile.children.cycles-pp.ordered_events__queue
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.__free_pages_ok
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.pick_next_entity
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.proc_invalidate_siblings_dcache
0.08 ? 6% +0.1 0.14 perf-profile.children.cycles-pp.native_sched_clock
0.00 +0.1 0.07 ? 28% perf-profile.children.cycles-pp.sched_ttwu_pending
0.00 +0.1 0.07 ? 28% perf-profile.children.cycles-pp.kmem_cache_alloc
0.00 +0.1 0.07 ? 14% perf-profile.children.cycles-pp.clockevents_program_event
0.00 +0.1 0.07 perf-profile.children.cycles-pp.put_prev_entity
0.17 ? 2% +0.1 0.24 ? 2% perf-profile.children.cycles-pp.__ksize
0.09 ? 4% +0.1 0.16 ? 6% perf-profile.children.cycles-pp.sched_clock_cpu
0.09 ? 5% +0.1 0.16 ? 3% perf-profile.children.cycles-pp.__switch_to_asm
0.10 ? 4% +0.1 0.17 ? 5% perf-profile.children.cycles-pp.set_next_entity
0.00 +0.1 0.08 ? 33% perf-profile.children.cycles-pp.sysvec_call_function_single
0.00 +0.1 0.08 ? 33% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.05 ? 7% +0.1 0.13 perf-profile.children.cycles-pp.cpuacct_charge
0.05 ? 45% +0.1 0.12 ? 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.12 ? 4% +0.1 0.20 ? 2% perf-profile.children.cycles-pp.wait_for_unix_gc
0.07 +0.1 0.15 perf-profile.children.cycles-pp.__list_del_entry_valid
0.08 ? 5% +0.1 0.16 ? 3% perf-profile.children.cycles-pp.__switch_to
0.00 +0.1 0.08 ? 37% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.00 +0.1 0.08 perf-profile.children.cycles-pp.bpf_ksym_find
0.00 +0.1 0.08 ? 25% perf-profile.children.cycles-pp.smpboot_thread_fn
0.00 +0.1 0.08 ? 25% perf-profile.children.cycles-pp.run_ksoftirqd
0.51 +0.1 0.59 ? 3% perf-profile.children.cycles-pp.tick_sched_handle
0.05 ? 8% +0.1 0.14 ? 3% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.17 ? 3% +0.1 0.26 perf-profile.children.cycles-pp.__cond_resched
0.00 +0.1 0.08 ? 5% perf-profile.children.cycles-pp.kvm_is_in_guest
0.00 +0.1 0.08 ? 5% perf-profile.children.cycles-pp.unlink_file_vma
0.08 ? 6% +0.1 0.16 perf-profile.children.cycles-pp.perf_output_begin_forward
0.52 +0.1 0.60 ? 4% perf-profile.children.cycles-pp.tick_sched_timer
0.09 ? 5% +0.1 0.18 perf-profile.children.cycles-pp.load_new_mm_cr3
0.07 ? 10% +0.1 0.16 ? 3% perf-profile.children.cycles-pp.update_rq_clock
0.01 ?223% +0.1 0.10 ? 5% perf-profile.children.cycles-pp.check_preempt_wakeup
0.49 +0.1 0.58 ? 3% perf-profile.children.cycles-pp.update_process_times
0.26 ? 2% +0.1 0.34 ? 10% perf-profile.children.cycles-pp.__virt_addr_valid
0.00 +0.1 0.09 ? 11% perf-profile.children.cycles-pp.worker_thread
0.27 ? 3% +0.1 0.36 perf-profile.children.cycles-pp.__entry_text_start
0.08 ? 8% +0.1 0.18 ? 2% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.24 ? 2% +0.1 0.34 perf-profile.children.cycles-pp.security_socket_sendmsg
0.00 +0.1 0.10 ? 15% perf-profile.children.cycles-pp.filemap_map_pages
0.00 +0.1 0.10 ? 5% perf-profile.children.cycles-pp.uncharge_batch
0.23 +0.1 0.32 perf-profile.children.cycles-pp.__might_sleep
0.01 ?223% +0.1 0.10 ? 4% perf-profile.children.cycles-pp.bad_get_user
0.00 +0.1 0.10 ? 40% perf-profile.children.cycles-pp.copy_page_range
0.00 +0.1 0.10 ? 10% perf-profile.children.cycles-pp.release_task
0.09 ? 4% +0.1 0.19 perf-profile.children.cycles-pp.reweight_entity
0.01 ?223% +0.1 0.11 perf-profile.children.cycles-pp.is_bpf_text_address
0.56 +0.1 0.66 ? 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.10 ? 4% +0.1 0.20 ? 2% perf-profile.children.cycles-pp.cmp_ex_search
0.02 ?141% +0.1 0.12 ? 4% perf-profile.children.cycles-pp.check_preempt_curr
0.00 +0.1 0.11 ? 18% perf-profile.children.cycles-pp.do_fault
0.00 +0.1 0.11 perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.00 +0.1 0.11 ? 9% perf-profile.children.cycles-pp.wait_task_zombie
0.08 ? 7% +0.1 0.20 ? 7% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.00 +0.1 0.12 ? 13% perf-profile.children.cycles-pp.irq_exit_rcu
0.00 +0.1 0.12 ? 13% perf-profile.children.cycles-pp.rcu_do_batch
0.17 ? 6% +0.1 0.28 perf-profile.children.cycles-pp.switch_fpu_return
0.00 +0.1 0.12 perf-profile.children.cycles-pp.remove_vma
0.20 ? 2% +0.1 0.32 perf-profile.children.cycles-pp.__perf_event_header__init_id
0.41 ? 2% +0.1 0.54 perf-profile.children.cycles-pp.__fget_light
0.00 +0.1 0.12 ? 12% perf-profile.children.cycles-pp.rcu_core
0.28 ? 2% +0.1 0.41 perf-profile.children.cycles-pp.__check_heap_object
0.01 ?223% +0.1 0.14 ? 3% perf-profile.children.cycles-pp.unfreeze_partials
0.00 +0.1 0.13 ? 38% perf-profile.children.cycles-pp.__alloc_pages
0.00 +0.1 0.13 ? 7% perf-profile.children.cycles-pp.perf_session__deliver_event
0.06 ? 7% +0.1 0.20 ? 4% perf-profile.children.cycles-pp.update_cfs_group
0.02 ?141% +0.1 0.16 ? 3% perf-profile.children.cycles-pp.put_cpu_partial
0.00 +0.1 0.14 ? 7% perf-profile.children.cycles-pp.perf_session__process_user_event
0.00 +0.1 0.14 ? 7% perf-profile.children.cycles-pp.__ordered_events__flush
0.46 ? 2% +0.1 0.60 perf-profile.children.cycles-pp.__fdget_pos
0.25 ? 2% +0.1 0.40 perf-profile.children.cycles-pp.security_socket_recvmsg
0.30 ? 4% +0.1 0.44 perf-profile.children.cycles-pp.fsnotify
0.64 ? 2% +0.2 0.80 ? 5% perf-profile.children.cycles-pp.hrtimer_interrupt
0.64 ? 2% +0.2 0.80 ? 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.15 ? 2% +0.2 0.31 ? 3% perf-profile.children.cycles-pp.drain_stock
0.33 ? 2% +0.2 0.50 ? 13% perf-profile.children.cycles-pp.copyin
0.66 ? 3% +0.2 0.82 ? 2% perf-profile.children.cycles-pp.unix_write_space
0.41 ? 3% +0.2 0.57 perf-profile.children.cycles-pp.common_file_perm
0.52 ? 3% +0.2 0.69 perf-profile.children.cycles-pp._copy_from_iter
0.00 +0.2 0.17 ? 5% perf-profile.children.cycles-pp.kthread
0.00 +0.2 0.17 ? 5% perf-profile.children.cycles-pp.__softirqentry_text_start
1.53 ? 3% +0.2 1.70 perf-profile.children.cycles-pp.copyout
0.30 ? 2% +0.2 0.47 ? 2% perf-profile.children.cycles-pp.sock_recvmsg
0.56 ? 3% +0.2 0.73 perf-profile.children.cycles-pp.__build_skb_around
0.39 ? 3% +0.2 0.56 perf-profile.children.cycles-pp.mutex_lock
0.15 ? 5% +0.2 0.32 perf-profile.children.cycles-pp.bsearch
0.34 ? 3% +0.2 0.52 perf-profile.children.cycles-pp.___might_sleep
0.23 ? 3% +0.2 0.41 ? 2% perf-profile.children.cycles-pp.pick_next_task_fair
0.38 +0.2 0.56 perf-profile.children.cycles-pp.aa_sk_perm
0.00 +0.2 0.19 ? 21% perf-profile.children.cycles-pp.wp_page_copy
0.00 +0.2 0.19 perf-profile.children.cycles-pp.ret_from_fork
0.15 ? 3% +0.2 0.34 perf-profile.children.cycles-pp.search_extable
0.47 ? 3% +0.2 0.66 perf-profile.children.cycles-pp.syscall_return_via_sysret
0.17 ? 4% +0.2 0.36 ? 4% perf-profile.children.cycles-pp.refill_stock
0.16 ? 4% +0.2 0.35 perf-profile.children.cycles-pp.search_exception_tables
0.00 +0.2 0.20 ? 33% perf-profile.children.cycles-pp.schedule_idle
0.00 +0.2 0.20 perf-profile.children.cycles-pp.__memcg_kmem_uncharge_page
0.76 ? 2% +0.2 0.96 perf-profile.children.cycles-pp.simple_copy_to_iter
0.00 +0.2 0.20 ? 7% perf-profile.children.cycles-pp.unlink_anon_vmas
0.00 +0.2 0.21 ? 4% perf-profile.children.cycles-pp.put_task_stack
0.18 ? 3% +0.2 0.40 perf-profile.children.cycles-pp.fixup_exception
0.00 +0.2 0.22 ? 4% perf-profile.children.cycles-pp.process_simple
0.00 +0.2 0.22 ? 33% perf-profile.children.cycles-pp.dup_mmap
0.10 ? 3% +0.2 0.33 ? 9% perf-profile.children.cycles-pp.prepare_to_wait
0.00 +0.2 0.24 ? 4% perf-profile.children.cycles-pp.cmd_record
0.00 +0.2 0.24 ? 4% perf-profile.children.cycles-pp.record__finish_output
0.00 +0.2 0.24 ? 4% perf-profile.children.cycles-pp.perf_session__process_events
0.21 ? 4% +0.2 0.46 perf-profile.children.cycles-pp.kernelmode_fixup_or_oops
1.71 ? 2% +0.2 1.95 ? 2% perf-profile.children.cycles-pp._copy_to_iter
0.00 +0.2 0.24 ? 34% perf-profile.children.cycles-pp.dup_mm
0.00 +0.2 0.25 ? 4% perf-profile.children.cycles-pp.page_remove_rmap
0.60 ? 2% +0.3 0.86 perf-profile.children.cycles-pp.security_file_permission
0.68 ? 2% +0.3 0.94 ? 6% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.28 ? 5% +0.3 0.56 perf-profile.children.cycles-pp.get_partial_node
0.00 +0.3 0.29 ? 3% perf-profile.children.cycles-pp.wait_consider_task
0.19 ? 4% +0.3 0.48 perf-profile.children.cycles-pp.kernel_text_address
0.00 +0.3 0.30 ? 42% perf-profile.children.cycles-pp.queued_write_lock_slowpath
0.27 ? 4% +0.3 0.56 perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ? 6% +0.3 0.50 perf-profile.children.cycles-pp.stack_access_ok
0.72 ? 4% +0.3 1.02 ? 6% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.14 ? 4% +0.3 0.44 ? 2% perf-profile.children.cycles-pp.try_charge
0.00 +0.3 0.30 ? 3% perf-profile.children.cycles-pp.release_pages
1.83 ? 2% +0.3 2.14 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.00 +0.3 0.32 ? 7% perf-profile.children.cycles-pp.free_pgtables
0.00 +0.3 0.32 ? 34% perf-profile.children.cycles-pp.copy_process
0.95 ? 2% +0.3 1.28 perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
0.00 +0.3 0.34 ? 2% perf-profile.children.cycles-pp.tlb_flush_mmu
0.00 +0.3 0.34 ? 36% perf-profile.children.cycles-pp.__do_sys_clone
0.00 +0.3 0.34 ? 36% perf-profile.children.cycles-pp.kernel_clone
1.11 +0.3 1.46 perf-profile.children.cycles-pp.__check_object_size
0.19 ? 3% +0.3 0.54 ? 6% perf-profile.children.cycles-pp.update_load_avg
0.00 +0.4 0.35 ? 2% perf-profile.children.cycles-pp.tlb_finish_mmu
0.00 +0.4 0.36 ? 21% perf-profile.children.cycles-pp.__handle_mm_fault
0.25 ? 4% +0.4 0.62 perf-profile.children.cycles-pp.__kernel_text_address
0.00 +0.4 0.38 ? 20% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
0.00 +0.4 0.38 ? 23% perf-profile.children.cycles-pp.handle_mm_fault
0.25 ? 6% +0.4 0.64 ? 5% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.04 ? 44% +0.4 0.45 ? 6% perf-profile.children.cycles-pp.finish_task_switch
0.00 +0.4 0.42 ? 36% perf-profile.children.cycles-pp.__libc_fork
0.00 +0.4 0.42 ? 19% perf-profile.children.cycles-pp.set_task_cpu
0.32 ? 4% +0.4 0.76 ? 4% perf-profile.children.cycles-pp.mutex_unlock
0.44 ? 5% +0.4 0.88 ? 2% perf-profile.children.cycles-pp.___slab_alloc
0.45 ? 4% +0.4 0.90 ? 2% perf-profile.children.cycles-pp.__slab_alloc
2.52 ? 2% +0.5 2.98 perf-profile.children.cycles-pp.__skb_datagram_iter
0.00 +0.5 0.46 ? 2% perf-profile.children.cycles-pp.__do_sys_wait4
0.00 +0.5 0.46 ? 2% perf-profile.children.cycles-pp.kernel_wait4
0.00 +0.5 0.46 ? 2% perf-profile.children.cycles-pp.do_wait
2.54 ? 2% +0.5 3.00 perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.00 +0.5 0.46 ? 3% perf-profile.children.cycles-pp.__libc_wait
0.32 ? 4% +0.5 0.79 ? 3% perf-profile.children.cycles-pp.orc_find
0.32 ? 3% +0.5 0.78 perf-profile.children.cycles-pp.unwind_get_return_address
0.00 +0.5 0.46 ? 20% perf-profile.children.cycles-pp.do_user_addr_fault
2.56 ? 2% +0.5 3.02 perf-profile.children.cycles-pp.unix_stream_read_actor
0.60 ? 2% +0.5 1.10 ? 3% perf-profile.children.cycles-pp.skb_set_owner_w
0.00 +0.5 0.54 ? 4% perf-profile.children.cycles-pp.zap_pte_range
0.00 +0.5 0.54 ? 4% perf-profile.children.cycles-pp.unmap_page_range
0.00 +0.6 0.56 ? 5% perf-profile.children.cycles-pp.unmap_vmas
0.18 ? 22% +0.6 0.80 ? 10% perf-profile.children.cycles-pp.__mod_memcg_state
0.35 ? 4% +0.6 0.99 ? 2% perf-profile.children.cycles-pp.skb_queue_tail
0.26 ? 15% +0.7 0.94 ? 8% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.28 ? 3% +0.7 1.00 ? 9% perf-profile.children.cycles-pp.exc_page_fault
2.26 ? 2% +0.8 3.04 perf-profile.children.cycles-pp.sock_wfree
2.33 ? 2% +0.8 3.13 perf-profile.children.cycles-pp.unix_destruct_scm
2.36 ? 2% +0.8 3.16 perf-profile.children.cycles-pp.skb_release_all
2.35 ? 2% +0.8 3.16 perf-profile.children.cycles-pp.skb_release_head_state
0.61 ? 3% +0.8 1.42 perf-profile.children.cycles-pp.__orc_find
0.44 ? 4% +0.8 1.26 perf-profile.children.cycles-pp.skb_unlink
0.91 ? 11% +1.0 1.92 perf-profile.children.cycles-pp.exit_to_user_mode_prepare
1.02 ? 10% +1.0 2.07 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.00 +1.2 1.17 ? 28% perf-profile.children.cycles-pp.intel_idle
0.80 ? 2% +1.4 2.16 perf-profile.children.cycles-pp.__slab_free
0.00 +1.4 1.36 ? 5% perf-profile.children.cycles-pp.exit_mmap
0.00 +1.4 1.36 ? 4% perf-profile.children.cycles-pp.mmput
0.00 +1.4 1.41 ? 28% perf-profile.children.cycles-pp.cpuidle_enter_state
0.00 +1.4 1.42 ? 28% perf-profile.children.cycles-pp.cpuidle_enter
0.84 ? 4% +1.5 2.33 ? 4% perf-profile.children.cycles-pp.asm_exc_page_fault
1.15 ? 3% +1.5 2.66 perf-profile.children.cycles-pp.__unwind_start
1.15 ? 3% +1.5 2.70 ? 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
1.40 ? 4% +1.6 3.00 perf-profile.children.cycles-pp.__get_user_nocheck_8
1.28 ? 3% +1.6 2.91 perf-profile.children.cycles-pp.enqueue_entity
0.00 +1.7 1.66 ? 28% perf-profile.children.cycles-pp.start_secondary
1.52 ? 2% +1.7 3.22 ? 2% perf-profile.children.cycles-pp.dequeue_entity
1.49 ? 4% +1.7 3.21 perf-profile.children.cycles-pp.perf_callchain_user
0.00 +1.7 1.72 ? 29% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
0.00 +1.7 1.72 ? 29% perf-profile.children.cycles-pp.cpu_startup_entry
0.00 +1.7 1.72 ? 29% perf-profile.children.cycles-pp.do_idle
1.43 ? 3% +1.8 3.26 perf-profile.children.cycles-pp.enqueue_task_fair
1.46 ? 3% +1.8 3.30 perf-profile.children.cycles-pp.ttwu_do_activate
1.46 ? 6% +1.9 3.32 ? 3% perf-profile.children.cycles-pp.perf_trace_sched_switch
0.00 +1.9 1.87 ? 11% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.00 +1.9 1.87 ? 11% perf-profile.children.cycles-pp.do_group_exit
0.00 +1.9 1.87 ? 11% perf-profile.children.cycles-pp.do_exit
1.66 ? 2% +1.9 3.56 ? 2% perf-profile.children.cycles-pp.dequeue_task_fair
0.84 ? 4% +2.8 3.64 ? 11% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.08 ? 20% +2.8 2.90 ? 16% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
2.80 ? 2% +2.9 5.66 perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
2.14 ? 4% +3.0 5.12 perf-profile.children.cycles-pp.unwind_next_frame
2.97 ? 2% +3.1 6.12 perf-profile.children.cycles-pp.update_curr
2.63 ? 3% +3.7 6.30 ? 2% perf-profile.children.cycles-pp._raw_spin_lock
2.82 ? 4% +3.9 6.71 perf-profile.children.cycles-pp.perf_callchain_kernel
3.48 ? 2% +4.4 7.93 perf-profile.children.cycles-pp.schedule_timeout
3.87 ? 10% +4.9 8.73 ? 2% perf-profile.children.cycles-pp.autoremove_wake_function
3.84 ? 10% +4.9 8.72 ? 2% perf-profile.children.cycles-pp.try_to_wake_up
3.91 ? 10% +4.9 8.80 ? 2% perf-profile.children.cycles-pp.__wake_up_common
1.53 ? 7% +5.3 6.82 ? 14% perf-profile.children.cycles-pp.propagate_protected_usage
4.04 ? 4% +5.3 9.35 perf-profile.children.cycles-pp.schedule
4.06 ? 4% +5.5 9.59 ? 2% perf-profile.children.cycles-pp.__schedule
4.42 ? 4% +5.7 10.11 perf-profile.children.cycles-pp.get_perf_callchain
4.45 ? 4% +5.7 10.16 perf-profile.children.cycles-pp.perf_callchain
4.79 ? 3% +5.9 10.72 perf-profile.children.cycles-pp.perf_prepare_sample
3.97 ? 10% +6.0 10.00 ? 5% perf-profile.children.cycles-pp.__wake_up_common_lock
4.93 ? 4% +6.4 11.28 ? 3% perf-profile.children.cycles-pp.perf_event_output_forward
5.03 ? 4% +6.4 11.39 ? 3% perf-profile.children.cycles-pp.perf_swevent_overflow
4.99 ? 4% +6.4 11.35 ? 3% perf-profile.children.cycles-pp.__perf_event_overflow
5.19 ? 3% +6.5 11.70 ? 3% perf-profile.children.cycles-pp.perf_tp_event
4.02 ? 10% +7.0 10.97 ? 5% perf-profile.children.cycles-pp.sock_def_readable
2.74 ? 9% +10.2 12.93 ? 2% perf-profile.children.cycles-pp.page_counter_cancel
5.47 ? 3% +11.5 16.93 ? 3% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
3.39 ? 7% +12.6 15.94 ? 4% perf-profile.children.cycles-pp.page_counter_uncharge
4.32 ? 6% +13.6 17.88 ? 6% perf-profile.children.cycles-pp.page_counter_try_charge
4.91 ? 5% +14.0 18.92 ? 5% perf-profile.children.cycles-pp.obj_cgroup_charge_pages
5.05 ? 5% +14.1 19.10 ? 5% perf-profile.children.cycles-pp.obj_cgroup_charge
17.91 ? 2% -17.4 0.49 ? 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current
11.30 ? 2% -10.7 0.62 perf-profile.self.cycles-pp.kfree
7.86 -6.9 0.94 perf-profile.self.cycles-pp.kmem_cache_free
7.66 ? 2% -6.9 0.76 perf-profile.self.cycles-pp.__kmalloc_node_track_caller
5.86 ? 3% -5.3 0.52 perf-profile.self.cycles-pp.kmem_cache_alloc_node
4.83 -4.0 0.80 perf-profile.self.cycles-pp.mod_objcg_state
2.44 ? 3% -1.4 1.02 ? 11% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg
0.36 ? 44% -0.2 0.18 perf-profile.self.cycles-pp.select_idle_cpu
0.05 +0.0 0.06 perf-profile.self.cycles-pp.__skb_datagram_iter
0.06 ? 8% +0.0 0.07 perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.07 ? 5% +0.0 0.08 ? 5% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.05 +0.0 0.07 perf-profile.self.cycles-pp.__fdget_pos
0.06 ? 7% +0.0 0.08 ? 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.05 ? 13% +0.0 0.08 ? 20% perf-profile.self.cycles-pp.perf_event_output_forward
0.06 ? 6% +0.0 0.08 perf-profile.self.cycles-pp.unix_destruct_scm
0.05 +0.0 0.08 ? 6% perf-profile.self.cycles-pp.sock_recvmsg
0.05 ? 7% +0.0 0.08 perf-profile.self.cycles-pp.reweight_entity
0.04 ? 73% +0.0 0.07 ? 14% perf-profile.self.cycles-pp.mem_cgroup_from_task
0.05 ? 8% +0.0 0.08 ? 5% perf-profile.self.cycles-pp.ksys_write
0.08 ? 4% +0.0 0.11 perf-profile.self.cycles-pp.unix_stream_recvmsg
0.11 ? 4% +0.0 0.14 ? 3% perf-profile.self.cycles-pp.aa_file_perm
0.05 ? 7% +0.0 0.08 ? 5% perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
0.04 ? 44% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.ksys_read
0.05 +0.0 0.08 ? 5% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.12 ? 5% +0.0 0.16 ? 3% perf-profile.self.cycles-pp._copy_to_iter
0.09 ? 4% +0.0 0.12 ? 4% perf-profile.self.cycles-pp.security_file_permission
0.06 ? 7% +0.0 0.10 perf-profile.self.cycles-pp.__cond_resched
0.03 ? 70% +0.0 0.07 perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.04 ? 44% +0.0 0.08 perf-profile.self.cycles-pp.rcu_all_qs
0.12 ? 4% +0.0 0.16 ? 3% perf-profile.self.cycles-pp._copy_from_iter
0.06 ? 7% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.sock_sendmsg
0.06 ? 7% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.enqueue_task_fair
0.06 +0.0 0.10 perf-profile.self.cycles-pp.rcu_read_unlock_strict
0.09 ? 4% +0.0 0.13 perf-profile.self.cycles-pp.sock_alloc_send_pskb
0.09 ? 5% +0.0 0.14 ? 3% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.03 ? 70% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.alloc_skb_with_frags
0.05 ? 7% +0.0 0.10 ? 5% perf-profile.self.cycles-pp.enqueue_entity
0.09 +0.0 0.14 ? 3% perf-profile.self.cycles-pp.do_syscall_64
0.04 ? 45% +0.0 0.09 perf-profile.self.cycles-pp.dequeue_task_fair
0.15 ? 3% +0.0 0.20 ? 2% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.10 ? 7% +0.1 0.16 ? 3% perf-profile.self.cycles-pp.get_partial_node
0.00 +0.1 0.05 perf-profile.self.cycles-pp.exc_page_fault
0.00 +0.1 0.05 perf-profile.self.cycles-pp.refill_stock
0.00 +0.1 0.05 perf-profile.self.cycles-pp.__calc_delta
0.01 ?223% +0.1 0.06 perf-profile.self.cycles-pp.kmalloc_reserve
0.10 ? 4% +0.1 0.15 perf-profile.self.cycles-pp.perf_prepare_sample
0.00 +0.1 0.06 ? 9% perf-profile.self.cycles-pp.update_min_vruntime
0.00 +0.1 0.06 ? 9% perf-profile.self.cycles-pp.copy_fpregs_to_fpstate
0.01 ?223% +0.1 0.06 ? 7% perf-profile.self.cycles-pp.perf_tp_event_match
0.11 ? 6% +0.1 0.17 perf-profile.self.cycles-pp.refill_obj_stock
0.06 ? 9% +0.1 0.12 perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.1 0.06 perf-profile.self.cycles-pp.schedule_timeout
0.00 +0.1 0.06 perf-profile.self.cycles-pp.kernelmode_fixup_or_oops
0.00 +0.1 0.06 perf-profile.self.cycles-pp.security_socket_recvmsg
0.00 +0.1 0.06 perf-profile.self.cycles-pp.security_socket_getpeersec_dgram
0.00 +0.1 0.06 perf-profile.self.cycles-pp.get_callchain_entry
0.00 +0.1 0.06 ? 16% perf-profile.self.cycles-pp.do_wait
0.00 +0.1 0.06 ? 16% perf-profile.self.cycles-pp.try_charge
0.08 ? 6% +0.1 0.14 ? 3% perf-profile.self.cycles-pp.native_sched_clock
0.08 ? 6% +0.1 0.14 perf-profile.self.cycles-pp.__update_load_avg_se
0.17 +0.1 0.24 ? 2% perf-profile.self.cycles-pp.__ksize
0.12 ? 4% +0.1 0.18 ? 13% perf-profile.self.cycles-pp.obj_cgroup_charge
0.07 ? 8% +0.1 0.14 ? 3% perf-profile.self.cycles-pp.get_perf_callchain
0.00 +0.1 0.06 ? 7% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.00 +0.1 0.06 ? 7% perf-profile.self.cycles-pp.check_preempt_wakeup
0.05 ? 8% +0.1 0.12 perf-profile.self.cycles-pp.__kernel_text_address
0.03 ? 70% +0.1 0.10 perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.06 ? 8% +0.1 0.12 ? 4% perf-profile.self.cycles-pp.prepare_to_wait
0.05 ? 7% +0.1 0.12 perf-profile.self.cycles-pp.bsearch
0.08 ? 7% +0.1 0.15 perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.00 +0.1 0.07 perf-profile.self.cycles-pp.select_idle_sibling
0.00 +0.1 0.07 perf-profile.self.cycles-pp.pick_next_task_fair
0.00 +0.1 0.07 perf-profile.self.cycles-pp.wait_for_unix_gc
0.00 +0.1 0.07 perf-profile.self.cycles-pp.unfreeze_partials
0.09 ? 5% +0.1 0.16 ? 3% perf-profile.self.cycles-pp.__switch_to_asm
0.07 ? 7% +0.1 0.14 perf-profile.self.cycles-pp.__list_del_entry_valid
0.05 +0.1 0.12 ? 4% perf-profile.self.cycles-pp.cpuacct_charge
0.00 +0.1 0.08 ? 6% perf-profile.self.cycles-pp.__wake_up_common
0.00 +0.1 0.08 ? 6% perf-profile.self.cycles-pp.kvm_is_in_guest
0.22 ? 3% +0.1 0.30 ? 3% perf-profile.self.cycles-pp.__entry_text_start
0.04 ? 71% +0.1 0.12 ? 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.00 +0.1 0.08 perf-profile.self.cycles-pp.bpf_ksym_find
0.06 ? 7% +0.1 0.14 ? 3% perf-profile.self.cycles-pp.perf_output_begin_forward
0.06 ? 9% +0.1 0.14 ? 3% perf-profile.self.cycles-pp.__switch_to
0.20 ? 5% +0.1 0.28 perf-profile.self.cycles-pp.vfs_read
0.25 ? 3% +0.1 0.33 ? 9% perf-profile.self.cycles-pp.__virt_addr_valid
0.09 ? 4% +0.1 0.18 ? 2% perf-profile.self.cycles-pp.perf_tp_event
0.09 ? 4% +0.1 0.18 ? 2% perf-profile.self.cycles-pp.load_new_mm_cr3
0.17 ? 4% +0.1 0.25 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.15 ? 3% +0.1 0.24 ? 2% perf-profile.self.cycles-pp.new_sync_read
0.06 +0.1 0.14 ? 3% perf-profile.self.cycles-pp.try_to_wake_up
0.20 ? 2% +0.1 0.29 ? 3% perf-profile.self.cycles-pp.__might_sleep
0.00 +0.1 0.09 perf-profile.self.cycles-pp.select_task_rq_fair
0.02 ?141% +0.1 0.11 perf-profile.self.cycles-pp.update_rq_clock
0.08 ? 11% +0.1 0.18 ? 5% perf-profile.self.cycles-pp.perf_callchain_user
0.00 +0.1 0.10 ? 5% perf-profile.self.cycles-pp.asm_exc_page_fault
0.20 ? 3% +0.1 0.30 perf-profile.self.cycles-pp.sock_read_iter
0.06 ? 7% +0.1 0.16 perf-profile.self.cycles-pp.unwind_get_return_address
0.24 ? 3% +0.1 0.34 perf-profile.self.cycles-pp.vfs_write
0.00 +0.1 0.10 ? 10% perf-profile.self.cycles-pp.page_counter_uncharge
0.23 ? 2% +0.1 0.33 perf-profile.self.cycles-pp.aa_sk_perm
0.15 ? 2% +0.1 0.26 perf-profile.self.cycles-pp.new_sync_write
0.09 ? 4% +0.1 0.20 perf-profile.self.cycles-pp.cmp_ex_search
0.21 ? 4% +0.1 0.32 ? 3% perf-profile.self.cycles-pp.sock_write_iter
0.00 +0.1 0.12 ? 4% perf-profile.self.cycles-pp.release_pages
0.40 ? 3% +0.1 0.52 perf-profile.self.cycles-pp.__fget_light
0.51 ? 3% +0.1 0.62 ? 2% perf-profile.self.cycles-pp.__check_object_size
0.16 ? 5% +0.1 0.28 perf-profile.self.cycles-pp.switch_fpu_return
0.28 ? 3% +0.1 0.40 perf-profile.self.cycles-pp.common_file_perm
0.16 ? 2% +0.1 0.28 perf-profile.self.cycles-pp.___slab_alloc
0.29 ? 2% +0.1 0.42 perf-profile.self.cycles-pp.mutex_lock
0.27 ? 2% +0.1 0.40 perf-profile.self.cycles-pp.__check_heap_object
0.32 ? 2% +0.1 0.46 ? 2% perf-profile.self.cycles-pp.__alloc_skb
0.08 ? 5% +0.1 0.22 perf-profile.self.cycles-pp.kernel_text_address
0.06 ? 7% +0.1 0.20 ? 4% perf-profile.self.cycles-pp.update_cfs_group
0.07 ? 7% +0.1 0.20 ? 2% perf-profile.self.cycles-pp.update_curr
0.00 +0.1 0.14 ? 28% perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.46 ? 4% +0.1 0.60 perf-profile.self.cycles-pp.__build_skb_around
0.09 ? 5% +0.1 0.24 perf-profile.self.cycles-pp.__unwind_start
0.28 ? 3% +0.2 0.43 perf-profile.self.cycles-pp.fsnotify
0.18 ? 4% +0.2 0.35 ? 2% perf-profile.self.cycles-pp.unix_write_space
0.33 ? 4% +0.2 0.51 perf-profile.self.cycles-pp.___might_sleep
0.47 ? 3% +0.2 0.65 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.24 ? 6% +0.2 0.43 ? 2% perf-profile.self.cycles-pp.__schedule
0.06 +0.2 0.26 ? 7% perf-profile.self.cycles-pp.update_load_avg
0.00 +0.2 0.20 ? 2% perf-profile.self.cycles-pp.page_remove_rmap
0.00 +0.2 0.23 ? 4% perf-profile.self.cycles-pp.zap_pte_range
0.20 ? 3% +0.2 0.44 ? 2% perf-profile.self.cycles-pp.consume_skb
0.32 +0.2 0.57 ? 3% perf-profile.self.cycles-pp.unix_stream_sendmsg
0.18 ? 6% +0.3 0.44 perf-profile.self.cycles-pp.stack_access_ok
0.00 +0.3 0.28 ? 3% perf-profile.self.cycles-pp.wait_consider_task
0.27 ? 4% +0.3 0.56 perf-profile.self.cycles-pp.native_irq_return_iret
0.16 ? 7% +0.3 0.45 ? 6% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.82 ? 3% +0.3 2.12 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.24 ? 5% +0.3 0.55 perf-profile.self.cycles-pp.perf_callchain_kernel
0.08 ? 19% +0.4 0.46 ? 9% perf-profile.self.cycles-pp.obj_cgroup_charge_pages
0.08 ? 19% +0.4 0.50 ? 13% perf-profile.self.cycles-pp.obj_cgroup_uncharge_pages
0.31 ? 5% +0.4 0.74 ? 4% perf-profile.self.cycles-pp.mutex_unlock
0.31 ? 3% +0.4 0.76 ? 3% perf-profile.self.cycles-pp.orc_find
0.59 ? 2% +0.5 1.09 ? 3% perf-profile.self.cycles-pp.skb_set_owner_w
1.58 ? 2% +0.6 2.18 perf-profile.self.cycles-pp.sock_wfree
0.18 ? 23% +0.6 0.79 ? 10% perf-profile.self.cycles-pp.__mod_memcg_state
0.61 ? 3% +0.8 1.41 perf-profile.self.cycles-pp.__orc_find
0.96 ? 2% +0.8 1.77 ? 2% perf-profile.self.cycles-pp.unix_stream_read_generic
0.53 ? 5% +0.9 1.44 perf-profile.self.cycles-pp.sock_def_readable
0.80 ? 4% +1.0 1.76 perf-profile.self.cycles-pp.__get_user_nocheck_8
0.00 +1.2 1.17 ? 28% perf-profile.self.cycles-pp.intel_idle
0.79 ? 2% +1.3 2.13 perf-profile.self.cycles-pp.__slab_free
0.95 ? 4% +1.4 2.31 perf-profile.self.cycles-pp.unwind_next_frame
0.84 ? 4% +1.7 2.54 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
2.53 ? 2% +2.1 4.63 ? 2% perf-profile.self.cycles-pp._raw_spin_lock
0.08 ? 20% +2.8 2.89 ? 16% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.52 ? 7% +5.2 6.76 ? 14% perf-profile.self.cycles-pp.propagate_protected_usage
2.74 ? 9% +10.1 12.82 ? 2% perf-profile.self.cycles-pp.page_counter_cancel
3.39 ? 8% +10.5 13.86 ? 4% perf-profile.self.cycles-pp.page_counter_try_charge
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang