Greeting,
FYI, we noticed a 31.4% improvement of tbench.throughput-MB/sec due to commit:
commit: c82a69629c53eda5233f13fc11c3c01585ef48a2 ("sched/fair: fix case with reduced capacity CPU")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: tbench
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz (Cascade Lake) with 512G memory
with following parameters:
nr_threads: 100%
cluster: cs-localhost
cpufreq_governor: performance
Details are as below:
=========================================================================================
cluster/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/testcase:
cs-localhost/gcc-11/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/lkp-csl-2sp7/tbench
commit:
c02d5546ea ("sched/core: Use try_cmpxchg in set_nr_{and_not,if}_polling")
c82a69629c ("sched/fair: fix case with reduced capacity CPU")
c02d5546ea34d589 c82a69629c53eda5233f13fc11c
---------------- ---------------------------
%stddev %change %stddev
\ | \
11.49 ? 10% +93.6% 22.24 ? 7% tbench.max_latency
10095 +31.4% 13264 ? 10% tbench.throughput-MB/sec
32056675 ? 23% +2751.4% 9.141e+08 ? 34% tbench.time.involuntary_context_switches
2521 +42.4% 3592 ? 11% tbench.time.percent_of_cpu_this_job_got
13943 +41.9% 19781 ? 10% tbench.time.system_time
4224 ? 3% +44.3% 6096 ? 20% tbench.time.user_time
1.318e+09 -31.5% 9.033e+08 ? 18% tbench.time.voluntary_context_switches
17117 -28.3% 12273 ? 16% uptime.idle
1.364e+10 -36.4% 8.685e+09 ? 22% cpuidle..time
1.731e+09 -31.8% 1.181e+09 ? 18% cpuidle..usage
29.32 +12.2% 32.90 boot-time.boot
14.26 +24.6% 17.77 boot-time.dhcp
2339 +13.7% 2660 boot-time.idle
22.67 -8.4 14.25 ? 23% mpstat.cpu.all.idle%
3.11 ? 3% -1.1 2.06 ? 19% mpstat.cpu.all.irq%
43.53 +8.1 51.66 ? 5% mpstat.cpu.all.sys%
8.80 ? 2% +2.4 11.22 ? 23% mpstat.cpu.all.usr%
1910029 ? 2% -41.7% 1114357 ? 32% numa-vmstat.node1.nr_inactive_anon
952400 ? 2% -45.2% 521490 ? 39% numa-vmstat.node1.nr_mapped
1908572 ? 2% -41.5% 1116461 ? 32% numa-vmstat.node1.nr_shmem
1910028 ? 2% -41.7% 1114358 ? 32% numa-vmstat.node1.nr_zone_inactive_anon
7641189 ? 2% -41.6% 4459143 ? 32% numa-meminfo.node1.Inactive
7640015 ? 2% -41.6% 4458204 ? 32% numa-meminfo.node1.Inactive(anon)
3809092 ? 2% -45.2% 2086458 ? 39% numa-meminfo.node1.Mapped
10329178 ? 12% -24.6% 7787191 ? 27% numa-meminfo.node1.MemUsed
7635879 ? 2% -41.5% 4466108 ? 32% numa-meminfo.node1.Shmem
22.17 ? 3% -36.1% 14.17 ? 23% vmstat.cpu.id
10481018 ? 2% -18.3% 8564745 ? 14% vmstat.memory.cache
97.00 +14.4% 111.00 ? 5% vmstat.procs.r
6128694 +9.4% 6706040 ? 3% vmstat.system.cs
810000 -31.2% 557330 ? 17% vmstat.system.in
2718 -2.6% 2648 turbostat.Bzy_MHz
1.681e+09 -33.0% 1.126e+09 ? 19% turbostat.C1
18.69 -7.1 11.58 ? 23% turbostat.C1%
16.59 -36.3% 10.56 ? 22% turbostat.CPU%c1
0.12 ? 3% +16.2% 0.14 ? 3% turbostat.IPC
5.867e+08 -31.2% 4.036e+08 ? 17% turbostat.IRQ
10364034 ? 2% -18.5% 8447939 ? 14% meminfo.Cached
9993526 ? 2% -29.3% 7066169 ? 18% meminfo.Committed_AS
551364 ? 3% +73.8% 958233 ? 6% meminfo.DirectMap4k
7806682 ? 2% -37.4% 4885485 ? 25% meminfo.Inactive
7805273 ? 2% -37.4% 4884076 ? 25% meminfo.Inactive(anon)
3900533 ? 2% -39.1% 2377370 ? 27% meminfo.Mapped
12167854 -16.2% 10198035 ? 12% meminfo.Memused
25096 ? 2% -11.5% 22220 ? 5% meminfo.PageTables
7642756 ? 2% -38.1% 4733449 ? 26% meminfo.Shmem
2719315 +36.5% 3712526 meminfo.Unevictable
90730 -5.4% 85831 proc-vmstat.nr_anon_pages
2590987 ? 2% -18.5% 2111929 ? 14% proc-vmstat.nr_file_pages
1951253 ? 2% -37.4% 1221001 ? 25% proc-vmstat.nr_inactive_anon
975082 ? 2% -39.0% 594342 ? 27% proc-vmstat.nr_mapped
6273 ? 2% -11.5% 5554 ? 5% proc-vmstat.nr_page_table_pages
1910667 ? 2% -38.1% 1183305 ? 26% proc-vmstat.nr_shmem
31783 -3.3% 30743 ? 2% proc-vmstat.nr_slab_reclaimable
679828 +36.5% 928131 proc-vmstat.nr_unevictable
1951253 ? 2% -37.4% 1221001 ? 25% proc-vmstat.nr_zone_inactive_anon
679828 +36.5% 928131 proc-vmstat.nr_zone_unevictable
627539 ? 10% +54.4% 968981 ? 39% proc-vmstat.numa_hint_faults
449238 ? 12% +74.9% 785532 ? 49% proc-vmstat.numa_hint_faults_local
1.505e+08 +23.8% 1.863e+08 ? 6% proc-vmstat.numa_hit
4282 ? 3% -53.2% 2003 ? 52% proc-vmstat.numa_huge_pte_updates
1.505e+08 +23.7% 1.862e+08 ? 6% proc-vmstat.numa_local
2952837 ? 4% -28.1% 2122117 ? 13% proc-vmstat.numa_pte_updates
1.505e+08 +23.7% 1.862e+08 ? 6% proc-vmstat.pgalloc_normal
2727576 ? 2% +11.5% 3040426 ? 12% proc-vmstat.pgfault
1.505e+08 +23.7% 1.862e+08 ? 6% proc-vmstat.pgfree
10982195 +41.3% 15515107 ? 7% sched_debug.cfs_rq:/.MIN_vruntime.max
1.00 ? 2% +11.8% 1.12 ? 5% sched_debug.cfs_rq:/.h_nr_running.avg
0.88 ? 3% -14.7% 0.75 ? 6% sched_debug.cfs_rq:/.h_nr_running.stddev
56664 ? 6% -23.8% 43171 ? 10% sched_debug.cfs_rq:/.load.max
12752 ? 3% -29.3% 9016 ? 16% sched_debug.cfs_rq:/.load.stddev
5.00 ? 3% -25.1% 3.74 ? 14% sched_debug.cfs_rq:/.load_avg.min
10982195 +41.3% 15515107 ? 7% sched_debug.cfs_rq:/.max_vruntime.max
10265230 +62.3% 16659900 ? 8% sched_debug.cfs_rq:/.min_vruntime.avg
12960623 ? 2% +49.2% 19341384 ? 9% sched_debug.cfs_rq:/.min_vruntime.max
9962156 +47.3% 14674448 ? 7% sched_debug.cfs_rq:/.min_vruntime.min
396620 ? 6% +384.1% 1919935 ? 15% sched_debug.cfs_rq:/.min_vruntime.stddev
0.67 ? 2% +15.5% 0.77 ? 5% sched_debug.cfs_rq:/.nr_running.avg
0.44 -18.3% 0.36 ? 10% sched_debug.cfs_rq:/.nr_running.stddev
753.58 +23.9% 933.72 ? 8% sched_debug.cfs_rq:/.runnable_avg.avg
1237 ? 2% +32.0% 1633 ? 6% sched_debug.cfs_rq:/.runnable_avg.max
499.14 ? 2% -13.9% 429.87 ? 5% sched_debug.cfs_rq:/.runnable_avg.min
154.08 +67.7% 258.43 ? 15% sched_debug.cfs_rq:/.runnable_avg.stddev
396613 ? 6% +384.1% 1919923 ? 15% sched_debug.cfs_rq:/.spread0.stddev
467.16 +38.5% 647.14 ? 12% sched_debug.cfs_rq:/.util_avg.avg
912.23 +16.9% 1066 ? 7% sched_debug.cfs_rq:/.util_avg.max
69.44 +179.8% 194.29 ? 28% sched_debug.cfs_rq:/.util_avg.stddev
14.94 ? 2% +1631.7% 258.71 ? 40% sched_debug.cfs_rq:/.util_est_enqueued.avg
55.72 ? 3% +364.2% 258.67 ? 40% sched_debug.cfs_rq:/.util_est_enqueued.stddev
70960 +46.2% 103773 ? 5% sched_debug.cpu.avg_idle.avg
175370 ? 25% +162.2% 459798 ? 14% sched_debug.cpu.avg_idle.max
26563 ? 11% +233.4% 88569 ? 14% sched_debug.cpu.avg_idle.stddev
3415 +18.2% 4038 ? 6% sched_debug.cpu.curr->pid.avg
2608 -15.4% 2206 ? 7% sched_debug.cpu.curr->pid.stddev
0.00 ? 5% +61.3% 0.00 ? 20% sched_debug.cpu.next_balance.stddev
0.95 ? 2% +16.1% 1.10 ? 5% sched_debug.cpu.nr_running.avg
0.84 -11.9% 0.74 ? 5% sched_debug.cpu.nr_running.stddev
22978694 +11.4% 25589455 ? 3% sched_debug.cpu.nr_switches.avg
24010750 ? 2% +16.0% 27851318 ? 5% sched_debug.cpu.nr_switches.max
19490849 ? 5% +19.8% 23353131 sched_debug.cpu.nr_switches.min
20.18 -30.9% 13.94 ? 16% perf-stat.i.MPKI
1.872e+10 +18.2% 2.213e+10 ? 5% perf-stat.i.branch-instructions
1.75 -0.1 1.69 ? 2% perf-stat.i.branch-miss-rate%
3.247e+08 +13.4% 3.683e+08 ? 4% perf-stat.i.branch-misses
3.10 ? 11% +2.0 5.09 ? 14% perf-stat.i.cache-miss-rate%
57504764 ? 10% +23.7% 71148870 ? 11% perf-stat.i.cache-misses
1.89e+09 ? 2% -20.9% 1.494e+09 ? 10% perf-stat.i.cache-references
6156451 +9.4% 6734364 ? 3% perf-stat.i.context-switches
2.35 -11.7% 2.08 ? 4% perf-stat.i.cpi
2.21e+11 +3.6% 2.289e+11 perf-stat.i.cpu-cycles
2487332 ? 3% -28.2% 1786056 ? 15% perf-stat.i.cpu-migrations
0.13 ? 3% -0.0 0.10 ? 17% perf-stat.i.dTLB-load-miss-rate%
2.827e+10 +19.6% 3.382e+10 ? 5% perf-stat.i.dTLB-loads
0.06 -0.0 0.04 ? 20% perf-stat.i.dTLB-store-miss-rate%
8801520 ? 3% -23.1% 6766285 ? 13% perf-stat.i.dTLB-store-misses
1.521e+10 +18.6% 1.803e+10 ? 5% perf-stat.i.dTLB-stores
55.17 +3.5 58.69 ? 2% perf-stat.i.iTLB-load-miss-rate%
1.201e+08 ? 3% +12.7% 1.354e+08 ? 4% perf-stat.i.iTLB-load-misses
9.46e+10 +18.4% 1.12e+11 ? 5% perf-stat.i.instructions
792.14 +4.3% 826.31 ? 2% perf-stat.i.instructions-per-iTLB-miss
0.43 +14.1% 0.49 ? 4% perf-stat.i.ipc
0.86 ? 4% +47.8% 1.27 ? 20% perf-stat.i.major-faults
2.30 +3.6% 2.38 perf-stat.i.metric.GHz
668.48 +17.7% 786.62 ? 5% perf-stat.i.metric.M/sec
3502 ? 2% +12.4% 3935 ? 13% perf-stat.i.minor-faults
78.57 +8.8 87.35 ? 2% perf-stat.i.node-load-miss-rate%
16081011 ? 12% +30.0% 20898766 ? 11% perf-stat.i.node-load-misses
5543925 ? 2% -40.0% 3326330 ? 10% perf-stat.i.node-loads
83.01 ? 3% +8.8 91.80 ? 2% perf-stat.i.node-store-miss-rate%
5420293 ? 8% +25.2% 6788239 ? 10% perf-stat.i.node-store-misses
1212242 ? 11% -48.9% 619862 ? 27% perf-stat.i.node-stores
3503 ? 2% +12.4% 3937 ? 13% perf-stat.i.page-faults
19.97 -32.8% 13.43 ? 14% perf-stat.overall.MPKI
1.73 -0.1 1.67 ? 2% perf-stat.overall.branch-miss-rate%
3.05 ? 11% +1.8 4.84 ? 18% perf-stat.overall.cache-miss-rate%
2.34 -12.3% 2.05 ? 4% perf-stat.overall.cpi
0.13 ? 3% -0.0 0.10 ? 16% perf-stat.overall.dTLB-load-miss-rate%
0.06 -0.0 0.04 ? 17% perf-stat.overall.dTLB-store-miss-rate%
55.17 +3.6 58.72 ? 2% perf-stat.overall.iTLB-load-miss-rate%
787.99 +4.9% 826.86 perf-stat.overall.instructions-per-iTLB-miss
0.43 +14.3% 0.49 ? 4% perf-stat.overall.ipc
74.13 ? 3% +12.0 86.08 ? 2% perf-stat.overall.node-load-miss-rate%
81.61 ? 3% +9.9 91.52 ? 2% perf-stat.overall.node-store-miss-rate%
1.869e+10 +18.2% 2.21e+10 ? 5% perf-stat.ps.branch-instructions
3.243e+08 +13.4% 3.678e+08 ? 4% perf-stat.ps.branch-misses
57432021 ? 10% +23.8% 71077827 ? 11% perf-stat.ps.cache-misses
1.887e+09 ? 2% -20.9% 1.492e+09 ? 10% perf-stat.ps.cache-references
6147733 +9.4% 6725226 ? 3% perf-stat.ps.context-switches
2.207e+11 +3.6% 2.286e+11 perf-stat.ps.cpu-cycles
2483875 ? 2% -28.2% 1783069 ? 15% perf-stat.ps.cpu-migrations
2.824e+10 +19.6% 3.377e+10 ? 5% perf-stat.ps.dTLB-loads
8789276 ? 3% -23.1% 6755194 ? 13% perf-stat.ps.dTLB-store-misses
1.519e+10 +18.6% 1.801e+10 ? 5% perf-stat.ps.dTLB-stores
1.2e+08 ? 3% +12.7% 1.352e+08 ? 4% perf-stat.ps.iTLB-load-misses
9.447e+10 +18.4% 1.119e+11 ? 5% perf-stat.ps.instructions
0.86 ? 4% +47.9% 1.27 ? 20% perf-stat.ps.major-faults
3503 ? 2% +12.4% 3937 ? 13% perf-stat.ps.minor-faults
16057997 ? 12% +30.0% 20872834 ? 11% perf-stat.ps.node-load-misses
5542818 ? 2% -39.8% 3334248 ? 10% perf-stat.ps.node-loads
5412847 ? 8% +25.3% 6779940 ? 10% perf-stat.ps.node-store-misses
1210963 ? 11% -48.9% 618995 ? 27% perf-stat.ps.node-stores
3503 ? 2% +12.4% 3938 ? 13% perf-stat.ps.page-faults
6.822e+13 +18.4% 8.077e+13 ? 5% perf-stat.total.instructions
24.89 -13.1 11.78 ? 4% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
24.65 -13.1 11.58 ? 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
24.64 -13.1 11.57 ? 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
24.62 -13.1 11.56 ? 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
16.56 -8.5 8.06 ? 4% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
15.49 -8.0 7.53 ? 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
15.47 -8.0 7.52 ? 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
15.06 -7.8 7.29 ? 4% perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
14.73 -7.6 7.14 ? 4% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
9.84 -4.3 5.52 ? 4% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets
5.15 -3.6 1.56 ? 17% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
19.38 -3.3 16.08 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.server
19.45 -3.1 16.32 ? 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.server.listener
19.50 -3.1 16.38 ? 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.server.listener.main
19.62 -3.1 16.57 ? 2% perf-profile.calltrace.cycles-pp.__send.server.listener.main.__libc_start_main
3.59 ? 5% -2.4 1.19 ? 5% perf-profile.calltrace.cycles-pp.perf_trace_sched_migrate_task.set_task_cpu.try_to_wake_up.__wake_up_common.__wake_up_common_lock
3.69 ? 2% -2.3 1.34 ? 4% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
3.31 ? 7% -2.1 1.18 ? 5% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_migrate_task.set_task_cpu.try_to_wake_up.__wake_up_common
3.26 ? 7% -2.1 1.16 ? 5% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_migrate_task.set_task_cpu.try_to_wake_up
3.16 ? 6% -2.0 1.15 ? 5% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_migrate_task.set_task_cpu
3.86 -2.0 1.89 ? 3% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
3.80 -1.9 1.86 ? 3% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
18.24 -1.9 16.33 perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
2.84 -1.8 1.07 ? 5% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_migrate_task
17.28 -1.7 15.57 perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
2.65 ? 3% -1.7 0.95 ? 4% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
16.94 -1.6 15.32 perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg
16.80 -1.6 15.23 perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked
16.67 -1.5 15.14 perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
21.29 -1.5 19.76 perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
7.69 -1.5 6.16 ? 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
8.46 -1.5 6.96 ? 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.wait_woken
2.24 ? 4% -1.5 0.76 ? 4% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry
2.20 ? 4% -1.5 0.74 ? 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle
21.74 -1.3 20.43 perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
21.80 -1.3 20.50 perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.93 ? 4% -1.3 0.64 ? 5% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue
3.31 -1.3 2.04 ? 3% perf-profile.calltrace.cycles-pp.recv.do_packets.sio_readx.child_op.child_run
3.21 -1.3 1.96 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_readx.child_op
3.19 -1.3 1.94 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_readx
2.26 -1.2 1.12 ? 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
2.15 -1.1 1.06 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
2.08 -1.1 1.02 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
2.10 -0.8 1.30 ? 4% perf-profile.calltrace.cycles-pp.recv.do_packets.sio_createx.child_op.child_run
2.03 -0.8 1.24 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_createx.child_op
2.02 -0.8 1.24 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_createx
1.91 -0.7 1.18 ? 4% perf-profile.calltrace.cycles-pp.recv.do_packets.sio_qpathinfo.child_op.child_run
1.85 -0.7 1.13 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_qpathinfo.child_op
1.84 -0.7 1.12 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_qpathinfo
2.89 -0.7 2.19 ? 2% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
0.98 ? 2% -0.6 0.35 ? 70% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.select_task_rq.try_to_wake_up.__wake_up_common
1.54 ? 2% -0.6 0.96 ? 4% perf-profile.calltrace.cycles-pp.recv.do_packets.sio_close.child_op.child_run
1.24 -0.6 0.66 ? 3% perf-profile.calltrace.cycles-pp.select_task_rq.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
1.49 ? 2% -0.6 0.92 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_close.child_op
2.47 -0.6 1.90 ? 2% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
1.48 ? 2% -0.6 0.91 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_close
1.19 -0.5 0.64 ? 4% perf-profile.calltrace.cycles-pp.select_task_rq_fair.select_task_rq.try_to_wake_up.__wake_up_common.__wake_up_common_lock
1.06 -0.4 0.66 ? 4% perf-profile.calltrace.cycles-pp.recv.do_packets.sio_writex.child_op.child_run
1.02 -0.4 0.63 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_writex
1.02 -0.4 0.64 ? 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.do_packets.sio_writex.child_op
0.70 ? 2% -0.1 0.65 ? 2% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
0.58 -0.0 0.54 ? 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter.skb_do_copy_data_nocache.tcp_sendmsg_locked
0.75 +0.1 0.81 ? 2% perf-profile.calltrace.cycles-pp.skb_do_copy_data_nocache.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
1.23 +0.1 1.35 ? 3% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
1.24 +0.1 1.38 ? 3% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
0.52 +0.2 0.71 ? 5% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
0.54 +0.2 0.74 ? 4% perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
0.57 +0.2 0.78 ? 3% perf-profile.calltrace.cycles-pp.recv
0.51 +0.2 0.73 ? 2% perf-profile.calltrace.cycles-pp.vsnprintf.snprintf.child_run.create_procs.main
0.52 +0.2 0.74 ? 2% perf-profile.calltrace.cycles-pp.snprintf.child_run.create_procs.main.__libc_start_main
0.91 +0.2 1.14 ? 3% perf-profile.calltrace.cycles-pp.do_packets.sio_qfileinfo.child_op.child_run.create_procs
0.94 +0.2 1.18 ? 3% perf-profile.calltrace.cycles-pp.sio_qfileinfo.child_op.child_run.create_procs.main
0.95 +0.2 1.20 ? 3% perf-profile.calltrace.cycles-pp.do_packets.sio_qfsinfo.child_op.child_run.create_procs
0.67 +0.3 0.92 ? 3% perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
0.98 +0.3 1.24 ? 2% perf-profile.calltrace.cycles-pp.sio_qfsinfo.child_op.child_run.create_procs.main
0.58 +0.3 0.86 ? 3% perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
1.16 +0.3 1.46 ? 3% perf-profile.calltrace.cycles-pp.do_packets.sio_unlink.child_op.child_run.create_procs
1.20 +0.3 1.51 ? 2% perf-profile.calltrace.cycles-pp.sio_unlink.child_op.child_run.create_procs.main
0.34 ? 70% +0.3 0.69 ? 5% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg
0.56 +0.4 0.91 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qfileinfo.child_op
0.55 +0.4 0.91 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qfileinfo
0.56 +0.4 0.92 ? 4% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_qfileinfo.child_op.child_run
0.58 +0.4 0.95 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qfsinfo
0.59 +0.4 0.96 ? 4% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_qfsinfo.child_op.child_run
0.58 +0.4 0.96 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qfsinfo.child_op
0.97 +0.4 1.36 ? 3% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
4.25 +0.4 4.68 ? 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
5.11 +0.4 5.54 ? 4% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
0.71 +0.4 1.16 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_unlink
0.72 +0.4 1.17 ? 4% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_unlink.child_op.child_run
0.71 +0.5 1.16 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_unlink.child_op
4.47 +0.5 4.93 ? 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
4.07 ? 5% +0.5 4.54 ? 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
2.04 +0.5 2.52 ? 2% perf-profile.calltrace.cycles-pp.do_packets.sio_findfirst.child_op.child_run.create_procs
0.17 ?141% +0.5 0.66 ? 2% perf-profile.calltrace.cycles-pp.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
2.11 +0.5 2.61 ? 2% perf-profile.calltrace.cycles-pp.sio_findfirst.child_op.child_run.create_procs.main
0.17 ?141% +0.5 0.67 ? 4% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked
0.08 ?223% +0.5 0.61 ? 2% perf-profile.calltrace.cycles-pp.sio_sfileinfo.child_op.child_run.create_procs.main
0.00 +0.6 0.58 ? 2% perf-profile.calltrace.cycles-pp.do_packets.sio_sfileinfo.child_op.child_run.create_procs
0.08 ?223% +0.6 0.70 perf-profile.calltrace.cycles-pp.dopr.vsnprintf.snprintf.child_run.create_procs
0.00 +0.7 0.65 ? 2% perf-profile.calltrace.cycles-pp.next_token.child_run.create_procs.main.__libc_start_main
3.27 +0.7 3.98 ? 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.wait_woken
3.15 +0.7 3.87 ? 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout
1.25 +0.8 2.00 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_findfirst
0.00 +0.8 0.76 ? 8% perf-profile.calltrace.cycles-pp.aa_get_task_label.apparmor_task_kill.security_task_kill.kill_pid_info.kill_something_info
1.25 +0.8 2.01 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_findfirst.child_op
0.00 +0.8 0.76 ? 14% perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
1.26 +0.8 2.02 ? 3% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_findfirst.child_op.child_run
4.22 +0.8 5.04 ? 4% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
0.00 +0.8 0.82 ? 13% perf-profile.calltrace.cycles-pp.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
0.00 +0.8 0.82 ? 13% perf-profile.calltrace.cycles-pp.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.91 ? 34% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_loop
5.86 +1.0 6.85 ? 9% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
0.00 +1.0 1.01 ? 45% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare
4.20 +1.0 5.24 ? 3% perf-profile.calltrace.cycles-pp.do_packets.sio_close.child_op.child_run.create_procs
4.34 +1.1 5.42 ? 2% perf-profile.calltrace.cycles-pp.sio_close.child_op.child_run.create_procs.main
33.90 +1.2 35.11 ? 2% perf-profile.calltrace.cycles-pp.server.listener.main.__libc_start_main
33.95 +1.2 35.18 ? 2% perf-profile.calltrace.cycles-pp.listener.main.__libc_start_main
3.50 +1.3 4.77 ? 3% perf-profile.calltrace.cycles-pp.do_packets.sio_writex.child_op.child_run.create_procs
5.26 +1.3 6.56 ? 2% perf-profile.calltrace.cycles-pp.do_packets.sio_qpathinfo.child_op.child_run.create_procs
3.62 +1.3 4.92 ? 3% perf-profile.calltrace.cycles-pp.sio_writex.child_op.child_run.create_procs.main
5.43 +1.4 6.79 ? 2% perf-profile.calltrace.cycles-pp.sio_qpathinfo.child_op.child_run.create_procs.main
5.80 +1.4 7.22 ? 2% perf-profile.calltrace.cycles-pp.do_packets.sio_createx.child_op.child_run.create_procs
3.07 +1.5 4.55 ? 6% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
5.99 +1.5 7.47 ? 2% perf-profile.calltrace.cycles-pp.sio_createx.child_op.child_run.create_procs.main
0.61 ? 2% +1.5 2.10 ? 6% perf-profile.calltrace.cycles-pp.kill_something_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
0.75 +1.5 2.28 ? 5% perf-profile.calltrace.cycles-pp.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill.create_procs
2.57 +1.6 4.17 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_close
2.57 +1.6 4.18 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_close.child_op
2.59 +1.6 4.21 ? 3% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_close.child_op.child_run
0.82 +1.6 2.46 ? 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill.create_procs.main
0.84 +1.7 2.50 ? 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.kill.create_procs.main.__libc_start_main
2.38 +1.7 4.03 ? 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_writex
2.38 +1.7 4.04 ? 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_writex.child_op
2.39 +1.7 4.06 ? 4% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_writex.child_op.child_run
0.26 ?100% +1.7 1.94 ? 6% perf-profile.calltrace.cycles-pp.kill_pid_info.kill_something_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.96 +1.7 2.68 ? 5% perf-profile.calltrace.cycles-pp.kill.create_procs.main.__libc_start_main
0.00 +1.8 1.80 ? 6% perf-profile.calltrace.cycles-pp.apparmor_task_kill.security_task_kill.kill_pid_info.kill_something_info.__x64_sys_kill
0.00 +1.8 1.82 ? 6% perf-profile.calltrace.cycles-pp.security_task_kill.kill_pid_info.kill_something_info.__x64_sys_kill.do_syscall_64
3.23 +2.0 5.25 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qpathinfo
3.24 +2.0 5.26 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_qpathinfo.child_op
3.26 +2.0 5.29 ? 3% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_qpathinfo.child_op.child_run
9.03 +2.2 11.21 ? 2% perf-profile.calltrace.cycles-pp.do_packets.sio_readx.child_op.child_run.create_procs
0.00 +2.2 2.19 ? 21% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode
3.57 +2.2 5.77 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_createx
3.58 +2.2 5.78 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_createx.child_op
3.60 +2.2 5.82 ? 3% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_createx.child_op.child_run
9.38 +2.3 11.64 ? 2% perf-profile.calltrace.cycles-pp.sio_readx.child_op.child_run.create_procs.main
0.00 +2.4 2.39 ? 16% perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.00 +2.4 2.42 ? 16% perf-profile.calltrace.cycles-pp.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.58 +2.5 15.12 ? 3% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
0.00 +2.6 2.55 ? 16% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
0.00 +2.9 2.89 ? 9% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets
26.14 +3.1 29.26 ? 2% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
0.52 ? 2% +3.2 3.69 ? 7% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity
26.00 +3.2 29.20 ? 2% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
0.53 ? 2% +3.2 3.76 ? 7% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair
28.21 +3.2 31.44 ? 2% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
0.54 +3.3 3.81 ? 7% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate
25.78 +3.3 29.06 ? 2% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
24.12 +3.3 27.43 ? 3% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
28.57 +3.4 31.94 ? 2% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
24.20 +3.4 27.58 ? 3% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
0.59 +3.4 4.00 ? 10% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
28.69 +3.4 32.10 ? 2% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb
5.50 +3.4 8.93 ? 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_readx
5.51 +3.4 8.95 ? 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.do_packets.sio_readx.child_op
28.74 +3.4 32.19 ? 2% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
5.54 +3.5 9.00 ? 3% perf-profile.calltrace.cycles-pp.__send.do_packets.sio_readx.child_op.child_run
30.27 +3.5 33.74 ? 2% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
0.72 +3.6 4.29 ? 10% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common
32.66 +3.6 36.23 ? 2% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
26.74 +3.6 30.32 ? 2% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
26.98 +3.6 30.62 ? 2% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
27.02 +3.6 30.67 ? 2% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
31.17 +3.7 34.88 ? 2% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
4.48 ? 6% +3.7 8.20 ? 4% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
15.58 +3.8 19.42 ? 5% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
15.64 +3.9 19.49 ? 5% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
33.80 +3.9 37.72 ? 2% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
33.84 +4.0 37.82 ? 2% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
12.68 +4.1 16.75 ? 2% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.server
13.21 +4.2 17.46 ? 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.server.listener
13.29 +4.3 17.57 ? 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.server.listener.main
13.75 +4.3 18.04 ? 2% perf-profile.calltrace.cycles-pp.recv.server.listener.main.__libc_start_main
3.25 +4.3 7.60 ? 4% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
0.87 +4.4 5.29 ? 7% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
0.87 +4.4 5.32 ? 7% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
38.72 +4.7 43.44 ? 2% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
39.07 +5.0 44.08 ? 2% perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
39.38 +5.6 44.97 ? 2% perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.57 +5.7 45.27 ? 2% perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
20.26 +9.0 29.28 ? 3% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.do_packets
35.82 +9.2 44.97 ? 2% perf-profile.calltrace.cycles-pp.child_op.child_run.create_procs.main.__libc_start_main
37.59 +9.8 47.36 ? 2% perf-profile.calltrace.cycles-pp.child_run.create_procs.main.__libc_start_main
38.72 +11.5 50.25 perf-profile.calltrace.cycles-pp.create_procs.main.__libc_start_main
72.68 +12.8 85.43 perf-profile.calltrace.cycles-pp.main.__libc_start_main
72.71 +12.8 85.47 perf-profile.calltrace.cycles-pp.__libc_start_main
24.89 -13.1 11.78 ? 4% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
24.89 -13.1 11.78 ? 4% perf-profile.children.cycles-pp.cpu_startup_entry
24.86 -13.1 11.77 ? 4% perf-profile.children.cycles-pp.do_idle
24.65 -13.1 11.58 ? 3% perf-profile.children.cycles-pp.start_secondary
16.73 -8.5 8.21 ? 4% perf-profile.children.cycles-pp.cpuidle_idle_call
15.64 -8.0 7.66 ? 4% perf-profile.children.cycles-pp.cpuidle_enter
15.62 -8.0 7.66 ? 4% perf-profile.children.cycles-pp.cpuidle_enter_state
15.20 -7.8 7.42 ? 4% perf-profile.children.cycles-pp.intel_idle_irq
15.15 -7.7 7.40 ? 4% perf-profile.children.cycles-pp.mwait_idle_with_hints
6.70 -3.5 3.24 ? 5% perf-profile.children.cycles-pp.set_task_cpu
5.37 -2.6 2.80 ? 5% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
3.79 ? 2% -2.4 1.39 ? 4% perf-profile.children.cycles-pp.flush_smp_call_function_queue
3.57 ? 3% -2.3 1.30 ? 3% perf-profile.children.cycles-pp.sched_ttwu_pending
3.90 -2.0 1.92 ? 4% perf-profile.children.cycles-pp.schedule_idle
19.65 -1.6 18.00 perf-profile.children.cycles-pp.sk_wait_data
23.24 -1.6 21.63 perf-profile.children.cycles-pp.tcp_recvmsg_locked
18.61 -1.5 17.16 perf-profile.children.cycles-pp.wait_woken
3.26 ? 3% -1.4 1.83 ? 3% perf-profile.children.cycles-pp.update_load_avg
23.79 -1.4 22.40 perf-profile.children.cycles-pp.tcp_recvmsg
2.50 ? 7% -1.4 1.11 ? 9% perf-profile.children.cycles-pp.update_cfs_group
23.86 -1.4 22.47 perf-profile.children.cycles-pp.inet_recvmsg
18.25 -1.4 16.89 perf-profile.children.cycles-pp.schedule_timeout
8.98 -1.1 7.85 perf-profile.children.cycles-pp.dequeue_entity
9.85 -1.1 8.75 perf-profile.children.cycles-pp.dequeue_task_fair
2.68 ? 2% -1.0 1.64 ? 3% perf-profile.children.cycles-pp.select_task_rq
2.58 ? 2% -1.0 1.55 ? 3% perf-profile.children.cycles-pp.select_task_rq_fair
2.15 ? 2% -0.9 1.22 ? 3% perf-profile.children.cycles-pp.select_idle_sibling
2.01 -0.9 1.15 ? 3% perf-profile.children.cycles-pp._raw_spin_lock
1.79 ? 2% -0.8 0.98 ? 4% perf-profile.children.cycles-pp.select_idle_cpu
1.52 ? 2% -0.7 0.79 ? 5% perf-profile.children.cycles-pp.select_idle_core
1.16 ? 3% -0.7 0.44 ? 3% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
1.05 ? 3% -0.7 0.39 ? 3% perf-profile.children.cycles-pp.sysvec_call_function_single
0.98 ? 3% -0.6 0.37 ? 3% perf-profile.children.cycles-pp.__sysvec_call_function_single
1.62 -0.6 1.03 ? 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.16 -0.6 0.60 ? 5% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
1.13 -0.5 0.60 ? 5% perf-profile.children.cycles-pp.available_idle_cpu
0.81 ? 2% -0.4 0.42 ? 5% perf-profile.children.cycles-pp.menu_select
0.93 ? 2% -0.3 0.61 perf-profile.children.cycles-pp.finish_task_switch
1.29 ? 2% -0.3 1.02 perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.52 -0.2 0.28 ? 5% perf-profile.children.cycles-pp.migrate_task_rq_fair
0.52 -0.2 0.28 ? 5% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.40 -0.2 0.20 ? 6% perf-profile.children.cycles-pp.napi_consume_skb
0.36 -0.2 0.18 ? 7% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.33 ? 2% -0.2 0.16 ? 5% perf-profile.children.cycles-pp.tcp_data_queue
1.61 -0.2 1.44 ? 6% perf-profile.children.cycles-pp.perf_output_sample
0.36 ? 3% -0.2 0.20 ? 8% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.54 ? 5% -0.2 0.38 ? 4% perf-profile.children.cycles-pp.read_sock
0.31 -0.2 0.16 ? 7% perf-profile.children.cycles-pp.__smp_call_single_queue
0.30 -0.2 0.15 ? 7% perf-profile.children.cycles-pp.llist_add_batch
0.22 ? 6% -0.1 0.08 ? 9% perf-profile.children.cycles-pp.bpf_ksym_find
0.34 ? 2% -0.1 0.20 ? 7% perf-profile.children.cycles-pp.perf_output_end
0.95 ? 2% -0.1 0.82 perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.30 -0.1 0.16 ? 4% perf-profile.children.cycles-pp.wake_affine
0.24 -0.1 0.12 ? 5% perf-profile.children.cycles-pp.tick_nohz_next_event
0.24 ? 5% -0.1 0.13 ? 3% perf-profile.children.cycles-pp._find_next_bit
1.01 -0.1 0.90 ? 5% perf-profile.children.cycles-pp.memcpy_erms
0.27 ? 6% -0.1 0.16 ? 3% perf-profile.children.cycles-pp.is_bpf_text_address
0.39 -0.1 0.29 ? 3% perf-profile.children.cycles-pp.get_stack_info
0.20 ? 5% -0.1 0.10 ? 4% perf-profile.children.cycles-pp.cpumask_next
0.28 ? 2% -0.1 0.19 ? 3% perf-profile.children.cycles-pp.get_stack_info_noinstr
0.19 ? 3% -0.1 0.10 ? 6% perf-profile.children.cycles-pp.remove_entity_load_avg
0.20 ? 3% -0.1 0.11 ? 3% perf-profile.children.cycles-pp.__irq_exit_rcu
0.88 -0.1 0.80 ? 3% perf-profile.children.cycles-pp.copyout
0.17 ? 2% -0.1 0.09 ? 12% perf-profile.children.cycles-pp.llist_reverse_order
0.84 -0.1 0.76 ? 2% perf-profile.children.cycles-pp.switch_fpu_return
0.28 ? 2% -0.1 0.20 ? 5% perf-profile.children.cycles-pp.remove_wait_queue
0.60 -0.1 0.53 ? 2% perf-profile.children.cycles-pp.__switch_to_asm
0.14 ? 5% -0.1 0.07 ? 8% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.14 ? 3% -0.1 0.07 ? 6% perf-profile.children.cycles-pp.task_h_load
0.23 ? 3% -0.1 0.16 ? 2% perf-profile.children.cycles-pp.kfree_skbmem
0.13 ? 2% -0.1 0.07 ? 7% perf-profile.children.cycles-pp.cpumask_next_wrap
0.51 ? 2% -0.1 0.44 ? 2% perf-profile.children.cycles-pp.update_process_times
0.74 ? 2% -0.1 0.68 ? 2% perf-profile.children.cycles-pp.hrtimer_interrupt
0.15 ? 3% -0.1 0.08 ? 5% perf-profile.children.cycles-pp.newidle_balance
0.51 -0.1 0.45 perf-profile.children.cycles-pp.tick_sched_handle
0.45 -0.1 0.39 perf-profile.children.cycles-pp.scheduler_tick
0.09 ? 4% -0.1 0.03 ? 70% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.13 ? 3% -0.1 0.07 ? 10% perf-profile.children.cycles-pp.pick_next_task_idle
0.12 ? 6% -0.1 0.06 ? 7% perf-profile.children.cycles-pp.rebalance_domains
0.61 ? 2% -0.1 0.55 perf-profile.children.cycles-pp.__hrtimer_run_queues
0.75 ? 3% -0.1 0.70 ? 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.54 -0.1 0.48 perf-profile.children.cycles-pp.tick_sched_timer
0.32 ? 2% -0.1 0.27 ? 3% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.22 -0.1 0.17 ? 2% perf-profile.children.cycles-pp.sock_rfree
0.11 -0.1 0.06 ? 11% perf-profile.children.cycles-pp.__update_idle_core
0.16 ? 5% -0.1 0.11 ? 5% perf-profile.children.cycles-pp.poll_idle
0.39 ? 2% -0.0 0.34 ? 2% perf-profile.children.cycles-pp.task_tick_fair
0.11 ? 8% -0.0 0.06 ? 7% perf-profile.children.cycles-pp.load_balance
0.10 ? 4% -0.0 0.05 ? 8% perf-profile.children.cycles-pp.__slab_free
0.44 ? 2% -0.0 0.40 ? 3% perf-profile.children.cycles-pp.__switch_to
0.28 ? 3% -0.0 0.24 ? 3% perf-profile.children.cycles-pp.skb_page_frag_refill
0.09 ? 5% -0.0 0.05 ? 7% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.53 -0.0 0.50 ? 3% perf-profile.children.cycles-pp.__inet_lookup_established
0.57 ? 2% -0.0 0.54 perf-profile.children.cycles-pp.restore_fpregs_from_fpstate
0.09 ? 12% -0.0 0.06 ? 46% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.23 ? 2% -0.0 0.20 ? 3% perf-profile.children.cycles-pp.tcp_queue_rcv
0.22 ? 3% -0.0 0.19 perf-profile.children.cycles-pp.update_rq_clock_task
0.06 ? 7% -0.0 0.03 ? 70% perf-profile.children.cycles-pp.rb_first
0.25 ? 3% -0.0 0.22 ? 3% perf-profile.children.cycles-pp.skb_attempt_defer_free
0.08 ? 7% -0.0 0.06 ? 8% perf-profile.children.cycles-pp.__build_skb_around
0.29 ? 2% -0.0 0.27 ? 3% perf-profile.children.cycles-pp.sk_page_frag_refill
0.17 ? 4% -0.0 0.15 ? 3% perf-profile.children.cycles-pp.cpuacct_charge
0.08 ? 6% -0.0 0.06 ? 6% perf-profile.children.cycles-pp.cubictcp_cwnd_event
0.07 ? 6% -0.0 0.06 ? 9% perf-profile.children.cycles-pp.ntohl@plt
0.05 +0.0 0.06 perf-profile.children.cycles-pp.audit_signal_info_syscall
0.05 +0.0 0.06 ? 6% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit
0.07 +0.0 0.08 ? 5% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.06 ? 7% +0.0 0.08 perf-profile.children.cycles-pp.tcp_rate_check_app_limited
0.05 ? 8% +0.0 0.07 ? 5% perf-profile.children.cycles-pp.clear_buddies
0.05 ? 7% +0.0 0.07 ? 5% perf-profile.children.cycles-pp.__list_add_valid
0.12 ? 4% +0.0 0.14 ? 5% perf-profile.children.cycles-pp.__list_del_entry_valid
0.06 +0.0 0.08 ? 8% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context
0.07 +0.0 0.09 perf-profile.children.cycles-pp.skb_release_head_state
0.25 ? 3% +0.0 0.27 ? 4% perf-profile.children.cycles-pp.__mod_timer
0.24 ? 3% +0.0 0.27 ? 5% perf-profile.children.cycles-pp.__update_load_avg_se
0.05 ? 7% +0.0 0.08 ? 6% perf-profile.children.cycles-pp.strchr@plt
0.52 +0.0 0.55 ? 2% perf-profile.children.cycles-pp.sched_clock_cpu
0.07 ? 6% +0.0 0.10 ? 6% perf-profile.children.cycles-pp.is_ftrace_trampoline
0.06 ? 7% +0.0 0.09 ? 4% perf-profile.children.cycles-pp.cubictcp_acked
0.06 ? 7% +0.0 0.09 ? 6% perf-profile.children.cycles-pp.rb_erase
0.05 ? 7% +0.0 0.08 ? 6% perf-profile.children.cycles-pp.rcu_all_qs
0.04 ? 45% +0.0 0.07 ? 5% perf-profile.children.cycles-pp.__usecs_to_jiffies
0.07 ? 5% +0.0 0.10 ? 6% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.12 ? 4% +0.0 0.14 ? 3% perf-profile.children.cycles-pp.__ip_local_out
0.07 ? 5% +0.0 0.10 ? 6% perf-profile.children.cycles-pp.check_kill_permission
0.10 ? 5% +0.0 0.13 ? 3% perf-profile.children.cycles-pp.nb_target_rate
0.08 ? 8% +0.0 0.11 ? 4% perf-profile.children.cycles-pp.import_single_range
0.15 ? 7% +0.0 0.18 ? 5% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.09 ? 4% +0.0 0.12 ? 5% perf-profile.children.cycles-pp.tcp_update_pacing_rate
0.06 ? 6% +0.0 0.09 ? 7% perf-profile.children.cycles-pp.sock_put
1.22 +0.0 1.26 perf-profile.children.cycles-pp._copy_from_iter
0.08 ? 4% +0.0 0.11 ? 4% perf-profile.children.cycles-pp.tcp_event_data_recv
0.09 ? 6% +0.0 0.12 ? 6% perf-profile.children.cycles-pp.__is_insn_slot_addr
0.08 ? 6% +0.0 0.11 ? 6% perf-profile.children.cycles-pp.__radix_tree_lookup
0.09 ? 6% +0.0 0.13 ? 5% perf-profile.children.cycles-pp.perf_instruction_pointer
0.36 +0.0 0.40 ? 2% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.15 ? 2% +0.0 0.19 ? 5% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.13 ? 2% +0.0 0.17 ? 2% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.02 ?141% +0.0 0.06 ? 9% perf-profile.children.cycles-pp.skb_clone
0.34 ? 3% +0.0 0.38 ? 2% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.02 ?141% +0.0 0.06 ? 8% perf-profile.children.cycles-pp.__xfrm_policy_check2
0.04 ? 44% +0.0 0.08 ? 8% perf-profile.children.cycles-pp.inet_ehashfn
0.09 ? 8% +0.0 0.13 ? 3% perf-profile.children.cycles-pp.tcp_rearm_rto
0.44 +0.0 0.48 ? 2% perf-profile.children.cycles-pp.native_sched_clock
0.12 ? 4% +0.0 0.17 ? 2% perf-profile.children.cycles-pp.ip_local_out
0.38 ? 2% +0.0 0.43 ? 3% perf-profile.children.cycles-pp.set_next_entity
0.19 ? 5% +0.0 0.23 ? 5% perf-profile.children.cycles-pp.kvm_guest_state
0.04 ? 44% +0.0 0.08 ? 5% perf-profile.children.cycles-pp.rb_insert_color
0.06 ? 13% +0.0 0.10 ? 5% perf-profile.children.cycles-pp.netif_skb_features
0.01 ?223% +0.0 0.06 ? 9% perf-profile.children.cycles-pp.kmalloc_slab
0.97 +0.0 1.02 ? 2% perf-profile.children.cycles-pp._copy_to_iter
0.40 ? 2% +0.0 0.45 ? 2% perf-profile.children.cycles-pp.__fdget
0.08 ? 7% +0.1 0.13 ? 4% perf-profile.children.cycles-pp.pick_next_entity
0.00 +0.1 0.05 perf-profile.children.cycles-pp.tcp_data_ready
0.00 +0.1 0.05 perf-profile.children.cycles-pp.schedule_debug
0.26 ? 2% +0.1 0.31 ? 2% perf-profile.children.cycles-pp.__skb_clone
0.00 +0.1 0.05 ? 7% perf-profile.children.cycles-pp.apparmor_socket_sock_rcv_skb
0.08 ? 5% +0.1 0.14 ? 6% perf-profile.children.cycles-pp.nf_hook_slow
0.09 ? 5% +0.1 0.14 ? 9% perf-profile.children.cycles-pp.ip_rcv_core
0.05 ? 8% +0.1 0.11 ? 6% perf-profile.children.cycles-pp.__copy_skb_header
0.11 ? 8% +0.1 0.17 ? 4% perf-profile.children.cycles-pp.write_sock
0.13 +0.1 0.19 ? 7% perf-profile.children.cycles-pp.perf_misc_flags
0.00 +0.1 0.06 ? 11% perf-profile.children.cycles-pp.tcp_chrono_start
0.01 ?223% +0.1 0.07 ? 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.09 ? 8% +0.1 0.14 ? 11% perf-profile.children.cycles-pp.ip_skb_dst_mtu
0.00 +0.1 0.06 perf-profile.children.cycles-pp.tcp_established_options
0.00 +0.1 0.06 perf-profile.children.cycles-pp.save_fpregs_to_fpstate
0.00 +0.1 0.06 ? 13% perf-profile.children.cycles-pp.eth_type_trans
0.09 ? 5% +0.1 0.15 ? 4% perf-profile.children.cycles-pp.all_string_sub
0.00 +0.1 0.06 ? 6% perf-profile.children.cycles-pp.is_module_text_address
0.25 ? 8% +0.1 0.31 ? 6% perf-profile.children.cycles-pp.ipv4_mtu
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.apparmor_ip_postroute
0.19 ? 7% +0.1 0.25 ? 10% perf-profile.children.cycles-pp.dst_release
0.26 +0.1 0.32 ? 3% perf-profile.children.cycles-pp.sio_rename
0.18 ? 5% +0.1 0.25 perf-profile.children.cycles-pp.os_xsave
0.38 +0.1 0.45 ? 2% perf-profile.children.cycles-pp.perf_output_begin_forward
0.44 ? 2% +0.1 0.52 ? 3% perf-profile.children.cycles-pp.check_heap_object
0.42 ? 4% +0.1 0.49 ? 2% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.00 +0.1 0.07 ? 5% perf-profile.children.cycles-pp.__tcp_select_window
0.00 +0.1 0.07 ? 6% perf-profile.children.cycles-pp.tcp_v4_fill_cb
0.16 ? 5% +0.1 0.24 ? 3% perf-profile.children.cycles-pp.tcp_skb_entail
0.64 +0.1 0.72 ? 3% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.00 +0.1 0.08 ? 6% perf-profile.children.cycles-pp.cubictcp_cong_avoid
0.07 ? 5% +0.1 0.15 ? 5% perf-profile.children.cycles-pp.tcp_tso_segs
0.00 +0.1 0.08 ? 4% perf-profile.children.cycles-pp.memcg_slab_free_hook
0.46 ? 2% +0.1 0.54 ? 3% perf-profile.children.cycles-pp.sockfd_lookup_light
0.08 +0.1 0.16 ? 2% perf-profile.children.cycles-pp.tcp_wfree
0.08 ? 6% +0.1 0.16 ? 6% perf-profile.children.cycles-pp.__calc_delta
0.33 ? 2% +0.1 0.41 ? 2% perf-profile.children.cycles-pp.tcp_rcv_space_adjust
0.15 ? 3% +0.1 0.24 ? 5% perf-profile.children.cycles-pp.__might_sleep
0.26 +0.1 0.34 ? 3% perf-profile.children.cycles-pp.enqueue_to_backlog
0.29 +0.1 0.38 perf-profile.children.cycles-pp.__might_resched
0.18 ? 12% +0.1 0.27 ? 10% perf-profile.children.cycles-pp.__sk_dst_check
0.18 ? 2% +0.1 0.27 ? 3% perf-profile.children.cycles-pp.dopr_outch
0.13 ? 2% +0.1 0.22 ? 4% perf-profile.children.cycles-pp.kmem_cache_free
0.20 ? 5% +0.1 0.29 ? 3% perf-profile.children.cycles-pp.validate_xmit_skb
0.00 +0.1 0.09 ? 9% perf-profile.children.cycles-pp.tcp_inbound_md5_hash
0.30 +0.1 0.39 ? 3% perf-profile.children.cycles-pp.__netif_rx
0.28 +0.1 0.38 ? 3% perf-profile.children.cycles-pp.netif_rx_internal
0.12 ? 5% +0.1 0.22 ? 8% perf-profile.children.cycles-pp.__ip_finish_output
0.20 ? 5% +0.1 0.29 ? 2% perf-profile.children.cycles-pp.irqtime_account_irq
0.19 ? 14% +0.1 0.28 ? 10% perf-profile.children.cycles-pp.ipv4_dst_check
0.12 ? 3% +0.1 0.22 ? 7% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.39 ? 2% +0.1 0.50 ? 3% perf-profile.children.cycles-pp.simple_copy_to_iter
0.42 +0.1 0.52 ? 3% perf-profile.children.cycles-pp.sio_flush
0.10 ? 3% +0.1 0.20 ? 6% perf-profile.children.cycles-pp.check_preempt_curr
0.19 ? 5% +0.1 0.29 ? 3% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.19 ? 11% +0.1 0.30 ? 11% perf-profile.children.cycles-pp.ip_rcv_finish_core
0.09 ? 6% +0.1 0.20 ? 8% perf-profile.children.cycles-pp.tcp_rtt_estimator
0.01 ?223% +0.1 0.12 ? 7% perf-profile.children.cycles-pp.set_next_buddy
0.00 +0.1 0.11 ? 5% perf-profile.children.cycles-pp.security_sock_rcv_skb
0.20 ? 2% +0.1 0.32 ? 6% perf-profile.children.cycles-pp.tcp_schedule_loss_probe
0.13 ? 4% +0.1 0.24 ? 5% perf-profile.children.cycles-pp.kfree
2.34 +0.1 2.45 perf-profile.children.cycles-pp.__kernel_text_address
0.29 ? 2% +0.1 0.40 ? 3% perf-profile.children.cycles-pp.lock_sock_nested
0.14 ? 3% +0.1 0.26 ? 2% perf-profile.children.cycles-pp.ip_output
0.23 ? 2% +0.1 0.35 ? 2% perf-profile.children.cycles-pp.read_tsc
0.48 ? 2% +0.1 0.61 ? 2% perf-profile.children.cycles-pp.sio_sfileinfo
0.16 ? 5% +0.1 0.29 ? 7% perf-profile.children.cycles-pp.__virt_addr_valid
0.08 ? 4% +0.1 0.21 ? 4% perf-profile.children.cycles-pp.__might_fault
1.39 +0.1 1.52 perf-profile.children.cycles-pp.skb_do_copy_data_nocache
1.73 +0.1 1.87 ? 2% perf-profile.children.cycles-pp.__skb_datagram_iter
0.24 ? 2% +0.1 0.38 ? 4% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.43 ? 2% +0.1 0.58 ? 3% perf-profile.children.cycles-pp.__kfree_skb
0.00 +0.2 0.15 ? 6% perf-profile.children.cycles-pp.check_preempt_wakeup
0.34 ? 2% +0.2 0.49 ? 3% perf-profile.children.cycles-pp.fmtstr
0.22 ? 3% +0.2 0.38 ? 5% perf-profile.children.cycles-pp.reweight_entity
0.02 ? 99% +0.2 0.18 ? 5% perf-profile.children.cycles-pp.put_prev_entity
1.75 +0.2 1.91 ? 3% perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.54 +0.2 0.70 ? 3% perf-profile.children.cycles-pp.__check_object_size
0.27 ? 3% +0.2 0.44 ? 4% perf-profile.children.cycles-pp.kmalloc_reserve
0.35 ? 5% +0.2 0.52 ? 6% perf-profile.children.cycles-pp.tcp_current_mss
0.13 ? 2% +0.2 0.30 ? 6% perf-profile.children.cycles-pp.tcp_ack_update_rtt
0.38 ? 5% +0.2 0.55 ? 5% perf-profile.children.cycles-pp.tcp_send_mss
0.21 ? 6% +0.2 0.39 ? 3% perf-profile.children.cycles-pp.__netif_receive_skb_core
2.86 +0.2 3.04 perf-profile.children.cycles-pp.unwind_get_return_address
0.22 ? 2% +0.2 0.41 ? 3% perf-profile.children.cycles-pp.tcp_mstamp_refresh
0.41 ? 3% +0.2 0.59 ? 4% perf-profile.children.cycles-pp.ktime_get
0.00 +0.2 0.18 ? 6% perf-profile.children.cycles-pp.aa_may_signal
0.00 +0.2 0.19 ? 7% perf-profile.children.cycles-pp.__release_sock
0.29 +0.2 0.48 ? 3% perf-profile.children.cycles-pp.__entry_text_start
0.33 ? 6% +0.2 0.53 ? 7% perf-profile.children.cycles-pp.ip_rcv
0.49 ? 2% +0.2 0.70 ? 2% perf-profile.children.cycles-pp.next_token
0.50 ? 2% +0.2 0.71 ? 2% perf-profile.children.cycles-pp.dopr
0.52 ? 2% +0.2 0.73 ? 2% perf-profile.children.cycles-pp.vsnprintf
0.52 +0.2 0.74 ? 2% perf-profile.children.cycles-pp.snprintf
0.94 +0.2 1.18 ? 3% perf-profile.children.cycles-pp.sio_qfileinfo
0.15 ? 3% +0.2 0.39 ? 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.83 +0.2 1.07 ? 3% perf-profile.children.cycles-pp.pick_next_task_fair
0.46 +0.2 0.70 ? 4% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.98 +0.3 1.24 ? 2% perf-profile.children.cycles-pp.sio_qfsinfo
0.62 +0.3 0.90 ? 2% perf-profile.children.cycles-pp.loopback_xmit
0.54 ? 5% +0.3 0.82 ? 4% perf-profile.children.cycles-pp.release_sock
0.12 ? 3% +0.3 0.42 ? 4% perf-profile.children.cycles-pp.__cond_resched
1.20 +0.3 1.51 ? 2% perf-profile.children.cycles-pp.sio_unlink
0.66 ? 2% +0.3 0.97 ? 2% perf-profile.children.cycles-pp.dev_hard_start_xmit
2.34 +0.4 2.70 ? 2% perf-profile.children.cycles-pp.orc_find
0.77 +0.4 1.18 ? 2% perf-profile.children.cycles-pp.__alloc_skb
3.81 +0.4 4.23 ? 2% perf-profile.children.cycles-pp.__orc_find
1.00 +0.5 1.46 ? 2% perf-profile.children.cycles-pp.tcp_stream_alloc_skb
2.36 +0.5 2.85 ? 2% perf-profile.children.cycles-pp.stack_access_ok
0.26 ? 3% +0.5 0.76 ? 8% perf-profile.children.cycles-pp.aa_get_task_label
2.11 +0.5 2.61 ? 2% perf-profile.children.cycles-pp.sio_findfirst
1.41 +0.5 1.95 ? 4% perf-profile.children.cycles-pp.__dev_queue_xmit
1.36 +0.6 1.94 ? 2% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
0.17 ? 4% +0.6 0.76 ? 14% perf-profile.children.cycles-pp.security_socket_sendmsg
2.17 +0.8 2.98 ? 2% perf-profile.children.cycles-pp.tcp_ack
6.14 +0.9 7.00 perf-profile.children.cycles-pp.__unwind_start
8.64 +0.9 9.55 ? 5% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
4.35 +1.1 5.43 ? 2% perf-profile.children.cycles-pp.sio_close
0.42 ? 4% +1.2 1.60 ? 13% perf-profile.children.cycles-pp.sock_recvmsg
0.39 ? 3% +1.2 1.57 ? 13% perf-profile.children.cycles-pp.security_socket_recvmsg
33.90 +1.2 35.11 ? 2% perf-profile.children.cycles-pp.server
33.95 +1.2 35.18 ? 2% perf-profile.children.cycles-pp.listener
3.62 +1.3 4.92 ? 3% perf-profile.children.cycles-pp.sio_writex
5.44 +1.4 6.80 ? 2% perf-profile.children.cycles-pp.sio_qpathinfo
0.41 ? 2% +1.4 1.81 ? 6% perf-profile.children.cycles-pp.apparmor_task_kill
0.41 ? 2% +1.4 1.82 ? 6% perf-profile.children.cycles-pp.security_task_kill
0.50 ? 2% +1.4 1.94 ? 6% perf-profile.children.cycles-pp.kill_pid_info
6.00 +1.5 7.48 ? 2% perf-profile.children.cycles-pp.sio_createx
0.61 ? 2% +1.5 2.11 ? 5% perf-profile.children.cycles-pp.kill_something_info
0.76 ? 2% +1.5 2.28 ? 5% perf-profile.children.cycles-pp.__x64_sys_kill
1.00 +1.7 2.73 ? 5% perf-profile.children.cycles-pp.kill
0.42 ? 3% +1.8 2.19 ? 14% perf-profile.children.cycles-pp.aa_sk_perm
26.32 +1.9 28.24 ? 3% perf-profile.children.cycles-pp.tcp_rcv_established
26.42 +2.0 28.39 ? 3% perf-profile.children.cycles-pp.tcp_v4_do_rcv
7.52 +2.0 9.53 ? 4% perf-profile.children.cycles-pp.perf_trace_sched_switch
30.22 +2.1 32.28 ? 2% perf-profile.children.cycles-pp.net_rx_action
27.78 +2.1 29.85 ? 3% perf-profile.children.cycles-pp.tcp_v4_rcv
27.92 +2.1 29.98 ? 3% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
27.98 +2.1 30.04 ? 2% perf-profile.children.cycles-pp.ip_local_deliver_finish
2.28 +2.1 4.37 ? 36% perf-profile.children.cycles-pp.__get_user_nocheck_8
30.76 +2.1 32.89 ? 2% perf-profile.children.cycles-pp.__softirqentry_text_start
30.73 +2.2 32.95 ? 2% perf-profile.children.cycles-pp.do_softirq
2.85 +2.2 5.08 ? 30% perf-profile.children.cycles-pp.perf_callchain_user
9.38 +2.3 11.64 ? 2% perf-profile.children.cycles-pp.sio_readx
30.88 +2.4 33.23 ? 2% perf-profile.children.cycles-pp.__local_bh_enable_ip
15.50 +2.4 17.88 ? 2% perf-profile.children.cycles-pp.unwind_next_frame
18.11 +2.5 20.60 ? 2% perf-profile.children.cycles-pp.schedule
28.64 +2.5 31.14 ? 2% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
28.89 +2.5 31.44 ? 2% perf-profile.children.cycles-pp.process_backlog
28.93 +2.6 31.49 ? 2% perf-profile.children.cycles-pp.__napi_poll
20.63 +2.8 23.39 ? 2% perf-profile.children.cycles-pp.perf_callchain_kernel
32.43 +2.9 35.34 ? 2% perf-profile.children.cycles-pp.ip_finish_output2
4.89 ? 2% +3.1 8.02 ? 4% perf-profile.children.cycles-pp.enqueue_task_fair
4.04 ? 2% +3.1 7.17 ? 4% perf-profile.children.cycles-pp.enqueue_entity
4.93 ? 2% +3.2 8.08 ? 4% perf-profile.children.cycles-pp.ttwu_do_activate
33.32 +3.2 36.53 ? 2% perf-profile.children.cycles-pp.__ip_queue_xmit
34.40 +3.6 37.96 ? 2% perf-profile.children.cycles-pp.__tcp_transmit_skb
0.93 ? 2% +3.8 4.77 ? 5% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.00 +3.9 3.86 ? 7% perf-profile.children.cycles-pp.exit_to_user_mode_loop
34.80 +3.9 38.73 ? 2% perf-profile.children.cycles-pp.__tcp_push_pending_frames
35.32 +4.1 39.41 ? 2% perf-profile.children.cycles-pp.tcp_write_xmit
1.06 +4.6 5.62 ? 23% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
7.76 +4.7 12.47 ? 3% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
8.48 +4.9 13.33 ? 3% perf-profile.children.cycles-pp.update_curr
27.72 +4.9 32.66 ? 4% perf-profile.children.cycles-pp.perf_event_output_forward
27.82 +5.0 32.77 ? 4% perf-profile.children.cycles-pp.__perf_event_overflow
39.42 +5.0 44.44 ? 2% perf-profile.children.cycles-pp.tcp_sendmsg_locked
23.95 +5.0 29.00 ? 4% perf-profile.children.cycles-pp.get_perf_callchain
24.04 +5.1 29.11 ? 4% perf-profile.children.cycles-pp.perf_callchain
28.56 +5.1 33.66 ? 4% perf-profile.children.cycles-pp.perf_tp_event
25.16 +5.2 30.40 ? 4% perf-profile.children.cycles-pp.perf_prepare_sample
39.78 +5.3 45.10 ? 2% perf-profile.children.cycles-pp.tcp_sendmsg
40.09 +5.9 45.99 ? 2% perf-profile.children.cycles-pp.sock_sendmsg
40.27 +6.0 46.29 ? 2% perf-profile.children.cycles-pp.__sys_sendto
40.33 +6.1 46.39 ? 2% perf-profile.children.cycles-pp.__x64_sys_sendto
34.03 +8.7 42.75 ? 2% perf-profile.children.cycles-pp.do_packets
35.85 +9.2 45.03 ? 2% perf-profile.children.cycles-pp.child_op
37.62 +9.8 47.41 ? 2% perf-profile.children.cycles-pp.child_run
40.90 +10.9 51.79 perf-profile.children.cycles-pp.__send
38.72 +11.5 50.25 perf-profile.children.cycles-pp.create_procs
68.40 +12.0 80.44 perf-profile.children.cycles-pp.do_syscall_64
68.64 +12.1 80.78 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
72.71 +12.8 85.46 perf-profile.children.cycles-pp.main
72.71 +12.8 85.47 perf-profile.children.cycles-pp.__libc_start_main
15.01 -7.7 7.34 ? 4% perf-profile.self.cycles-pp.mwait_idle_with_hints
2.49 ? 7% -1.4 1.10 ? 9% perf-profile.self.cycles-pp.update_cfs_group
2.48 ? 4% -1.3 1.16 ? 3% perf-profile.self.cycles-pp.update_load_avg
0.80 ? 8% -0.6 0.16 ? 8% perf-profile.self.cycles-pp.set_task_cpu
1.61 -0.6 1.02 ? 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.16 ? 2% -0.6 0.60 ? 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.13 -0.5 0.60 ? 5% perf-profile.self.cycles-pp.available_idle_cpu
0.87 -0.3 0.56 ? 3% perf-profile.self.cycles-pp._raw_spin_lock
0.46 ? 3% -0.2 0.24 ? 5% perf-profile.self.cycles-pp.select_idle_core
0.87 -0.2 0.71 perf-profile.self.cycles-pp.__schedule
0.32 ? 2% -0.2 0.16 ? 5% perf-profile.self.cycles-pp.menu_select
0.30 -0.1 0.15 ? 7% perf-profile.self.cycles-pp.llist_add_batch
0.32 ? 2% -0.1 0.17 ? 6% perf-profile.self.cycles-pp.migrate_task_rq_fair
0.22 ? 6% -0.1 0.08 ? 9% perf-profile.self.cycles-pp.bpf_ksym_find
0.32 ? 2% -0.1 0.19 ? 8% perf-profile.self.cycles-pp.perf_output_end
0.23 ? 4% -0.1 0.12 ? 6% perf-profile.self.cycles-pp._find_next_bit
0.99 -0.1 0.88 ? 6% perf-profile.self.cycles-pp.memcpy_erms
0.36 -0.1 0.26 perf-profile.self.cycles-pp.wait_woken
0.19 ? 3% -0.1 0.09 ? 7% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.55 -0.1 0.45 ? 3% perf-profile.self.cycles-pp.skb_release_data
0.18 ? 5% -0.1 0.09 ? 5% perf-profile.self.cycles-pp.sched_ttwu_pending
0.18 ? 2% -0.1 0.09 ? 7% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.54 -0.1 0.45 ? 3% perf-profile.self.cycles-pp.recv
0.28 ? 3% -0.1 0.19 ? 3% perf-profile.self.cycles-pp.perf_event_output_forward
0.14 ? 2% -0.1 0.06 ? 8% perf-profile.self.cycles-pp.sk_filter_trim_cap
0.17 ? 2% -0.1 0.09 ? 10% perf-profile.self.cycles-pp.llist_reverse_order
0.36 ? 7% -0.1 0.27 ? 5% perf-profile.self.cycles-pp.read_sock
0.24 ? 3% -0.1 0.17 ? 5% perf-profile.self.cycles-pp.finish_task_switch
0.48 -0.1 0.41 ? 4% perf-profile.self.cycles-pp.__inet_lookup_established
0.60 -0.1 0.52 perf-profile.self.cycles-pp.__switch_to_asm
0.14 ? 3% -0.1 0.07 ? 6% perf-profile.self.cycles-pp.task_h_load
0.23 ? 4% -0.1 0.16 ? 5% perf-profile.self.cycles-pp.kfree_skbmem
0.16 ? 2% -0.1 0.10 ? 4% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.66 -0.1 0.59 perf-profile.self.cycles-pp.tcp_recvmsg_locked
0.39 ? 3% -0.1 0.32 ? 3% perf-profile.self.cycles-pp.net_rx_action
0.27 ? 2% -0.1 0.21 ? 3% perf-profile.self.cycles-pp.check_heap_object
0.18 ? 2% -0.1 0.11 ? 6% perf-profile.self.cycles-pp.get_stack_info_noinstr
0.27 ? 3% -0.1 0.21 ? 4% perf-profile.self.cycles-pp.switch_fpu_return
0.23 ? 2% -0.1 0.17 ? 4% perf-profile.self.cycles-pp.skb_page_frag_refill
0.08 ? 5% -0.1 0.02 ? 99% perf-profile.self.cycles-pp.__update_idle_core
0.22 ? 2% -0.1 0.16 ? 3% perf-profile.self.cycles-pp.sock_rfree
0.10 ? 4% -0.1 0.05 ? 8% perf-profile.self.cycles-pp.do_idle
0.24 ? 3% -0.0 0.19 ? 3% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.10 ? 4% -0.0 0.05 ? 7% perf-profile.self.cycles-pp.__slab_free
0.15 ? 4% -0.0 0.11 perf-profile.self.cycles-pp.select_idle_sibling
0.18 ? 5% -0.0 0.13 ? 2% perf-profile.self.cycles-pp.tcp_rcv_space_adjust
0.16 ? 2% -0.0 0.12 ? 4% perf-profile.self.cycles-pp.get_stack_info
0.43 ? 3% -0.0 0.39 ? 3% perf-profile.self.cycles-pp.__switch_to
0.10 ? 3% -0.0 0.06 perf-profile.self.cycles-pp.newidle_balance
0.15 -0.0 0.11 ? 6% perf-profile.self.cycles-pp.schedule_timeout
0.33 ? 3% -0.0 0.29 ? 4% perf-profile.self.cycles-pp.__skb_datagram_iter
0.19 -0.0 0.15 ? 4% perf-profile.self.cycles-pp.try_to_wake_up
0.10 ? 3% -0.0 0.06 ? 7% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.23 -0.0 0.20 ? 3% perf-profile.self.cycles-pp.tcp_queue_rcv
0.57 ? 2% -0.0 0.54 perf-profile.self.cycles-pp.restore_fpregs_from_fpstate
0.16 ? 3% -0.0 0.12 ? 4% perf-profile.self.cycles-pp.update_rq_clock_task
0.12 ? 7% -0.0 0.09 ? 6% perf-profile.self.cycles-pp.poll_idle
0.11 ? 3% -0.0 0.08 ? 5% perf-profile.self.cycles-pp.select_task_rq
0.08 ? 6% -0.0 0.06 ? 9% perf-profile.self.cycles-pp.cubictcp_cwnd_event
0.08 ? 6% -0.0 0.06 ? 8% perf-profile.self.cycles-pp.__build_skb_around
0.13 ? 3% -0.0 0.11 ? 4% perf-profile.self.cycles-pp.dequeue_entity
0.07 ? 6% -0.0 0.06 ? 9% perf-profile.self.cycles-pp.ntohl@plt
0.21 ? 2% -0.0 0.19 ? 3% perf-profile.self.cycles-pp.prepare_task_switch
0.05 +0.0 0.06 perf-profile.self.cycles-pp.audit_signal_info_syscall
0.06 ? 7% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.__perf_event_overflow
0.05 +0.0 0.06 ? 6% perf-profile.self.cycles-pp.clear_buddies
0.05 +0.0 0.06 ? 6% perf-profile.self.cycles-pp.perf_trace_run_bpf_submit
0.07 ? 5% +0.0 0.08 ? 4% perf-profile.self.cycles-pp.skb_release_head_state
0.05 +0.0 0.07 ? 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 +0.0 0.10 ? 4% perf-profile.self.cycles-pp.set_next_entity
0.26 ? 2% +0.0 0.28 ? 2% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.05 ? 7% +0.0 0.07 ? 14% perf-profile.self.cycles-pp.kill
0.08 ? 6% +0.0 0.10 ? 7% perf-profile.self.cycles-pp.sock_sendmsg
0.06 ? 8% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.nb_target_rate
0.27 ? 3% +0.0 0.30 ? 3% perf-profile.self.cycles-pp.perf_prepare_sample
0.13 +0.0 0.15 ? 3% perf-profile.self.cycles-pp.__alloc_skb
0.07 ? 6% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.is_ftrace_trampoline
0.06 ? 9% +0.0 0.08 ? 8% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.12 ? 4% +0.0 0.14 ? 3% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.17 ? 4% +0.0 0.19 ? 5% perf-profile.self.cycles-pp.__mod_timer
0.06 ? 7% +0.0 0.09 ? 4% perf-profile.self.cycles-pp.cubictcp_acked
0.06 ? 11% +0.0 0.08 ? 5% perf-profile.self.cycles-pp.rb_erase
0.12 ? 3% +0.0 0.15 ? 4% perf-profile.self.cycles-pp.do_softirq
0.07 ? 7% +0.0 0.09 ? 7% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.18 ? 5% +0.0 0.21 ? 5% perf-profile.self.cycles-pp.kvm_guest_state
0.12 ? 7% +0.0 0.15 ? 7% perf-profile.self.cycles-pp.release_sock
0.18 ? 2% +0.0 0.20 ? 3% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.08 ? 6% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.tcp_event_data_recv
0.08 ? 6% +0.0 0.11 ? 4% perf-profile.self.cycles-pp.import_single_range
0.09 ? 4% +0.0 0.12 ? 3% perf-profile.self.cycles-pp.tcp_update_pacing_rate
0.04 ? 44% +0.0 0.07 ? 9% perf-profile.self.cycles-pp.is_bpf_text_address
0.06 ? 6% +0.0 0.09 ? 12% perf-profile.self.cycles-pp.sock_put
0.07 ? 7% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.irqtime_account_irq
0.20 ? 3% +0.0 0.23 ? 5% perf-profile.self.cycles-pp.__softirqentry_text_start
0.34 ? 2% +0.0 0.37 ? 2% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.15 ? 4% +0.0 0.18 ? 6% perf-profile.self.cycles-pp.reweight_entity
0.06 ? 6% +0.0 0.09 ? 6% perf-profile.self.cycles-pp.sockfd_lookup_light
0.32 +0.0 0.35 ? 2% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.08 ? 4% +0.0 0.12 ? 4% perf-profile.self.cycles-pp._copy_from_iter
0.15 ? 3% +0.0 0.18 ? 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.30 ? 4% +0.0 0.33 ? 3% perf-profile.self.cycles-pp.do_packets
0.14 ? 5% +0.0 0.18 ? 2% perf-profile.self.cycles-pp.tcp_skb_entail
0.07 ? 6% +0.0 0.11 ? 5% perf-profile.self.cycles-pp.__radix_tree_lookup
0.12 ? 3% +0.0 0.16 ? 5% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.08 ? 5% +0.0 0.12 ? 4% perf-profile.self.cycles-pp.__is_insn_slot_addr
0.06 ? 7% +0.0 0.10 perf-profile.self.cycles-pp.__x64_sys_sendto
0.42 +0.0 0.46 ? 2% perf-profile.self.cycles-pp.native_sched_clock
0.08 ? 9% +0.0 0.12 ? 4% perf-profile.self.cycles-pp.tcp_rearm_rto
0.30 ? 4% +0.0 0.34 ? 2% perf-profile.self.cycles-pp.get_perf_callchain
0.08 ? 6% +0.0 0.11 ? 4% perf-profile.self.cycles-pp.schedule
0.07 ? 8% +0.0 0.11 ? 3% perf-profile.self.cycles-pp.pick_next_entity
0.06 ? 7% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.__cond_resched
0.06 ? 11% +0.0 0.10 ? 6% perf-profile.self.cycles-pp.tcp_recvmsg
0.04 ? 44% +0.0 0.08 ? 4% perf-profile.self.cycles-pp.__x64_sys_kill
0.16 ? 4% +0.0 0.20 ? 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.03 ? 70% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.nf_hook_slow
0.06 ? 9% +0.0 0.10 ? 4% perf-profile.self.cycles-pp.ip_rcv
0.24 ? 5% +0.0 0.28 ? 3% perf-profile.self.cycles-pp.__ip_queue_xmit
0.10 ? 4% +0.0 0.14 ? 6% perf-profile.self.cycles-pp.write_sock
0.39 ? 2% +0.0 0.44 ? 3% perf-profile.self.cycles-pp.__fdget
0.01 ?223% +0.0 0.06 ? 9% perf-profile.self.cycles-pp.kmalloc_slab
0.01 ?223% +0.0 0.06 ? 9% perf-profile.self.cycles-pp.__xfrm_policy_check2
0.88 +0.0 0.92 ? 2% perf-profile.self.cycles-pp.kernel_text_address
0.10 ? 7% +0.0 0.15 ? 3% perf-profile.self.cycles-pp.__send
0.02 ? 99% +0.0 0.08 ? 6% perf-profile.self.cycles-pp.tcp_sendmsg
0.08 ? 6% +0.1 0.12 ? 6% perf-profile.self.cycles-pp.__netif_receive_skb_one_core
0.08 ? 5% +0.1 0.14 ? 7% perf-profile.self.cycles-pp.ip_rcv_core
0.00 +0.1 0.05 perf-profile.self.cycles-pp.skb_clone
0.00 +0.1 0.05 perf-profile.self.cycles-pp.tcp_data_ready
0.20 ? 2% +0.1 0.26 ? 3% perf-profile.self.cycles-pp.do_syscall_64
0.16 ? 4% +0.1 0.21 ? 4% perf-profile.self.cycles-pp.dopr
0.22 ? 2% +0.1 0.27 ? 3% perf-profile.self.cycles-pp.tcp_stream_alloc_skb
0.00 +0.1 0.05 ? 7% perf-profile.self.cycles-pp.__wake_up_common_lock
0.00 +0.1 0.05 ? 7% perf-profile.self.cycles-pp.exit_to_user_mode_loop
0.00 +0.1 0.05 ? 7% perf-profile.self.cycles-pp.skb_do_copy_data_nocache
0.00 +0.1 0.05 ? 7% perf-profile.self.cycles-pp.kmalloc_reserve
0.00 +0.1 0.05 ? 7% perf-profile.self.cycles-pp.save_fpregs_to_fpstate
0.09 +0.1 0.14 ? 4% perf-profile.self.cycles-pp.__sys_sendto
0.05 ? 8% +0.1 0.11 ? 4% perf-profile.self.cycles-pp.__copy_skb_header
0.00 +0.1 0.05 ? 8% perf-profile.self.cycles-pp.perf_instruction_pointer
0.00 +0.1 0.06 ? 9% perf-profile.self.cycles-pp.__napi_poll
0.00 +0.1 0.06 ? 9% perf-profile.self.cycles-pp.netif_skb_features
0.09 ? 5% +0.1 0.15 ? 6% perf-profile.self.cycles-pp.all_string_sub
0.08 ? 8% +0.1 0.14 ? 11% perf-profile.self.cycles-pp.ip_skb_dst_mtu
0.06 +0.1 0.12 ? 4% perf-profile.self.cycles-pp.ip_output
0.00 +0.1 0.06 ? 8% perf-profile.self.cycles-pp.security_sock_rcv_skb
0.19 ? 7% +0.1 0.24 ? 9% perf-profile.self.cycles-pp.dst_release
0.04 ? 44% +0.1 0.10 ? 8% perf-profile.self.cycles-pp.perf_misc_flags
0.00 +0.1 0.06 ? 6% perf-profile.self.cycles-pp.rcu_all_qs
0.00 +0.1 0.06 ? 6% perf-profile.self.cycles-pp.is_module_text_address
0.00 +0.1 0.06 ? 6% perf-profile.self.cycles-pp.tcp_established_options
0.00 +0.1 0.06 ? 11% perf-profile.self.cycles-pp.tcp_chrono_start
0.00 +0.1 0.06 ? 11% perf-profile.self.cycles-pp.eth_type_trans
0.07 ? 5% +0.1 0.13 ? 5% perf-profile.self.cycles-pp.enqueue_to_backlog
0.00 +0.1 0.06 perf-profile.self.cycles-pp.apparmor_ip_postroute
0.00 +0.1 0.06 ? 6% perf-profile.self.cycles-pp.__tcp_push_pending_frames
0.00 +0.1 0.06 ? 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.00 +0.1 0.06 ? 7% perf-profile.self.cycles-pp.check_preempt_wakeup
0.00 +0.1 0.06 ? 17% perf-profile.self.cycles-pp.__sk_dst_check
0.18 ? 5% +0.1 0.25 ? 9% perf-profile.self.cycles-pp.ktime_get
0.14 ? 3% +0.1 0.20 ? 4% perf-profile.self.cycles-pp.__might_sleep
0.50 ? 3% +0.1 0.56 perf-profile.self.cycles-pp.tcp_ack
0.00 +0.1 0.06 ? 7% perf-profile.self.cycles-pp.tcp_inbound_md5_hash
0.15 ? 3% +0.1 0.22 ? 3% perf-profile.self.cycles-pp.loopback_xmit
0.36 ? 2% +0.1 0.43 ? 3% perf-profile.self.cycles-pp.perf_output_begin_forward
0.00 +0.1 0.07 ? 11% perf-profile.self.cycles-pp.tcp_v4_do_rcv
0.52 +0.1 0.59 ? 2% perf-profile.self.cycles-pp.unwind_get_return_address
0.24 ? 9% +0.1 0.31 ? 6% perf-profile.self.cycles-pp.ipv4_mtu
0.18 ? 5% +0.1 0.25 perf-profile.self.cycles-pp.os_xsave
0.01 ?223% +0.1 0.08 ? 8% perf-profile.self.cycles-pp.rb_insert_color
0.01 ?223% +0.1 0.08 ? 4% perf-profile.self.cycles-pp.dev_hard_start_xmit
0.00 +0.1 0.07 perf-profile.self.cycles-pp.__tcp_select_window
0.07 ? 7% +0.1 0.14 ? 5% perf-profile.self.cycles-pp._copy_to_iter
0.10 ? 3% +0.1 0.17 ? 4% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.07 ? 5% +0.1 0.14 ? 5% perf-profile.self.cycles-pp.tcp_tso_segs
0.01 ?223% +0.1 0.08 ? 7% perf-profile.self.cycles-pp.inet_ehashfn
0.06 ? 7% +0.1 0.14 ? 6% perf-profile.self.cycles-pp.__check_object_size
0.17 ? 2% +0.1 0.24 ? 3% perf-profile.self.cycles-pp.fmtstr
0.00 +0.1 0.07 ? 6% perf-profile.self.cycles-pp.tcp_ack_update_rtt
0.00 +0.1 0.07 ? 6% perf-profile.self.cycles-pp.tcp_v4_fill_cb
0.00 +0.1 0.08 ? 6% perf-profile.self.cycles-pp.cubictcp_cong_avoid
0.08 +0.1 0.16 ? 3% perf-profile.self.cycles-pp.tcp_wfree
0.12 ? 4% +0.1 0.20 ? 6% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.00 +0.1 0.08 ? 6% perf-profile.self.cycles-pp.__ip_finish_output
0.00 +0.1 0.08 ? 6% perf-profile.self.cycles-pp.memcg_slab_free_hook
0.12 ? 5% +0.1 0.20 ? 6% perf-profile.self.cycles-pp.tcp_schedule_loss_probe
0.00 +0.1 0.08 ? 4% perf-profile.self.cycles-pp.__might_fault
0.50 ? 2% +0.1 0.58 ? 2% perf-profile.self.cycles-pp.child_op
0.38 +0.1 0.46 ? 2% perf-profile.self.cycles-pp.__kernel_text_address
0.29 ? 2% +0.1 0.37 ? 2% perf-profile.self.cycles-pp.__might_resched
0.18 ? 3% +0.1 0.26 ? 2% perf-profile.self.cycles-pp.dopr_outch
0.08 ? 6% +0.1 0.16 ? 6% perf-profile.self.cycles-pp.__calc_delta
0.40 ? 4% +0.1 0.48 perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.32 ? 2% +0.1 0.40 ? 3% perf-profile.self.cycles-pp.update_curr
0.00 +0.1 0.08 ? 5% perf-profile.self.cycles-pp.put_prev_entity
0.07 ? 7% +0.1 0.16 ? 3% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.06 ? 6% +0.1 0.15 ? 4% perf-profile.self.cycles-pp.kmem_cache_free
0.12 ? 3% +0.1 0.21 ? 4% perf-profile.self.cycles-pp.__entry_text_start
0.18 ? 14% +0.1 0.28 ? 11% perf-profile.self.cycles-pp.ipv4_dst_check
0.08 ? 8% +0.1 0.18 ? 7% perf-profile.self.cycles-pp.tcp_current_mss
0.37 ? 4% +0.1 0.47 ? 3% perf-profile.self.cycles-pp.child_run
0.24 ? 2% +0.1 0.33 ? 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.08 ? 7% +0.1 0.18 ? 8% perf-profile.self.cycles-pp.tcp_rtt_estimator
0.01 ?223% +0.1 0.11 ? 4% perf-profile.self.cycles-pp.set_next_buddy
0.18 ? 4% +0.1 0.28 ? 3% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.18 ? 12% +0.1 0.29 ? 11% perf-profile.self.cycles-pp.ip_rcv_finish_core
0.11 ? 6% +0.1 0.22 ? 3% perf-profile.self.cycles-pp.__sys_recvfrom
0.54 +0.1 0.66 ? 4% perf-profile.self.cycles-pp.perf_callchain_user
0.43 ? 2% +0.1 0.55 ? 7% perf-profile.self.cycles-pp.__dev_queue_xmit
0.22 ? 4% +0.1 0.34 ? 2% perf-profile.self.cycles-pp.read_tsc
0.50 +0.1 0.61 ? 2% perf-profile.self.cycles-pp.__unwind_start
0.20 ? 3% +0.1 0.32 ? 4% perf-profile.self.cycles-pp.ip_finish_output2
0.06 ? 6% +0.1 0.18 ? 7% perf-profile.self.cycles-pp.kfree
0.15 ? 4% +0.1 0.28 ? 7% perf-profile.self.cycles-pp.__virt_addr_valid
0.14 ? 3% +0.1 0.27 ? 3% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.53 ? 2% +0.1 0.66 ? 3% perf-profile.self.cycles-pp.perf_tp_event
0.22 ? 2% +0.2 0.38 perf-profile.self.cycles-pp.tcp_write_xmit
0.21 ? 6% +0.2 0.38 ? 3% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.42 ? 2% +0.2 0.60 ? 3% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.00 +0.2 0.18 ? 5% perf-profile.self.cycles-pp.aa_may_signal
0.49 +0.2 0.69 perf-profile.self.cycles-pp.next_token
1.66 +0.2 1.88 ? 3% perf-profile.self.cycles-pp.perf_callchain_kernel
0.46 +0.2 0.70 ? 4% perf-profile.self.cycles-pp.switch_mm_irqs_off
2.24 +0.3 2.58 ? 2% perf-profile.self.cycles-pp.orc_find
3.80 +0.4 4.22 ? 2% perf-profile.self.cycles-pp.__orc_find
2.10 +0.4 2.53 ? 2% perf-profile.self.cycles-pp.stack_access_ok
0.26 ? 2% +0.5 0.75 ? 8% perf-profile.self.cycles-pp.aa_get_task_label
0.06 ? 11% +0.7 0.76 ?200% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.10 ? 4% +0.8 0.86 ? 9% perf-profile.self.cycles-pp.apparmor_task_kill
6.86 +1.1 7.99 ? 2% perf-profile.self.cycles-pp.unwind_next_frame
2.24 +1.2 3.46 ? 20% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.34 ? 3% +1.7 2.05 ? 15% perf-profile.self.cycles-pp.aa_sk_perm
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://01.org/lkp