2022-05-02 23:29:27

by kernel test robot

[permalink] [raw]
Subject: [net] 5aa32dd9bd: netperf.Throughput_tps -23.5% regression



Greeting,

FYI, we noticed a -23.5% regression of netperf.Throughput_tps due to commit:


commit: 5aa32dd9bdb57e4a26143216df5e90f5e5c3dd2c ("net: add support for fast socket release")
https://github.com/ammarfaizi2/linux-block axboe/linux-block/sock-nolock.2

in testcase: netperf
on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz with 128G memory
with following parameters:

ip: ipv4
runtime: 300s
nr_threads: 50%
cluster: cs-localhost
test: SCTP_RR
cpufreq_governor: performance
ucode: 0xd000331

test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/

In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------------------------+
| testcase: change | netperf: netperf.Throughput_Mbps -89.4% regression |
| test machine | 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz with 128G memory |
| test parameters | cluster=cs-localhost |
| | cpufreq_governor=performance |
| | ip=ipv4 |
| | nr_threads=1 |
| | runtime=300s |
| | send_size=10K |
| | test=SCTP_STREAM_MANY |
| | ucode=0xd000331 |
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | netperf: netperf.Throughput_Mbps -86.8% regression |
| test machine | 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz with 128G memory |
| test parameters | cluster=cs-localhost |
| | cpufreq_governor=performance |
| | ip=ipv4 |
| | nr_threads=25% |
| | runtime=300s |
| | send_size=10K |
| | test=SCTP_STREAM_MANY |
| | ucode=0xd000331 |
+------------------+-------------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file

# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.

=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
cs-localhost/gcc-11/performance/ipv4/x86_64-rhel-8.3/50%/debian-10.4-x86_64-20200603.cgz/300s/lkp-icl-2sp4/SCTP_RR/netperf/0xd000331

commit:
4033d677de ("Merge branch 'for-5.19/io_uring-socket' into sock-nolock.2")
5aa32dd9bd ("net: add support for fast socket release")

4033d677de27560d 5aa32dd9bdb57e4a26143216df5
---------------- ---------------------------
%stddev %change %stddev
\ | \
3433333 -23.5% 2626482 ? 2% netperf.Throughput_total_tps
53645 -23.5% 41038 ? 2% netperf.Throughput_tps
26529 +220.7% 85082 ? 2% netperf.time.involuntary_context_switches
20836 +15.6% 24085 netperf.time.minor_page_faults
2728 -33.4% 1818 ? 3% netperf.time.percent_of_cpu_this_job_got
8062 -33.4% 5372 ? 3% netperf.time.system_time
261.11 -32.8% 175.44 ? 3% netperf.time.user_time
1.03e+09 -23.5% 7.879e+08 ? 2% netperf.time.voluntary_context_switches
1.03e+09 -23.5% 7.879e+08 ? 2% netperf.workload
18864 +42.3% 26841 uptime.idle
1.414e+10 +57.9% 2.232e+10 ? 2% cpuidle..time
2.069e+09 -22.2% 1.609e+09 ? 2% cpuidle..usage
36.85 +20.0 56.90 ? 2% mpstat.cpu.all.idle%
20.45 -6.6 13.85 ? 3% mpstat.cpu.all.soft%
40.36 -13.0 27.40 ? 3% mpstat.cpu.all.sys%
1.60 -0.4 1.19 ? 2% mpstat.cpu.all.usr%
37.00 +53.6% 56.83 ? 2% vmstat.cpu.id
10728248 +30.2% 13968645 vmstat.memory.cache
81.83 -33.8% 54.17 ? 3% vmstat.procs.r
13355055 -23.4% 10223620 ? 2% vmstat.system.cs
270894 -1.9% 265686 vmstat.system.in
2570 ? 20% +129.0% 5886 ? 37% numa-meminfo.node0.Active
2233 ? 13% +157.1% 5741 ? 38% numa-meminfo.node0.Active(anon)
357686 ? 23% +374.5% 1697374 ? 61% numa-meminfo.node0.Inactive
356300 ? 23% +376.3% 1697227 ? 61% numa-meminfo.node0.Inactive(anon)
55002 ?155% +2490.2% 1424655 ? 74% numa-meminfo.node0.Shmem
158899 ? 2% -60.1% 63469 ? 5% numa-meminfo.node1.Active
158875 ? 2% -60.2% 63227 ? 5% numa-meminfo.node1.Active(anon)
558.50 ? 13% +157.2% 1436 ? 38% numa-vmstat.node0.nr_active_anon
89148 ? 23% +376.9% 425139 ? 61% numa-vmstat.node0.nr_inactive_anon
13801 ?155% +2486.8% 357010 ? 74% numa-vmstat.node0.nr_shmem
558.50 ? 13% +157.2% 1436 ? 38% numa-vmstat.node0.nr_zone_active_anon
89148 ? 23% +376.9% 425139 ? 61% numa-vmstat.node0.nr_zone_inactive_anon
39849 ? 2% -60.2% 15855 ? 4% numa-vmstat.node1.nr_active_anon
39849 ? 2% -60.2% 15855 ? 4% numa-vmstat.node1.nr_zone_active_anon
161692 ? 2% -57.0% 69501 ? 3% meminfo.Active
161330 ? 3% -57.2% 69113 ? 3% meminfo.Active(anon)
10573214 +30.3% 13781428 meminfo.Cached
8859541 ? 2% +36.1% 12057551 ? 2% meminfo.Committed_AS
8504943 ? 2% +38.7% 11795316 ? 2% meminfo.Inactive
8503533 ? 2% +38.7% 11793931 ? 2% meminfo.Inactive(anon)
12718214 +25.2% 15927055 meminfo.Memused
8309280 ? 2% +38.6% 11517491 ? 2% meminfo.Shmem
12745350 +25.2% 15956173 meminfo.max_used_kB
2241 -30.4% 1560 ? 3% turbostat.Avg_MHz
69.91 -22.3 47.58 ? 3% turbostat.Busy%
3212 +2.3% 3287 turbostat.Bzy_MHz
1.321e+09 -13.0% 1.149e+09 turbostat.C1
20.90 -5.0 15.86 ? 2% turbostat.C1%
7.435e+08 -39.4% 4.508e+08 ? 4% turbostat.C1E
29.50 ? 2% +69.9% 50.11 ? 7% turbostat.CPU%c1
149.34 -102.7 46.64 ? 16% turbostat.PKG_%
4171491 -22.1% 3248601 ? 2% turbostat.POLL
0.12 -0.0 0.08 ? 5% turbostat.POLL%
493.52 -2.6% 480.46 turbostat.PkgWatt
46.41 +16.5% 54.05 turbostat.RAMWatt
40372 ? 3% -57.3% 17245 ? 3% proc-vmstat.nr_active_anon
88810 -2.6% 86525 proc-vmstat.nr_anon_pages
2954748 -2.7% 2874535 proc-vmstat.nr_dirty_background_threshold
5916722 -2.7% 5756100 proc-vmstat.nr_dirty_threshold
2637514 +30.4% 3440467 proc-vmstat.nr_file_pages
29744635 -2.7% 28941332 proc-vmstat.nr_free_pages
2119986 ? 2% +38.9% 2943612 ? 2% proc-vmstat.nr_inactive_anon
2071529 ? 2% +38.8% 2874482 ? 2% proc-vmstat.nr_shmem
33158 +5.3% 34918 proc-vmstat.nr_slab_reclaimable
40372 ? 3% -57.3% 17245 ? 3% proc-vmstat.nr_zone_active_anon
2119986 ? 2% +38.9% 2943612 ? 2% proc-vmstat.nr_zone_inactive_anon
6354131 ? 2% +33.8% 8498909 ? 2% proc-vmstat.numa_hit
6243316 ? 2% +34.4% 8388940 ? 2% proc-vmstat.numa_local
53928 ? 5% -65.7% 18516 ? 2% proc-vmstat.pgactivate
6302302 +32.4% 8344162 proc-vmstat.pgalloc_normal
75331 +6.8% 80462 proc-vmstat.pgreuse
485653 ? 6% -70.0% 145855 ? 16% sched_debug.cfs_rq:/.MIN_vruntime.avg
4889101 -58.2% 2043999 ? 11% sched_debug.cfs_rq:/.MIN_vruntime.max
1435112 ? 3% -65.2% 499003 ? 13% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.61 ? 2% -28.4% 0.43 ? 6% sched_debug.cfs_rq:/.h_nr_running.avg
1.58 ? 8% -36.8% 1.00 sched_debug.cfs_rq:/.h_nr_running.max
3.61 ? 9% -100.0% 0.00 sched_debug.cfs_rq:/.load_avg.min
485653 ? 6% -70.0% 145855 ? 16% sched_debug.cfs_rq:/.max_vruntime.avg
4889101 -58.2% 2043999 ? 11% sched_debug.cfs_rq:/.max_vruntime.max
1435112 ? 3% -65.2% 499003 ? 13% sched_debug.cfs_rq:/.max_vruntime.stddev
4779886 -60.8% 1875015 ? 11% sched_debug.cfs_rq:/.min_vruntime.avg
5195166 ? 2% -59.2% 2121407 ? 11% sched_debug.cfs_rq:/.min_vruntime.max
4718128 -65.7% 1617047 ? 11% sched_debug.cfs_rq:/.min_vruntime.min
54838 ? 17% +123.7% 122676 ? 9% sched_debug.cfs_rq:/.min_vruntime.stddev
0.59 ? 3% -27.2% 0.43 ? 5% sched_debug.cfs_rq:/.nr_running.avg
170.67 +130.0% 392.53 ? 38% sched_debug.cfs_rq:/.removed.load_avg.max
89.36 ? 4% +125.3% 201.32 ? 36% sched_debug.cfs_rq:/.removed.runnable_avg.max
89.36 ? 4% +125.3% 201.32 ? 36% sched_debug.cfs_rq:/.removed.util_avg.max
528.93 -27.5% 383.41 ? 4% sched_debug.cfs_rq:/.runnable_avg.avg
1181 ? 6% -19.5% 950.95 ? 3% sched_debug.cfs_rq:/.runnable_avg.max
142.14 ? 25% -100.0% 0.00 sched_debug.cfs_rq:/.runnable_avg.min
145.62 ? 5% +77.3% 258.24 ? 3% sched_debug.cfs_rq:/.runnable_avg.stddev
1064 ?1490% +22998.7% 245807 ? 23% sched_debug.cfs_rq:/.spread0.avg
-60742 -80.1% -12113 sched_debug.cfs_rq:/.spread0.min
54851 ? 17% +123.6% 122647 ? 9% sched_debug.cfs_rq:/.spread0.stddev
527.04 -27.3% 382.97 ? 4% sched_debug.cfs_rq:/.util_avg.avg
1177 ? 7% -19.3% 949.82 ? 3% sched_debug.cfs_rq:/.util_avg.max
142.25 ? 25% -100.0% 0.00 sched_debug.cfs_rq:/.util_avg.min
145.27 ? 5% +77.5% 257.90 ? 3% sched_debug.cfs_rq:/.util_avg.stddev
317.21 ? 2% -29.5% 223.78 ? 6% sched_debug.cfs_rq:/.util_est_enqueued.avg
946.06 ? 10% -18.0% 775.41 ? 2% sched_debug.cfs_rq:/.util_est_enqueued.max
153421 +53.6% 235687 ? 4% sched_debug.cpu.avg_idle.avg
265800 ? 7% +301.8% 1068044 ? 8% sched_debug.cpu.avg_idle.max
3500 ? 5% -9.6% 3163 ? 3% sched_debug.cpu.avg_idle.min
34824 ? 6% +545.9% 224936 ? 8% sched_debug.cpu.avg_idle.stddev
185175 -13.3% 160515 ? 6% sched_debug.cpu.clock.avg
185182 -13.3% 160520 ? 6% sched_debug.cpu.clock.max
185168 -13.3% 160509 ? 6% sched_debug.cpu.clock.min
3.72 ? 4% -20.3% 2.96 ? 5% sched_debug.cpu.clock.stddev
571.80 ? 30% +126.1% 1292 ? 9% sched_debug.cpu.clock_task.stddev
3426 -34.1% 2256 ? 4% sched_debug.cpu.curr->pid.avg
0.59 ? 2% -33.4% 0.39 ? 5% sched_debug.cpu.nr_running.avg
1.56 ? 10% -33.6% 1.03 ? 7% sched_debug.cpu.nr_running.max
16149103 -36.7% 10228798 ? 9% sched_debug.cpu.nr_switches.avg
16701556 -30.4% 11624015 ? 8% sched_debug.cpu.nr_switches.max
14782067 ? 2% -43.0% 8430068 ? 10% sched_debug.cpu.nr_switches.min
351193 ? 9% +136.9% 831814 ? 10% sched_debug.cpu.nr_switches.stddev
185169 -13.3% 160510 ? 6% sched_debug.cpu_clk
184449 -13.4% 159790 ? 6% sched_debug.ktime
10.22 ? 2% +24.4% 12.71 ? 2% perf-stat.i.MPKI
4.027e+10 -24.3% 3.047e+10 ? 2% perf-stat.i.branch-instructions
4.187e+08 -25.8% 3.105e+08 ? 2% perf-stat.i.branch-misses
1.24 ? 12% +6.7 7.92 ? 5% perf-stat.i.cache-miss-rate%
20998336 ? 11% +557.9% 1.381e+08 ? 4% perf-stat.i.cache-misses
1.989e+09 -9.2% 1.805e+09 perf-stat.i.cache-references
13504754 -23.5% 10324651 ? 2% perf-stat.i.context-switches
1.59 -5.7% 1.50 perf-stat.i.cpi
3.15e+11 -30.1% 2.201e+11 ? 2% perf-stat.i.cpu-cycles
45199 -95.1% 2197 ? 3% perf-stat.i.cpu-migrations
19508 ? 25% -90.9% 1767 ? 5% perf-stat.i.cycles-between-cache-misses
0.04 ? 14% +0.0 0.06 ? 11% perf-stat.i.dTLB-load-miss-rate%
5.827e+10 -24.7% 4.39e+10 ? 2% perf-stat.i.dTLB-loads
3.517e+10 -24.7% 2.65e+10 ? 2% perf-stat.i.dTLB-stores
2.03e+11 -24.5% 1.533e+11 ? 2% perf-stat.i.instructions
0.64 +8.2% 0.69 perf-stat.i.ipc
2.46 -30.1% 1.72 ? 2% perf-stat.i.metric.GHz
220.84 ? 4% +212.3% 689.61 ? 3% perf-stat.i.metric.K/sec
1060 -24.3% 802.22 ? 2% perf-stat.i.metric.M/sec
85.79 +6.8 92.57 perf-stat.i.node-load-miss-rate%
6746106 ? 9% +565.5% 44893575 ? 4% perf-stat.i.node-load-misses
1469235 ? 9% +95.2% 2868455 ? 6% perf-stat.i.node-loads
63.12 ? 3% +6.4 69.52 perf-stat.i.node-store-miss-rate%
3224894 ? 8% +535.6% 20495885 ? 4% perf-stat.i.node-store-misses
2375887 ? 7% +276.1% 8935513 ? 4% perf-stat.i.node-stores
9.80 +20.2% 11.78 perf-stat.overall.MPKI
1.04 -0.0 1.02 perf-stat.overall.branch-miss-rate%
1.06 ? 11% +6.6 7.65 ? 5% perf-stat.overall.cache-miss-rate%
1.55 -7.5% 1.44 perf-stat.overall.cpi
15203 ? 12% -89.5% 1596 ? 5% perf-stat.overall.cycles-between-cache-misses
0.01 ? 3% +0.0 0.02 ? 5% perf-stat.overall.dTLB-load-miss-rate%
0.64 +8.1% 0.70 perf-stat.overall.ipc
82.03 ? 2% +12.0 93.98 perf-stat.overall.node-load-miss-rate%
57.54 ? 2% +12.1 69.62 perf-stat.overall.node-store-miss-rate%
60057 -1.1% 59408 perf-stat.overall.path-length
4.014e+10 -24.3% 3.038e+10 ? 2% perf-stat.ps.branch-instructions
4.173e+08 -25.8% 3.095e+08 ? 2% perf-stat.ps.branch-misses
20930523 ? 11% +557.8% 1.377e+08 ? 4% perf-stat.ps.cache-misses
1.982e+09 -9.2% 1.799e+09 perf-stat.ps.cache-references
13459625 -23.5% 10290156 ? 2% perf-stat.ps.context-switches
3.14e+11 -30.1% 2.193e+11 ? 2% perf-stat.ps.cpu-cycles
45141 -95.1% 2190 ? 3% perf-stat.ps.cpu-migrations
5.808e+10 -24.7% 4.376e+10 ? 2% perf-stat.ps.dTLB-loads
3.505e+10 -24.7% 2.641e+10 ? 2% perf-stat.ps.dTLB-stores
2.023e+11 -24.5% 1.528e+11 ? 2% perf-stat.ps.instructions
6723213 ? 9% +565.4% 44738975 ? 4% perf-stat.ps.node-load-misses
1463122 ? 9% +95.4% 2859072 ? 6% perf-stat.ps.node-loads
3214426 ? 8% +535.4% 20425091 ? 4% perf-stat.ps.node-store-misses
2369833 ? 7% +275.9% 8907536 ? 4% perf-stat.ps.node-stores
6.186e+13 -24.3% 4.681e+13 ? 2% perf-stat.total.instructions
51.77 -4.7 47.02 perf-profile.calltrace.cycles-pp.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
52.57 -4.6 47.94 perf-profile.calltrace.cycles-pp.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg
49.67 -4.4 45.27 perf-profile.calltrace.cycles-pp.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64
49.15 -4.3 44.80 perf-profile.calltrace.cycles-pp.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg
48.34 -4.3 44.09 perf-profile.calltrace.cycles-pp.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
43.17 -3.7 39.49 perf-profile.calltrace.cycles-pp.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
42.04 -3.6 38.42 perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc
34.99 -3.5 31.47 perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND
30.32 -3.4 26.90 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
29.73 -3.3 26.39 perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
28.05 -3.1 24.92 perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit
28.17 -3.1 25.04 perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush
27.90 -3.1 24.78 perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
26.60 -3.0 23.64 perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
26.90 -2.9 23.96 perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
26.68 -2.9 23.74 perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
25.86 -2.9 22.96 perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
26.87 -2.8 24.08 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendmsg.process_requests.spawn_child.accept_connection
26.76 -2.8 24.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.process_requests.spawn_child
26.63 -2.7 23.88 perf-profile.calltrace.cycles-pp.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.process_requests
25.32 -2.7 22.60 perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
25.22 -2.7 22.54 perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
25.02 -2.6 22.44 perf-profile.calltrace.cycles-pp.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
26.78 -2.1 24.66 perf-profile.calltrace.cycles-pp.sendmsg.main.__libc_start_main
26.54 -2.1 24.43 ? 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendmsg.main.__libc_start_main
21.68 -2.1 19.59 ? 2% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
26.44 -2.1 24.35 ? 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.main.__libc_start_main
23.76 -2.1 21.68 perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg
15.78 -2.0 13.76 perf-profile.calltrace.cycles-pp.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.56 ? 4% -2.0 3.57 ? 3% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg
5.47 ? 4% -2.0 3.48 ? 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg
16.64 -2.0 14.66 perf-profile.calltrace.cycles-pp.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
26.31 -2.0 24.33 perf-profile.calltrace.cycles-pp.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.main
16.95 -2.0 14.97 perf-profile.calltrace.cycles-pp.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
15.17 -1.9 13.25 perf-profile.calltrace.cycles-pp.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64
15.10 -1.9 13.20 perf-profile.calltrace.cycles-pp.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg
18.73 -1.9 16.85 perf-profile.calltrace.cycles-pp.recvmsg
17.27 -1.8 15.43 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
5.87 ? 4% -1.8 4.06 ? 2% perf-profile.calltrace.cycles-pp.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
17.50 -1.8 15.70 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recvmsg
8.28 ? 2% -1.7 6.54 perf-profile.calltrace.cycles-pp.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
22.16 -1.7 20.48 perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
18.42 -1.4 16.99 ? 2% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
17.64 -1.3 16.34 ? 2% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu
8.26 ? 2% -1.0 7.29 perf-profile.calltrace.cycles-pp.consume_skb.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm
8.61 ? 2% -1.0 7.65 perf-profile.calltrace.cycles-pp.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
2.06 ? 2% -0.9 1.18 ? 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
2.01 ? 2% -0.9 1.13 ? 5% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
6.98 ? 2% -0.8 6.19 perf-profile.calltrace.cycles-pp.sctp_wfree.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_outq_sack
7.08 ? 2% -0.8 6.30 perf-profile.calltrace.cycles-pp.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter
2.18 -0.7 1.46 ? 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
5.32 ? 2% -0.7 4.65 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.__sctp_write_space.sctp_wfree
5.23 ? 2% -0.6 4.59 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.__sctp_write_space
4.06 -0.6 3.43 ? 2% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
4.19 -0.6 3.56 ? 2% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
5.76 ? 2% -0.6 5.14 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.__sctp_write_space.sctp_wfree.skb_release_head_state
13.51 -0.6 12.92 perf-profile.calltrace.cycles-pp.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv
5.91 ? 2% -0.6 5.33 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.__sctp_write_space.sctp_wfree.skb_release_head_state.consume_skb
2.63 ? 2% -0.6 2.08 perf-profile.calltrace.cycles-pp.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
2.17 ? 2% -0.5 1.64 perf-profile.calltrace.cycles-pp._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
6.23 ? 2% -0.5 5.75 perf-profile.calltrace.cycles-pp.__sctp_write_space.sctp_wfree.skb_release_head_state.consume_skb.sctp_chunk_put
1.82 -0.4 1.38 ? 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.sctp_skb_recv_datagram
2.29 ? 2% -0.4 1.87 ? 2% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv
4.10 ? 2% -0.4 3.69 perf-profile.calltrace.cycles-pp.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
1.57 -0.4 1.20 ? 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
1.86 ? 3% -0.3 1.52 ? 4% perf-profile.calltrace.cycles-pp.sctp_ulpq_tail_data.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv
3.60 ? 2% -0.3 3.26 ? 2% perf-profile.calltrace.cycles-pp.sctp_packet_append_chunk.sctp_packet_transmit_chunk.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter
3.63 ? 2% -0.3 3.30 ? 2% perf-profile.calltrace.cycles-pp.sctp_packet_transmit_chunk.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
1.33 -0.3 1.05 ? 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule_idle.do_idle
0.71 -0.3 0.43 ? 44% perf-profile.calltrace.cycles-pp.kmem_cache_free.sctp_ulpevent_free.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
2.48 -0.2 2.25 perf-profile.calltrace.cycles-pp.sctp_ulpevent_free.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
0.63 -0.2 0.43 ? 44% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.schedule_timeout.sctp_skb_recv_datagram
2.18 ? 2% -0.2 1.99 perf-profile.calltrace.cycles-pp.sctp_make_sack.sctp_packet_append_chunk.sctp_packet_transmit_chunk.sctp_outq_flush_data.sctp_outq_flush
1.32 -0.2 1.16 perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
1.54 ? 3% -0.1 1.40 ? 2% perf-profile.calltrace.cycles-pp.__sctp_rcv_lookup.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
1.22 ? 2% -0.1 1.08 ? 4% perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush
1.50 ? 3% -0.1 1.36 ? 2% perf-profile.calltrace.cycles-pp.sctp_addrs_lookup_transport.__sctp_rcv_lookup.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
0.78 -0.1 0.66 ? 3% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
1.27 ? 3% -0.1 1.15 ? 3% perf-profile.calltrace.cycles-pp.__rhashtable_lookup.sctp_addrs_lookup_transport.__sctp_rcv_lookup.sctp_rcv.ip_protocol_deliver_rcu
0.67 -0.1 0.58 ? 2% perf-profile.calltrace.cycles-pp.tick_nohz_idle_exit.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
0.88 -0.1 0.81 ? 3% perf-profile.calltrace.cycles-pp.kfree_skb_reason.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
0.73 ? 2% -0.0 0.69 ? 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_bh.lock_sock_nested.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg
0.81 +0.0 0.85 ? 2% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg
0.63 +0.0 0.67 perf-profile.calltrace.cycles-pp.prepare_to_wait_exclusive.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
0.60 +0.0 0.65 ? 2% perf-profile.calltrace.cycles-pp.sctp_inet_skb_msgname.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
0.56 ? 4% +0.1 0.61 ? 2% perf-profile.calltrace.cycles-pp.dst_release.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_ulpevent_free
0.69 +0.1 0.74 ? 2% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg
0.66 +0.1 0.72 ? 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
1.57 +0.1 1.64 ? 2% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
1.54 +0.1 1.62 ? 2% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
0.61 ? 4% +0.1 0.68 ? 3% perf-profile.calltrace.cycles-pp.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_ulpevent_free.sctp_recvmsg
0.90 ? 2% +0.1 1.00 ? 2% perf-profile.calltrace.cycles-pp.consume_skb.sctp_chunk_put.sctp_ulpevent_free.sctp_recvmsg.inet_recvmsg
1.14 ? 2% +0.2 1.29 perf-profile.calltrace.cycles-pp.__alloc_skb.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
1.72 ? 2% +0.2 1.95 perf-profile.calltrace.cycles-pp.sctp_packet_pack.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
1.83 ? 2% +0.4 2.20 ? 3% perf-profile.calltrace.cycles-pp.sctp_outq_select_transport.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
0.96 ? 20% +0.4 1.34 ? 2% perf-profile.calltrace.cycles-pp.sctp_chunk_put.sctp_packet_pack.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
0.08 ?223% +0.5 0.56 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_exclusive.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg
0.00 +1.0 0.96 ? 12% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_from_idle.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
2.12 +1.0 3.12 ? 3% perf-profile.calltrace.cycles-pp.sctp_check_transmitted.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
0.00 +1.2 1.18 ? 32% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
0.00 +1.4 1.35 ? 31% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.08 ?223% +1.4 1.44 ? 2% perf-profile.calltrace.cycles-pp.sctp_packet_config.sctp_outq_select_transport.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter
0.00 +1.4 1.44 ? 12% perf-profile.calltrace.cycles-pp.flush_smp_call_function_from_idle.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
15.63 +4.5 20.17 ? 2% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
15.89 +4.6 20.48 ? 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
16.90 +6.1 22.97 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
16.97 +6.2 23.18 ? 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
19.28 +6.3 25.55 ? 3% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
24.93 +7.0 31.98 ? 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
25.02 +7.1 32.07 ? 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.secondary_startup_64_no_verify
25.26 +7.1 32.37 ? 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
71.34 -6.8 64.49 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
70.91 -6.8 64.09 perf-profile.children.cycles-pp.do_syscall_64
54.38 -5.1 49.29 perf-profile.children.cycles-pp.sendmsg
52.12 -5.0 47.12 perf-profile.children.cycles-pp.____sys_sendmsg
53.30 -5.0 48.31 perf-profile.children.cycles-pp.__sys_sendmsg
52.93 -4.9 48.05 perf-profile.children.cycles-pp.___sys_sendmsg
50.00 -4.6 45.36 perf-profile.children.cycles-pp.sock_sendmsg
49.80 -4.6 45.19 perf-profile.children.cycles-pp.sctp_sendmsg
48.69 -4.5 44.21 perf-profile.children.cycles-pp.sctp_sendmsg_to_asoc
43.46 -3.8 39.65 perf-profile.children.cycles-pp.sctp_primitive_SEND
43.31 -3.8 39.54 perf-profile.children.cycles-pp.sctp_do_sm
42.46 -3.8 38.70 perf-profile.children.cycles-pp.sctp_outq_flush
43.13 -3.7 39.38 perf-profile.children.cycles-pp.sctp_cmd_interpreter
35.24 -3.6 31.62 perf-profile.children.cycles-pp.sctp_packet_transmit
30.53 -3.5 27.02 perf-profile.children.cycles-pp.__ip_queue_xmit
29.94 -3.4 26.51 perf-profile.children.cycles-pp.ip_finish_output2
28.50 -3.3 25.21 perf-profile.children.cycles-pp.__local_bh_enable_ip
28.25 -3.2 25.02 perf-profile.children.cycles-pp.do_softirq
28.19 -3.1 25.10 perf-profile.children.cycles-pp.__softirqentry_text_start
26.78 -3.0 23.74 perf-profile.children.cycles-pp.process_backlog
26.86 -3.0 23.85 perf-profile.children.cycles-pp.__napi_poll
27.09 -3.0 24.10 perf-profile.children.cycles-pp.net_rx_action
26.04 -3.0 23.07 perf-profile.children.cycles-pp.__netif_receive_skb_one_core
27.54 -2.9 24.68 perf-profile.children.cycles-pp.accept_connection
27.54 -2.9 24.68 perf-profile.children.cycles-pp.spawn_child
27.54 -2.9 24.68 perf-profile.children.cycles-pp.process_requests
25.50 -2.8 22.70 perf-profile.children.cycles-pp.ip_local_deliver_finish
25.39 -2.8 22.64 perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
25.23 -2.7 22.56 perf-profile.children.cycles-pp.sctp_rcv
9.66 ? 2% -2.6 7.02 ? 2% perf-profile.children.cycles-pp.__schedule
21.85 -2.2 19.69 perf-profile.children.cycles-pp.sctp_assoc_bh_rcv
3.77 ? 6% -2.1 1.71 ? 3% perf-profile.children.cycles-pp.pick_next_task_fair
15.79 -2.0 13.77 perf-profile.children.cycles-pp.____sys_recvmsg
16.66 -2.0 14.68 perf-profile.children.cycles-pp.___sys_recvmsg
5.56 ? 4% -2.0 3.58 ? 3% perf-profile.children.cycles-pp.schedule
16.97 -2.0 14.99 perf-profile.children.cycles-pp.__sys_recvmsg
15.46 -1.9 13.53 perf-profile.children.cycles-pp.sctp_recvmsg
15.18 -1.9 13.26 perf-profile.children.cycles-pp.inet_recvmsg
18.87 -1.9 16.98 perf-profile.children.cycles-pp.recvmsg
5.89 ? 4% -1.8 4.08 ? 2% perf-profile.children.cycles-pp.schedule_timeout
8.32 ? 2% -1.7 6.59 perf-profile.children.cycles-pp.sctp_skb_recv_datagram
1.50 ? 16% -1.3 0.17 ? 10% perf-profile.children.cycles-pp.newidle_balance
9.61 -1.1 8.54 perf-profile.children.cycles-pp.consume_skb
7.16 -0.9 6.22 perf-profile.children.cycles-pp.sctp_wfree
8.16 -0.9 7.25 perf-profile.children.cycles-pp.skb_release_head_state
11.66 -0.9 10.76 perf-profile.children.cycles-pp.sctp_chunk_put
5.47 -0.8 4.68 perf-profile.children.cycles-pp.autoremove_wake_function
13.75 -0.8 13.00 perf-profile.children.cycles-pp.sctp_outq_sack
5.91 -0.7 5.17 perf-profile.children.cycles-pp.__wake_up_common
5.40 -0.7 4.65 perf-profile.children.cycles-pp.try_to_wake_up
6.07 -0.7 5.36 perf-profile.children.cycles-pp.__wake_up_common_lock
1.56 -0.7 0.86 perf-profile.children.cycles-pp.kmem_cache_alloc
3.38 -0.7 2.69 perf-profile.children.cycles-pp._sctp_make_chunk
1.61 -0.6 0.98 perf-profile.children.cycles-pp.sctp_chunkify
4.24 -0.6 3.62 perf-profile.children.cycles-pp.schedule_idle
6.40 -0.6 5.78 perf-profile.children.cycles-pp.__sctp_write_space
2.71 -0.6 2.09 perf-profile.children.cycles-pp.sctp_make_datafrag_empty
0.73 ? 18% -0.6 0.11 ? 17% perf-profile.children.cycles-pp.load_balance
0.61 ? 17% -0.5 0.09 ? 12% perf-profile.children.cycles-pp.find_busiest_group
4.22 -0.5 3.72 perf-profile.children.cycles-pp.sctp_datamsg_from_user
2.94 -0.5 2.45 ? 2% perf-profile.children.cycles-pp.__slab_free
0.56 ? 16% -0.5 0.08 ? 11% perf-profile.children.cycles-pp.update_sd_lb_stats
1.83 -0.4 1.40 ? 2% perf-profile.children.cycles-pp.dequeue_task_fair
1.17 ? 3% -0.4 0.74 ? 5% perf-profile.children.cycles-pp.sctp_ulpevent_make_rcvmsg
1.61 -0.4 1.20 perf-profile.children.cycles-pp.enqueue_entity
3.72 -0.4 3.32 ? 2% perf-profile.children.cycles-pp.sctp_packet_transmit_chunk
3.70 -0.4 3.30 ? 2% perf-profile.children.cycles-pp.sctp_packet_append_chunk
0.47 ? 18% -0.4 0.07 ? 29% perf-profile.children.cycles-pp.update_blocked_averages
0.77 -0.4 0.37 ? 9% perf-profile.children.cycles-pp.sock_kmalloc
2.08 -0.4 1.68 perf-profile.children.cycles-pp.enqueue_task_fair
0.46 ? 16% -0.4 0.07 ? 8% perf-profile.children.cycles-pp.update_sg_lb_stats
1.93 -0.4 1.54 ? 4% perf-profile.children.cycles-pp.sctp_ulpq_tail_data
2.12 -0.4 1.74 perf-profile.children.cycles-pp.ttwu_do_activate
1.62 -0.4 1.24 ? 2% perf-profile.children.cycles-pp.dequeue_entity
2.32 -0.3 1.98 ? 3% perf-profile.children.cycles-pp.skb_release_data
0.51 ? 9% -0.3 0.17 ? 8% perf-profile.children.cycles-pp.__skb_clone
1.06 ? 2% -0.3 0.73 ? 3% perf-profile.children.cycles-pp.sctp_sf_eat_data_6_2
3.00 -0.3 2.67 ? 3% perf-profile.children.cycles-pp.kfree
0.41 ? 11% -0.3 0.09 ? 7% perf-profile.children.cycles-pp.__copy_skb_header
3.11 -0.3 2.82 perf-profile.children.cycles-pp.kmem_cache_free
1.38 -0.3 1.10 ? 3% perf-profile.children.cycles-pp.set_next_entity
1.09 ? 2% -0.3 0.82 ? 2% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.93 ? 2% -0.3 0.67 ? 3% perf-profile.children.cycles-pp.sctp_sf_do_no_pending_tsn
0.91 ? 3% -0.3 0.66 ? 3% perf-profile.children.cycles-pp.sctp_ulpevent_make_sender_dry_event
0.92 -0.3 0.67 ? 3% perf-profile.children.cycles-pp.kmalloc_reserve
0.80 -0.3 0.55 ? 2% perf-profile.children.cycles-pp.update_curr
0.54 -0.2 0.30 ? 11% perf-profile.children.cycles-pp.resched_curr
0.84 -0.2 0.60 ? 3% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
1.11 -0.2 0.88 ? 2% perf-profile.children.cycles-pp.__list_del_entry_valid
2.24 -0.2 2.01 perf-profile.children.cycles-pp.sctp_make_sack
2.61 -0.2 2.38 perf-profile.children.cycles-pp.sctp_ulpevent_free
0.88 -0.2 0.66 ? 3% perf-profile.children.cycles-pp.sctp_addto_chunk
0.45 -0.2 0.25 ? 4% perf-profile.children.cycles-pp.skb_put
1.16 -0.2 0.96 ? 3% perf-profile.children.cycles-pp.sctp_make_control
0.85 -0.2 0.65 ? 3% perf-profile.children.cycles-pp.sctp_eat_data
0.91 ? 2% -0.2 0.72 ? 3% perf-profile.children.cycles-pp.do_ulpq_tail_event
0.55 ? 3% -0.2 0.36 ? 18% perf-profile.children.cycles-pp.enqueue_timer
0.22 ? 4% -0.2 0.04 ? 71% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
1.58 ? 2% -0.2 1.40 ? 2% perf-profile.children.cycles-pp.__sctp_rcv_lookup
1.55 ? 2% -0.2 1.38 ? 3% perf-profile.children.cycles-pp.sctp_addrs_lookup_transport
1.27 ? 3% -0.2 1.10 ? 4% perf-profile.children.cycles-pp.__dev_queue_xmit
0.60 -0.2 0.44 ? 8% perf-profile.children.cycles-pp.check_preempt_curr
0.64 -0.2 0.48 ? 7% perf-profile.children.cycles-pp.ttwu_do_wakeup
3.48 -0.2 3.31 perf-profile.children.cycles-pp.__alloc_skb
0.57 -0.2 0.41 ? 5% perf-profile.children.cycles-pp.finish_task_switch
0.69 -0.2 0.53 ? 3% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.46 -0.1 0.31 ? 6% perf-profile.children.cycles-pp.pick_next_entity
0.39 -0.1 0.24 ? 2% perf-profile.children.cycles-pp.__cond_resched
0.70 -0.1 0.57 ? 2% perf-profile.children.cycles-pp.__might_resched
1.30 ? 3% -0.1 1.17 ? 3% perf-profile.children.cycles-pp.__rhashtable_lookup
0.79 -0.1 0.66 ? 4% perf-profile.children.cycles-pp.prepare_task_switch
0.52 -0.1 0.38 ? 5% perf-profile.children.cycles-pp.___perf_sw_event
0.55 -0.1 0.42 ? 3% perf-profile.children.cycles-pp.__next_timer_interrupt
0.80 -0.1 0.67 ? 3% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.30 ? 11% -0.1 0.17 ? 10% perf-profile.children.cycles-pp.sctp_outq_tail
0.35 ? 2% -0.1 0.24 ? 3% perf-profile.children.cycles-pp._find_next_bit
1.46 -0.1 1.35 perf-profile.children.cycles-pp.sctp_ulpq_tail_event
1.12 -0.1 1.01 ? 2% perf-profile.children.cycles-pp.kfree_skb_reason
0.83 -0.1 0.72 perf-profile.children.cycles-pp.skb_set_owner_w
0.27 ? 8% -0.1 0.17 ? 10% perf-profile.children.cycles-pp.validate_xmit_skb
0.26 ? 2% -0.1 0.15 ? 4% perf-profile.children.cycles-pp.ip_rcv
0.75 ? 2% -0.1 0.64 ? 3% perf-profile.children.cycles-pp.sctp_queue_purge_ulpevents
0.54 -0.1 0.44 ? 2% perf-profile.children.cycles-pp.__list_add_valid
0.26 -0.1 0.16 ? 3% perf-profile.children.cycles-pp.security_socket_recvmsg
0.41 ? 2% -0.1 0.31 ? 3% perf-profile.children.cycles-pp.sockfd_lookup_light
0.28 -0.1 0.18 ? 2% perf-profile.children.cycles-pp.sock_recvmsg
1.22 -0.1 1.13 ? 2% perf-profile.children.cycles-pp.update_load_avg
0.53 -0.1 0.44 ? 4% perf-profile.children.cycles-pp.sctp_association_put
0.36 -0.1 0.27 ? 2% perf-profile.children.cycles-pp.aa_sk_perm
0.33 ? 3% -0.1 0.24 ? 4% perf-profile.children.cycles-pp.release_sock
0.18 ? 6% -0.1 0.10 ? 5% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.77 -0.1 0.68 perf-profile.children.cycles-pp.import_iovec
0.68 -0.1 0.59 ? 2% perf-profile.children.cycles-pp.tick_nohz_idle_exit
1.14 -0.1 1.05 perf-profile.children.cycles-pp._copy_from_user
0.27 ? 11% -0.1 0.18 ? 12% perf-profile.children.cycles-pp.sctp_v4_xmit
0.74 -0.1 0.66 perf-profile.children.cycles-pp.__import_iovec
0.17 ? 5% -0.1 0.09 ? 5% perf-profile.children.cycles-pp.sctp_cmp_addr_exact
0.36 -0.1 0.28 ? 2% perf-profile.children.cycles-pp.__update_load_avg_se
0.35 -0.1 0.28 ? 3% perf-profile.children.cycles-pp.security_socket_sendmsg
0.50 -0.1 0.43 ? 3% perf-profile.children.cycles-pp.nr_iowait_cpu
0.32 ? 2% -0.1 0.25 ? 2% perf-profile.children.cycles-pp.__fget_light
0.30 ? 2% -0.1 0.23 perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
0.56 -0.1 0.49 ? 3% perf-profile.children.cycles-pp.sctp_sf_eat_sack_6_2
0.70 -0.1 0.63 ? 4% perf-profile.children.cycles-pp.irqtime_account_irq
0.63 -0.1 0.57 ? 2% perf-profile.children.cycles-pp.iovec_from_user
0.11 ? 9% -0.1 0.06 ? 9% perf-profile.children.cycles-pp.sctp_validate_data
0.44 -0.1 0.39 ? 5% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.60 -0.1 0.54 perf-profile.children.cycles-pp.memset_erms
0.39 ? 4% -0.1 0.34 ? 5% perf-profile.children.cycles-pp.__switch_to_asm
0.74 ? 2% -0.0 0.69 perf-profile.children.cycles-pp.memcg_slab_free_hook
0.28 ? 3% -0.0 0.23 ? 3% perf-profile.children.cycles-pp.recv_sctp_rr
0.39 ? 2% -0.0 0.34 ? 3% perf-profile.children.cycles-pp.__might_fault
0.15 ? 2% -0.0 0.11 ? 4% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.27 ? 2% -0.0 0.22 ? 2% perf-profile.children.cycles-pp.send_sctp_rr
0.10 -0.0 0.06 ? 8% perf-profile.children.cycles-pp.sk_filter_trim_cap
0.12 ? 3% -0.0 0.08 ? 4% perf-profile.children.cycles-pp.ip_output
0.30 ? 5% -0.0 0.26 ? 2% perf-profile.children.cycles-pp.sctp_chunk_hold
0.19 ? 10% -0.0 0.15 ? 4% perf-profile.children.cycles-pp.sctp_datamsg_put
0.16 ? 5% -0.0 0.13 ? 5% perf-profile.children.cycles-pp.update_cfs_group
0.15 ? 3% -0.0 0.12 ? 4% perf-profile.children.cycles-pp.perf_tp_event
0.17 ? 2% -0.0 0.14 ? 4% perf-profile.children.cycles-pp.__virt_addr_valid
0.08 ? 6% -0.0 0.04 ? 47% perf-profile.children.cycles-pp.sctp_sm_lookup_event
0.24 ? 2% -0.0 0.22 ? 5% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.21 ? 2% -0.0 0.18 ? 2% perf-profile.children.cycles-pp.skb_dequeue
0.17 ? 6% -0.0 0.14 ? 7% perf-profile.children.cycles-pp.sctp_ulpevent_init
0.07 ? 5% -0.0 0.04 ? 44% perf-profile.children.cycles-pp.put_prev_task_fair
0.10 ? 4% -0.0 0.08 ? 4% perf-profile.children.cycles-pp.update_min_vruntime
0.24 -0.0 0.22 ? 3% perf-profile.children.cycles-pp.__sctp_packet_append_chunk
0.10 ? 5% -0.0 0.07 ? 8% perf-profile.children.cycles-pp.pick_next_task_idle
0.09 ? 5% -0.0 0.07 ? 8% perf-profile.children.cycles-pp.kmalloc_slab
0.23 ? 2% -0.0 0.21 ? 2% perf-profile.children.cycles-pp.sctp_inq_pop
0.09 ? 9% -0.0 0.07 ? 8% perf-profile.children.cycles-pp.__ip_finish_output
0.09 ? 4% -0.0 0.07 ? 11% perf-profile.children.cycles-pp.should_failslab
0.08 ? 5% -0.0 0.06 ? 7% perf-profile.children.cycles-pp.check_stack_object
0.08 ? 6% -0.0 0.05 ? 8% perf-profile.children.cycles-pp.__update_idle_core
0.13 ? 2% -0.0 0.11 ? 4% perf-profile.children.cycles-pp.__put_user_nocheck_4
0.24 ? 2% -0.0 0.22 ? 4% perf-profile.children.cycles-pp.save_fpregs_to_fpstate
0.11 ? 3% -0.0 0.09 ? 9% perf-profile.children.cycles-pp.update_irq_load_avg
0.10 ? 7% -0.0 0.08 ? 5% perf-profile.children.cycles-pp.sctp_bind_addr_match
0.18 ? 2% -0.0 0.16 ? 6% perf-profile.children.cycles-pp.memcpy_erms
0.17 ? 2% -0.0 0.16 ? 4% perf-profile.children.cycles-pp.copyin
0.10 ? 3% -0.0 0.08 ? 5% perf-profile.children.cycles-pp.sctp_ulpq_order
0.10 ? 9% -0.0 0.08 ? 5% perf-profile.children.cycles-pp.ip_rcv_core
0.08 ? 6% -0.0 0.06 perf-profile.children.cycles-pp.perf_trace_buf_update
0.11 -0.0 0.10 ? 8% perf-profile.children.cycles-pp.__build_skb_around
0.12 ? 4% -0.0 0.10 ? 4% perf-profile.children.cycles-pp.sctp_sock_rfree
0.32 +0.0 0.35 ? 3% perf-profile.children.cycles-pp.__entry_text_start
0.11 ? 3% +0.0 0.13 ? 3% perf-profile.children.cycles-pp.call_cpuidle
0.14 ? 3% +0.0 0.17 ? 4% perf-profile.children.cycles-pp.sock_wfree
0.11 ? 4% +0.0 0.14 ? 7% perf-profile.children.cycles-pp.rcu_dynticks_inc
0.82 +0.0 0.86 ? 2% perf-profile.children.cycles-pp._copy_to_iter
0.30 ? 3% +0.0 0.34 ? 3% perf-profile.children.cycles-pp._copy_from_iter
0.08 ? 5% +0.0 0.12 ? 12% perf-profile.children.cycles-pp.rebalance_domains
0.64 +0.0 0.68 perf-profile.children.cycles-pp.prepare_to_wait_exclusive
1.55 +0.0 1.59 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.60 +0.0 0.65 ? 2% perf-profile.children.cycles-pp.sctp_inet_skb_msgname
0.69 +0.1 0.74 ? 2% perf-profile.children.cycles-pp.copyout
0.44 ? 2% +0.1 0.50 ? 4% perf-profile.children.cycles-pp.select_task_rq_fair
0.77 +0.1 0.83 ? 3% perf-profile.children.cycles-pp.select_task_rq
1.59 +0.1 1.66 perf-profile.children.cycles-pp.__check_object_size
0.00 +0.1 0.06 ? 11% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.00 +0.1 0.06 ? 7% perf-profile.children.cycles-pp.sctp_v4_ecn_capable
0.00 +0.1 0.07 ? 33% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.14 ? 7% +0.1 0.21 ? 9% perf-profile.children.cycles-pp.clockevents_program_event
0.14 ? 14% +0.1 0.21 ? 13% perf-profile.children.cycles-pp.rcu_eqs_exit
1.58 +0.1 1.65 ? 2% perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.00 +0.1 0.07 ? 12% perf-profile.children.cycles-pp.lapic_next_deadline
1.55 +0.1 1.62 ? 2% perf-profile.children.cycles-pp.__skb_datagram_iter
0.52 +0.1 0.59 ? 4% perf-profile.children.cycles-pp.sctp_data_ready
0.38 +0.1 0.46 ? 2% perf-profile.children.cycles-pp.read_tsc
0.23 ? 2% +0.1 0.31 ? 4% perf-profile.children.cycles-pp.sctp_association_hold
0.00 +0.1 0.08 ? 35% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.1 0.08 ? 14% perf-profile.children.cycles-pp.llist_add_batch
0.00 +0.1 0.09 ? 40% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.00 +0.1 0.09 ? 11% perf-profile.children.cycles-pp.__smp_call_single_queue
0.07 ? 6% +0.1 0.17 ? 32% perf-profile.children.cycles-pp.scheduler_tick
0.00 +0.1 0.10 ? 8% perf-profile.children.cycles-pp.native_irq_return_iret
0.36 +0.1 0.46 ? 2% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.31 ? 5% +0.1 0.41 ? 7% perf-profile.children.cycles-pp.rcu_idle_exit
0.26 ? 3% +0.1 0.36 ? 4% perf-profile.children.cycles-pp.available_idle_cpu
0.27 ? 3% +0.1 0.37 ? 3% perf-profile.children.cycles-pp.wake_affine
0.69 +0.1 0.81 perf-profile.children.cycles-pp.sendmsg_copy_msghdr
0.61 +0.1 0.74 ? 2% perf-profile.children.cycles-pp.sctp_transport_hold
0.00 +0.1 0.14 ? 32% perf-profile.children.cycles-pp.tick_irq_enter
0.00 +0.1 0.14 ? 2% perf-profile.children.cycles-pp.sctp_auth_send_cid
0.00 +0.1 0.15 ? 32% perf-profile.children.cycles-pp.irq_enter_rcu
0.30 ? 7% +0.2 0.45 ? 3% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.57 +0.2 0.73 ? 2% perf-profile.children.cycles-pp.__copy_msghdr_from_user
0.12 ? 6% +0.2 0.28 ? 20% perf-profile.children.cycles-pp.__irq_exit_rcu
0.13 ? 3% +0.2 0.30 ? 3% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.08 +0.2 0.25 ? 4% perf-profile.children.cycles-pp.switch_fpu_return
0.48 +0.2 0.66 perf-profile.children.cycles-pp.sctp_chunk_abandoned
0.70 +0.2 0.88 ? 5% perf-profile.children.cycles-pp.ktime_get
0.10 ? 4% +0.2 0.30 ? 39% perf-profile.children.cycles-pp.update_process_times
1.80 +0.2 1.99 perf-profile.children.cycles-pp.sctp_packet_pack
0.32 +0.2 0.52 ? 6% perf-profile.children.cycles-pp.sctp_transport_put
0.00 +0.2 0.21 ? 11% perf-profile.children.cycles-pp.llist_reverse_order
0.11 ? 6% +0.2 0.32 ? 43% perf-profile.children.cycles-pp.tick_sched_handle
0.14 ? 3% +0.2 0.38 ? 37% perf-profile.children.cycles-pp.tick_sched_timer
0.22 ? 10% +0.2 0.46 ? 2% perf-profile.children.cycles-pp.ipv4_dst_check
0.15 ? 3% +0.3 0.42 ? 6% perf-profile.children.cycles-pp.sctp_tsnmap_check
1.10 +0.3 1.39 ? 4% perf-profile.children.cycles-pp.del_timer
0.85 +0.3 1.16 ? 4% perf-profile.children.cycles-pp._raw_spin_lock
0.00 +0.3 0.32 ? 13% perf-profile.children.cycles-pp.flush_smp_call_function_queue
1.89 +0.3 2.22 ? 2% perf-profile.children.cycles-pp.sctp_outq_select_transport
0.00 +0.3 0.34 ? 13% perf-profile.children.cycles-pp.send_call_function_single_ipi
1.13 +0.4 1.51 ? 2% perf-profile.children.cycles-pp.__ksize
0.17 ? 2% +0.4 0.56 ? 40% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.30 ? 4% +0.5 1.76 perf-profile.children.cycles-pp.lock_timer_base
0.98 ? 2% +0.5 1.45 ? 2% perf-profile.children.cycles-pp.sctp_packet_config
0.07 ? 10% +0.5 0.54 ? 12% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.36 ? 4% +0.5 0.89 ? 29% perf-profile.children.cycles-pp.hrtimer_interrupt
0.36 ? 4% +0.6 0.91 ? 31% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
2.13 ? 2% +0.7 2.87 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.54 ? 3% +0.9 1.48 ? 26% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
2.21 +1.0 3.17 ? 3% perf-profile.children.cycles-pp.sctp_check_transmitted
0.00 +1.0 0.98 ? 12% perf-profile.children.cycles-pp.sched_ttwu_pending
0.72 ? 3% +1.1 1.87 ? 22% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +1.5 1.46 ? 12% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle
15.91 +4.6 20.52 ? 2% perf-profile.children.cycles-pp.mwait_idle_with_hints
16.04 +4.6 20.67 ? 2% perf-profile.children.cycles-pp.intel_idle
17.13 +6.3 23.38 ? 4% perf-profile.children.cycles-pp.cpuidle_enter
17.08 +6.3 23.34 ? 4% perf-profile.children.cycles-pp.cpuidle_enter_state
19.48 +6.3 25.80 ? 4% perf-profile.children.cycles-pp.cpuidle_idle_call
25.20 +7.1 32.30 ? 2% perf-profile.children.cycles-pp.do_idle
25.26 +7.1 32.37 ? 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
25.26 +7.1 32.37 ? 2% perf-profile.children.cycles-pp.cpu_startup_entry
2.90 -0.5 2.42 ? 2% perf-profile.self.cycles-pp.__slab_free
1.51 -0.5 1.04 ? 2% perf-profile.self.cycles-pp.sctp_outq_sack
1.01 -0.4 0.56 perf-profile.self.cycles-pp.enqueue_entity
0.55 -0.4 0.16 ? 4% perf-profile.self.cycles-pp.sctp_eat_data
0.50 -0.4 0.14 ? 14% perf-profile.self.cycles-pp.sock_kmalloc
0.69 ? 2% -0.3 0.37 ? 4% perf-profile.self.cycles-pp.sctp_wfree
0.36 ? 17% -0.3 0.04 ? 45% perf-profile.self.cycles-pp.update_sg_lb_stats
0.40 ? 12% -0.3 0.09 ? 7% perf-profile.self.cycles-pp.__copy_skb_header
0.69 -0.3 0.42 perf-profile.self.cycles-pp.kmem_cache_alloc
0.70 -0.3 0.44 ? 3% perf-profile.self.cycles-pp.sctp_packet_transmit
0.54 -0.2 0.30 ? 11% perf-profile.self.cycles-pp.resched_curr
1.06 -0.2 0.83 ? 2% perf-profile.self.cycles-pp.__list_del_entry_valid
0.99 -0.2 0.77 ? 4% perf-profile.self.cycles-pp.set_next_entity
0.41 -0.2 0.21 ? 7% perf-profile.self.cycles-pp.skb_put
0.90 ? 2% -0.2 0.72 ? 2% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.54 ? 3% -0.2 0.36 ? 19% perf-profile.self.cycles-pp.enqueue_timer
0.21 ? 3% -0.2 0.03 ?100% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.49 -0.2 0.32 ? 3% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.35 ? 9% -0.2 0.20 ? 19% perf-profile.self.cycles-pp.__rhashtable_lookup
1.19 -0.2 1.04 ? 5% perf-profile.self.cycles-pp.kfree
0.62 -0.1 0.47 ? 2% perf-profile.self.cycles-pp.sctp_rcv
0.38 -0.1 0.24 ? 6% perf-profile.self.cycles-pp.sctp_chunkify
0.38 -0.1 0.23 ? 4% perf-profile.self.cycles-pp.process_backlog
0.27 ? 2% -0.1 0.12 ? 3% perf-profile.self.cycles-pp.lock_timer_base
0.45 -0.1 0.31 ? 5% perf-profile.self.cycles-pp.pick_next_entity
0.51 ? 3% -0.1 0.38 ? 4% perf-profile.self.cycles-pp.sctp_sendmsg_to_asoc
0.25 -0.1 0.11 ? 7% perf-profile.self.cycles-pp.sctp_packet_append_chunk
0.51 -0.1 0.38 ? 5% perf-profile.self.cycles-pp.finish_task_switch
0.21 ? 7% -0.1 0.08 ? 7% perf-profile.self.cycles-pp.sctp_sf_eat_data_6_2
0.64 -0.1 0.52 ? 3% perf-profile.self.cycles-pp.__might_resched
0.89 -0.1 0.76 ? 4% perf-profile.self.cycles-pp.sctp_outq_select_transport
0.22 ? 7% -0.1 0.10 ? 5% perf-profile.self.cycles-pp.newidle_balance
0.46 ? 2% -0.1 0.34 ? 4% perf-profile.self.cycles-pp.___perf_sw_event
0.64 -0.1 0.53 ? 2% perf-profile.self.cycles-pp.__alloc_skb
0.24 ? 14% -0.1 0.12 ? 14% perf-profile.self.cycles-pp.sctp_outq_tail
0.44 ? 2% -0.1 0.33 ? 3% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.82 -0.1 0.71 perf-profile.self.cycles-pp.skb_set_owner_w
0.33 -0.1 0.23 ? 3% perf-profile.self.cycles-pp.sctp_assoc_bh_rcv
0.30 ? 4% -0.1 0.20 ? 4% perf-profile.self.cycles-pp._find_next_bit
0.50 -0.1 0.40 ? 2% perf-profile.self.cycles-pp.__list_add_valid
0.52 -0.1 0.42 ? 4% perf-profile.self.cycles-pp.sctp_association_put
0.34 ? 2% -0.1 0.25 ? 5% perf-profile.self.cycles-pp.skb_release_data
0.18 ? 6% -0.1 0.10 ? 5% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.14 ? 6% -0.1 0.05 ? 7% perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
0.26 ? 11% -0.1 0.18 ? 13% perf-profile.self.cycles-pp.sctp_v4_xmit
0.31 ? 2% -0.1 0.23 ? 4% perf-profile.self.cycles-pp.__softirqentry_text_start
0.50 -0.1 0.42 ? 4% perf-profile.self.cycles-pp.irqtime_account_irq
0.16 ? 3% -0.1 0.07 ? 6% perf-profile.self.cycles-pp.sctp_cmp_addr_exact
0.16 ? 7% -0.1 0.08 ? 6% perf-profile.self.cycles-pp.validate_xmit_skb
0.24 ? 2% -0.1 0.16 ? 2% perf-profile.self.cycles-pp.aa_sk_perm
0.32 ? 2% -0.1 0.24 ? 3% perf-profile.self.cycles-pp.__update_load_avg_se
0.30 -0.1 0.23 ? 5% perf-profile.self.cycles-pp.__next_timer_interrupt
0.49 -0.1 0.42 ? 3% perf-profile.self.cycles-pp.nr_iowait_cpu
0.22 ? 4% -0.1 0.15 ? 8% perf-profile.self.cycles-pp.sctp_ulpevent_make_sender_dry_event
0.31 ? 2% -0.1 0.25 ? 4% perf-profile.self.cycles-pp.__fget_light
0.10 ? 4% -0.1 0.04 ? 73% perf-profile.self.cycles-pp.ip_local_deliver_finish
0.54 -0.1 0.48 ? 4% perf-profile.self.cycles-pp.sctp_sf_eat_sack_6_2
0.16 ? 3% -0.1 0.10 ? 3% perf-profile.self.cycles-pp.sctp_ulpq_tail_event
0.34 ? 2% -0.1 0.28 ? 5% perf-profile.self.cycles-pp.sctp_do_sm
0.19 ? 2% -0.1 0.13 perf-profile.self.cycles-pp.dequeue_task_fair
0.38 ? 4% -0.1 0.32 ? 5% perf-profile.self.cycles-pp.__switch_to_asm
0.26 ? 2% -0.1 0.20 perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook
0.44 ? 2% -0.1 0.38 ? 5% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.23 ? 2% -0.1 0.18 ? 2% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.38 ? 2% -0.1 0.32 ? 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.70 ? 2% -0.0 0.65 ? 2% perf-profile.self.cycles-pp.memcg_slab_free_hook
1.22 -0.0 1.18 perf-profile.self.cycles-pp.update_rq_clock
0.27 ? 3% -0.0 0.22 ? 4% perf-profile.self.cycles-pp.recv_sctp_rr
0.56 -0.0 0.52 perf-profile.self.cycles-pp.memset_erms
0.15 ? 7% -0.0 0.10 ? 4% perf-profile.self.cycles-pp.update_cfs_group
0.26 -0.0 0.22 ? 2% perf-profile.self.cycles-pp.send_sctp_rr
0.14 ? 3% -0.0 0.10 ? 7% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.07 ? 7% -0.0 0.03 ?100% perf-profile.self.cycles-pp.sctp_sm_lookup_event
0.30 ? 4% -0.0 0.26 ? 4% perf-profile.self.cycles-pp.sctp_chunk_hold
0.17 ? 10% -0.0 0.13 ? 5% perf-profile.self.cycles-pp.sctp_datamsg_put
0.27 ? 4% -0.0 0.23 ? 4% perf-profile.self.cycles-pp.ip_finish_output2
0.14 ? 2% -0.0 0.11 ? 6% perf-profile.self.cycles-pp.security_socket_sendmsg
0.21 -0.0 0.18 ? 7% perf-profile.self.cycles-pp.sendmsg
0.18 ? 2% -0.0 0.14 ? 4% perf-profile.self.cycles-pp.__cond_resched
0.08 ? 6% -0.0 0.04 ? 72% perf-profile.self.cycles-pp.sockfd_lookup_light
0.14 ? 3% -0.0 0.11 ? 7% perf-profile.self.cycles-pp.sctp_primitive_SEND
0.26 ? 5% -0.0 0.22 ? 5% perf-profile.self.cycles-pp.sctp_outq_flush
0.18 ? 2% -0.0 0.14 ? 7% perf-profile.self.cycles-pp._copy_from_user
0.16 ? 2% -0.0 0.13 ? 4% perf-profile.self.cycles-pp.__virt_addr_valid
0.09 ? 7% -0.0 0.06 ? 9% perf-profile.self.cycles-pp.sctp_bind_addr_match
0.22 -0.0 0.19 ? 4% perf-profile.self.cycles-pp.dequeue_entity
0.12 ? 4% -0.0 0.09 ? 12% perf-profile.self.cycles-pp.sctp_hash_cmp
0.08 ? 4% -0.0 0.06 ? 9% perf-profile.self.cycles-pp.perf_tp_event
0.15 ? 7% -0.0 0.12 ? 7% perf-profile.self.cycles-pp.__dev_queue_xmit
0.12 ? 6% -0.0 0.09 ? 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.10 ? 6% -0.0 0.08 ? 8% perf-profile.self.cycles-pp.__skb_clone
0.12 ? 4% -0.0 0.10 ? 6% perf-profile.self.cycles-pp.__netif_receive_skb_one_core
0.09 -0.0 0.07 ? 5% perf-profile.self.cycles-pp.update_min_vruntime
0.32 -0.0 0.30 ? 4% perf-profile.self.cycles-pp.update_curr
0.24 ? 2% -0.0 0.22 ? 4% perf-profile.self.cycles-pp.loopback_xmit
0.24 -0.0 0.22 ? 4% perf-profile.self.cycles-pp.save_fpregs_to_fpstate
0.11 ? 5% -0.0 0.09 ? 9% perf-profile.self.cycles-pp.__import_iovec
0.12 ? 3% -0.0 0.10 ? 4% perf-profile.self.cycles-pp.__put_user_nocheck_4
0.11 ? 6% -0.0 0.09 ? 10% perf-profile.self.cycles-pp.update_irq_load_avg
0.08 -0.0 0.06 ? 6% perf-profile.self.cycles-pp.rcu_all_qs
0.08 ? 6% -0.0 0.06 ? 8% perf-profile.self.cycles-pp.kmalloc_slab
0.07 ? 5% -0.0 0.05 perf-profile.self.cycles-pp.check_stack_object
0.09 ? 4% -0.0 0.07 perf-profile.self.cycles-pp.kmalloc_reserve
0.07 -0.0 0.05 ? 8% perf-profile.self.cycles-pp.__update_idle_core
0.12 ? 4% -0.0 0.10 ? 8% perf-profile.self.cycles-pp.pick_next_task_fair
0.10 ? 9% -0.0 0.08 perf-profile.self.cycles-pp.ip_rcv_core
0.09 -0.0 0.08 ? 6% perf-profile.self.cycles-pp.__put_user_nocheck_8
0.11 ? 3% -0.0 0.10 ? 4% perf-profile.self.cycles-pp.sctp_sock_rfree
0.18 ? 2% -0.0 0.17 ? 2% perf-profile.self.cycles-pp.sctp_packet_pack
0.10 ? 5% -0.0 0.08 ? 5% perf-profile.self.cycles-pp.___sys_sendmsg
0.07 -0.0 0.06 ? 6% perf-profile.self.cycles-pp.lock_sock_nested
0.09 -0.0 0.08 ? 4% perf-profile.self.cycles-pp.__build_skb_around
0.08 -0.0 0.07 ? 5% perf-profile.self.cycles-pp._copy_to_iter
0.12 ? 3% +0.0 0.14 ? 3% perf-profile.self.cycles-pp.__entry_text_start
0.10 +0.0 0.12 ? 4% perf-profile.self.cycles-pp.schedule_idle
0.06 +0.0 0.08 ? 7% perf-profile.self.cycles-pp.tick_nohz_next_event
0.09 ? 4% +0.0 0.11 ? 5% perf-profile.self.cycles-pp.poll_idle
0.10 ? 3% +0.0 0.12 ? 4% perf-profile.self.cycles-pp.call_cpuidle
0.07 ? 9% +0.0 0.10 ? 14% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.25 ? 3% +0.0 0.28 ? 4% perf-profile.self.cycles-pp.cpuidle_idle_call
0.32 +0.0 0.35 ? 3% perf-profile.self.cycles-pp.do_idle
0.14 ? 2% +0.0 0.17 ? 4% perf-profile.self.cycles-pp.skb_release_head_state
0.14 ? 2% +0.0 0.17 ? 6% perf-profile.self.cycles-pp.sock_wfree
0.16 ? 3% +0.0 0.20 ? 4% perf-profile.self.cycles-pp.rcu_idle_exit
0.13 ? 2% +0.0 0.16 ? 4% perf-profile.self.cycles-pp.____sys_recvmsg
0.10 +0.0 0.13 ? 6% perf-profile.self.cycles-pp.rcu_dynticks_inc
0.45 +0.0 0.48 ? 2% perf-profile.self.cycles-pp.__wake_up_common
0.02 ?141% +0.0 0.06 ? 11% perf-profile.self.cycles-pp.ttwu_do_activate
0.06 ? 7% +0.0 0.11 ? 8% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.07 ? 5% +0.0 0.12 ? 6% perf-profile.self.cycles-pp._copy_from_iter
0.55 +0.0 0.59 ? 3% perf-profile.self.cycles-pp.try_to_wake_up
1.51 +0.0 1.56 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.20 +0.0 0.24 ? 3% perf-profile.self.cycles-pp._sctp_make_chunk
0.60 +0.0 0.64 ? 2% perf-profile.self.cycles-pp.sctp_inet_skb_msgname
0.21 ? 2% +0.1 0.26 ? 2% perf-profile.self.cycles-pp.net_rx_action
0.00 +0.1 0.06 ? 11% perf-profile.self.cycles-pp.sctp_v4_ecn_capable
0.60 +0.1 0.66 perf-profile.self.cycles-pp.sctp_make_sack
1.24 +0.1 1.31 ? 2% perf-profile.self.cycles-pp.kmem_cache_free
0.77 +0.1 0.84 ? 2% perf-profile.self.cycles-pp.sctp_skb_recv_datagram
0.51 +0.1 0.58 ? 4% perf-profile.self.cycles-pp.sctp_data_ready
0.00 +0.1 0.07 ? 12% perf-profile.self.cycles-pp.lapic_next_deadline
0.07 ? 5% +0.1 0.14 ? 9% perf-profile.self.cycles-pp.check_preempt_curr
0.17 ? 2% +0.1 0.24 ? 3% perf-profile.self.cycles-pp.sctp_outq_flush_data
0.37 +0.1 0.45 ? 2% perf-profile.self.cycles-pp.read_tsc
0.22 +0.1 0.30 ? 5% perf-profile.self.cycles-pp.sctp_association_hold
0.00 +0.1 0.08 ? 11% perf-profile.self.cycles-pp.llist_add_batch
0.00 +0.1 0.08 ? 42% perf-profile.self.cycles-pp.timekeeping_max_deferment
0.23 ? 4% +0.1 0.31 ? 6% perf-profile.self.cycles-pp.consume_skb
0.26 ? 4% +0.1 0.35 ? 4% perf-profile.self.cycles-pp.available_idle_cpu
0.16 ? 3% +0.1 0.25 ? 4% perf-profile.self.cycles-pp.___sys_recvmsg
0.00 +0.1 0.10 ? 8% perf-profile.self.cycles-pp.native_irq_return_iret
0.72 +0.1 0.82 perf-profile.self.cycles-pp.sctp_datamsg_from_user
0.36 ? 2% +0.1 0.46 ? 2% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.33 ? 4% +0.1 0.44 ? 10% perf-profile.self.cycles-pp.ktime_get
0.30 ? 3% +0.1 0.41 ? 5% perf-profile.self.cycles-pp.__sctp_write_space
0.00 +0.1 0.12 ? 14% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.98 +0.1 1.09 ? 2% perf-profile.self.cycles-pp.sctp_recvmsg
0.93 +0.1 1.05 ? 3% perf-profile.self.cycles-pp.__schedule
1.15 +0.1 1.27 ? 2% perf-profile.self.cycles-pp.__check_object_size
0.07 ? 6% +0.1 0.20 ? 4% perf-profile.self.cycles-pp.release_sock
1.02 +0.1 1.15 ? 2% perf-profile.self.cycles-pp.sctp_cmd_interpreter
0.60 ? 2% +0.1 0.74 ? 2% perf-profile.self.cycles-pp.sctp_transport_hold
0.59 +0.2 0.75 ? 3% perf-profile.self.cycles-pp.sctp_chunk_put
0.00 +0.2 0.16 ? 13% perf-profile.self.cycles-pp.flush_smp_call_function_from_idle
0.48 +0.2 0.65 perf-profile.self.cycles-pp.sctp_chunk_abandoned
0.08 ? 6% +0.2 0.25 ? 5% perf-profile.self.cycles-pp.switch_fpu_return
0.32 ? 3% +0.2 0.51 ? 5% perf-profile.self.cycles-pp.schedule_timeout
0.11 ? 4% +0.2 0.30 ? 5% perf-profile.self.cycles-pp.__copy_msghdr_from_user
0.32 +0.2 0.51 ? 6% perf-profile.self.cycles-pp.sctp_transport_put
0.00 +0.2 0.21 ? 11% perf-profile.self.cycles-pp.llist_reverse_order
0.74 +0.2 0.95 ? 2% perf-profile.self.cycles-pp.sctp_packet_config
0.30 +0.2 0.52 ? 11% perf-profile.self.cycles-pp.cpuidle_enter_state
0.21 ? 9% +0.2 0.46 ? 2% perf-profile.self.cycles-pp.ipv4_dst_check
0.15 ? 3% +0.3 0.42 ? 6% perf-profile.self.cycles-pp.sctp_tsnmap_check
0.48 ? 2% +0.3 0.78 ? 2% perf-profile.self.cycles-pp.sctp_check_transmitted
0.82 ? 2% +0.3 1.13 ? 4% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +0.3 0.34 ? 14% perf-profile.self.cycles-pp.send_call_function_single_ipi
1.12 +0.4 1.50 perf-profile.self.cycles-pp.__ksize
2.08 ? 2% +0.8 2.84 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
15.73 +4.7 20.40 ? 2% perf-profile.self.cycles-pp.mwait_idle_with_hints


***************************************************************************************************
lkp-icl-2sp4: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz with 128G memory
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/send_size/tbox_group/test/testcase/ucode:
cs-localhost/gcc-11/performance/ipv4/x86_64-rhel-8.3/1/debian-10.4-x86_64-20200603.cgz/300s/10K/lkp-icl-2sp4/SCTP_STREAM_MANY/netperf/0xd000331

commit:
4033d677de ("Merge branch 'for-5.19/io_uring-socket' into sock-nolock.2")
5aa32dd9bd ("net: add support for fast socket release")

4033d677de27560d 5aa32dd9bdb57e4a26143216df5
---------------- ---------------------------
%stddev %change %stddev
\ | \
26120 ? 3% -89.4% 2765 ? 43% netperf.Throughput_Mbps
26120 ? 3% -89.4% 2765 ? 43% netperf.Throughput_total_Mbps
68.00 ? 3% -90.2% 6.67 ? 45% netperf.time.percent_of_cpu_this_job_got
200.48 ? 3% -89.4% 21.27 ? 43% netperf.time.system_time
95657220 ? 3% -89.4% 10129003 ? 43% netperf.workload
1.037e+08 -29.8% 72801990 ? 17% cpuidle..usage
1.00 -100.0% 0.00 vmstat.procs.r
181178 ? 3% -88.7% 20405 ? 40% vmstat.system.cs
9846 ? 3% -26.4% 7249 ? 4% meminfo.Active
9565 ? 3% -25.7% 7108 ? 4% meminfo.Active(anon)
43557 ? 5% -40.3% 26003 meminfo.Shmem
0.26 ? 9% -0.1 0.12 ? 39% mpstat.cpu.all.soft%
0.87 ? 3% -0.8 0.11 ? 35% mpstat.cpu.all.sys%
0.05 ? 2% -0.0 0.02 ? 8% mpstat.cpu.all.usr%
7848 ? 14% -28.2% 5634 ? 17% numa-meminfo.node1.Active
7688 ? 12% -27.0% 5610 ? 17% numa-meminfo.node1.Active(anon)
25304 ? 14% -61.1% 9832 ? 33% numa-meminfo.node1.Shmem
1921 ? 13% -26.9% 1403 ? 17% numa-vmstat.node1.nr_active_anon
6113 ? 17% -60.0% 2446 ? 33% numa-vmstat.node1.nr_shmem
1921 ? 13% -26.9% 1403 ? 17% numa-vmstat.node1.nr_zone_active_anon
91.17 ? 12% -49.4% 46.17 ? 22% turbostat.Avg_MHz
1977 ? 3% -47.4% 1040 ? 6% turbostat.Bzy_MHz
27146212 ? 3% -88.7% 3079005 ? 41% turbostat.C1
0.19 ? 17% -50.0% 0.10 ? 25% turbostat.IPC
64301 ? 3% -71.0% 18646 ? 42% turbostat.POLL
2391 ? 3% -25.7% 1775 ? 4% proc-vmstat.nr_active_anon
75563 -5.0% 71769 proc-vmstat.nr_inactive_anon
10126 -7.4% 9372 proc-vmstat.nr_mapped
10974 ? 5% -40.7% 6505 proc-vmstat.nr_shmem
2391 ? 3% -25.7% 1775 ? 4% proc-vmstat.nr_zone_active_anon
75563 -5.0% 71769 proc-vmstat.nr_zone_inactive_anon
1.262e+08 ? 3% -88.8% 14105228 ? 40% proc-vmstat.numa_hit
1.254e+08 ? 3% -88.9% 13940984 ? 40% proc-vmstat.numa_local
41165 ? 27% -72.1% 11505 ? 94% proc-vmstat.numa_pte_updates
185099 ? 11% -97.0% 5540 ? 23% proc-vmstat.pgactivate
3.291e+08 ? 3% -89.1% 35736732 ? 42% proc-vmstat.pgalloc_normal
1082855 -1.6% 1065851 proc-vmstat.pgfault
3.291e+08 ? 3% -89.1% 35774585 ? 42% proc-vmstat.pgfree
0.09 ? 19% -33.5% 0.06 ? 14% sched_debug.cfs_rq:/.h_nr_running.avg
0.27 ? 7% -18.2% 0.22 ? 7% sched_debug.cfs_rq:/.h_nr_running.stddev
339173 ? 16% +101.0% 681903 ? 17% sched_debug.cfs_rq:/.load.max
72238 ? 14% +48.3% 107141 ? 11% sched_debug.cfs_rq:/.load.stddev
637.81 ? 9% +56.9% 1000 ? 30% sched_debug.cfs_rq:/.load_avg.max
12493 ? 20% -26.6% 9173 ? 19% sched_debug.cfs_rq:/.min_vruntime.avg
55491 ? 16% -50.9% 27233 ? 24% sched_debug.cfs_rq:/.min_vruntime.max
9118 ? 12% -55.2% 4080 ? 17% sched_debug.cfs_rq:/.min_vruntime.stddev
0.09 ? 19% -33.3% 0.06 ? 14% sched_debug.cfs_rq:/.nr_running.avg
0.26 ? 7% -17.7% 0.22 ? 7% sched_debug.cfs_rq:/.nr_running.stddev
94.00 ? 10% -31.0% 64.83 ? 14% sched_debug.cfs_rq:/.runnable_avg.avg
1011 ? 4% -34.3% 664.78 ? 3% sched_debug.cfs_rq:/.runnable_avg.max
203.48 ? 5% -37.2% 127.70 ? 13% sched_debug.cfs_rq:/.runnable_avg.stddev
28619 ? 49% -87.4% 3612 ?109% sched_debug.cfs_rq:/.spread0.max
9119 ? 12% -55.2% 4082 ? 17% sched_debug.cfs_rq:/.spread0.stddev
93.77 ? 10% -31.0% 64.68 ? 14% sched_debug.cfs_rq:/.util_avg.avg
1009 ? 4% -34.2% 664.03 ? 3% sched_debug.cfs_rq:/.util_avg.max
203.16 ? 5% -37.2% 127.54 ? 13% sched_debug.cfs_rq:/.util_avg.stddev
29.56 ? 25% -77.9% 6.53 ? 37% sched_debug.cfs_rq:/.util_est_enqueued.avg
901.92 ? 11% -74.2% 232.78 ? 16% sched_debug.cfs_rq:/.util_est_enqueued.max
139.45 ? 11% -76.0% 33.42 ? 27% sched_debug.cfs_rq:/.util_est_enqueued.stddev
7123 ?124% +2073.3% 154810 ? 70% sched_debug.cpu.avg_idle.min
145981 ? 4% -20.7% 115693 ? 12% sched_debug.cpu.avg_idle.stddev
1171 ? 16% -54.3% 535.47 ? 26% sched_debug.cpu.clock_task.stddev
210.71 ? 9% -19.0% 170.71 ? 7% sched_debug.cpu.curr->pid.avg
1207 ? 3% -10.3% 1083 ? 2% sched_debug.cpu.curr->pid.stddev
0.19 ? 5% -15.8% 0.16 ? 3% sched_debug.cpu.nr_running.stddev
215273 ? 3% -87.3% 27274 ? 38% sched_debug.cpu.nr_switches.avg
5835932 ? 20% -78.6% 1247699 ? 30% sched_debug.cpu.nr_switches.max
806025 ? 15% -82.6% 139959 ? 29% sched_debug.cpu.nr_switches.stddev
1.301e+09 ? 2% -75.2% 3.222e+08 ? 8% perf-stat.i.branch-instructions
0.20 ? 31% +1.2 1.40 ? 84% perf-stat.i.cache-miss-rate%
363891 ? 7% -26.6% 267229 ? 4% perf-stat.i.cache-misses
2.12e+08 ? 21% -66.3% 71465630 ? 76% perf-stat.i.cache-references
182433 ? 3% -88.8% 20505 ? 40% perf-stat.i.context-switches
1.78 ? 12% +90.9% 3.40 ? 23% perf-stat.i.cpi
1.146e+10 ? 10% -58.5% 4.754e+09 ? 19% perf-stat.i.cpu-cycles
47305 ? 10% -56.1% 20788 ? 14% perf-stat.i.cycles-between-cache-misses
1.864e+09 ? 2% -76.2% 4.436e+08 ? 9% perf-stat.i.dTLB-loads
1.085e+09 ? 2% -77.4% 2.447e+08 ? 10% perf-stat.i.dTLB-stores
6.539e+09 ? 2% -75.5% 1.605e+09 ? 8% perf-stat.i.instructions
0.58 ? 13% -42.8% 0.33 ? 20% perf-stat.i.ipc
0.94 ? 5% +27.4% 1.20 ? 9% perf-stat.i.major-faults
0.09 ? 10% -58.6% 0.04 ? 19% perf-stat.i.metric.GHz
62.43 ?134% +580.9% 425.09 ? 69% perf-stat.i.metric.K/sec
34.80 -76.9% 8.03 ? 9% perf-stat.i.metric.M/sec
3213 -2.1% 3144 perf-stat.i.minor-faults
18739 ? 12% -45.6% 10195 ? 6% perf-stat.i.node-loads
65638 ? 14% -26.4% 48301 ? 11% perf-stat.i.node-stores
3214 -2.1% 3145 perf-stat.i.page-faults
0.18 ? 27% +0.7 0.86 ? 74% perf-stat.overall.cache-miss-rate%
1.76 ? 12% +70.4% 3.00 ? 23% perf-stat.overall.cpi
31731 ? 13% -44.2% 17693 ? 15% perf-stat.overall.cycles-between-cache-misses
0.58 ? 13% -39.3% 0.35 ? 20% perf-stat.overall.ipc
76.18 ? 3% +8.2 84.36 perf-stat.overall.node-load-miss-rate%
20575 +163.9% 54303 ? 29% perf-stat.overall.path-length
1.296e+09 ? 2% -75.2% 3.211e+08 ? 8% perf-stat.ps.branch-instructions
362514 ? 7% -26.5% 266277 ? 4% perf-stat.ps.cache-misses
2.113e+08 ? 21% -66.3% 71226434 ? 76% perf-stat.ps.cache-references
181839 ? 3% -88.8% 20437 ? 40% perf-stat.ps.context-switches
1.143e+10 ? 10% -58.5% 4.738e+09 ? 19% perf-stat.ps.cpu-cycles
1.857e+09 ? 2% -76.2% 4.421e+08 ? 9% perf-stat.ps.dTLB-loads
1.081e+09 ? 2% -77.4% 2.439e+08 ? 10% perf-stat.ps.dTLB-stores
6.517e+09 ? 2% -75.5% 1.599e+09 ? 8% perf-stat.ps.instructions
0.94 ? 5% +27.3% 1.19 ? 9% perf-stat.ps.major-faults
3200 -2.1% 3131 perf-stat.ps.minor-faults
18658 ? 12% -45.6% 10158 ? 6% perf-stat.ps.node-loads
65389 ? 14% -26.4% 48115 ? 11% perf-stat.ps.node-stores
3201 -2.1% 3132 perf-stat.ps.page-faults
1.968e+12 ? 2% -75.3% 4.853e+11 ? 8% perf-stat.total.instructions
28.26 ? 22% -23.2 5.06 ? 35% perf-profile.calltrace.cycles-pp.__libc_start_main
28.07 ? 22% -23.2 4.90 ? 37% perf-profile.calltrace.cycles-pp.main.__libc_start_main
27.53 ? 22% -22.7 4.80 ? 37% perf-profile.calltrace.cycles-pp.sendmsg.main.__libc_start_main
27.16 ? 22% -22.4 4.73 ? 37% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendmsg.main.__libc_start_main
27.00 ? 22% -22.3 4.70 ? 37% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.main.__libc_start_main
26.82 ? 22% -22.2 4.66 ? 37% perf-profile.calltrace.cycles-pp.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.main
26.63 ? 22% -22.0 4.62 ? 37% perf-profile.calltrace.cycles-pp.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg
25.53 ? 22% -21.1 4.42 ? 37% perf-profile.calltrace.cycles-pp.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
24.53 ? 22% -20.3 4.24 ? 38% perf-profile.calltrace.cycles-pp.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64
24.27 ? 22% -20.1 4.19 ? 38% perf-profile.calltrace.cycles-pp.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg
13.89 ? 21% -11.6 2.34 ? 39% perf-profile.calltrace.cycles-pp.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
10.92 ? 5% -8.4 2.54 ? 31% perf-profile.calltrace.cycles-pp.recvmsg
10.46 ? 5% -8.0 2.42 ? 31% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recvmsg
10.37 ? 5% -8.0 2.39 ? 31% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
10.24 ? 5% -7.9 2.36 ? 31% perf-profile.calltrace.cycles-pp.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
10.10 ? 5% -7.8 2.32 ? 31% perf-profile.calltrace.cycles-pp.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
9.45 ? 23% -7.8 1.69 ? 37% perf-profile.calltrace.cycles-pp.release_sock.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
9.38 ? 23% -7.7 1.68 ? 37% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
9.34 ? 23% -7.7 1.68 ? 37% perf-profile.calltrace.cycles-pp.sctp_backlog_rcv.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg
9.47 ? 5% -7.3 2.18 ? 31% perf-profile.calltrace.cycles-pp.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.05 ? 5% -7.0 2.09 ? 31% perf-profile.calltrace.cycles-pp.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64
9.01 ? 5% -6.9 2.08 ? 31% perf-profile.calltrace.cycles-pp.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg
8.31 ? 21% -6.9 1.40 ? 41% perf-profile.calltrace.cycles-pp.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
7.25 ? 21% -6.1 1.20 ? 43% perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc
7.03 ? 22% -6.0 0.98 ? 86% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
7.29 ? 24% -6.0 1.26 ? 37% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock.release_sock.sctp_sendmsg
7.30 ? 24% -6.0 1.27 ? 36% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock
6.92 ? 22% -6.0 0.96 ? 86% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
7.14 ? 24% -5.9 1.24 ? 36% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock.release_sock
6.74 ? 22% -5.8 0.95 ? 86% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush
6.70 ? 22% -5.8 0.94 ? 86% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit
6.66 ? 22% -5.7 0.94 ? 86% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
6.50 ? 22% -5.7 0.83 ?102% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
6.44 ? 22% -5.6 0.83 ?102% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
6.42 ? 22% -5.6 0.83 ?102% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
6.30 ? 22% -5.5 0.81 ?102% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
6.69 ? 23% -5.5 1.22 ? 37% perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv
6.54 ? 27% -5.4 1.14 ? 37% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
6.09 ? 22% -5.4 0.71 ?111% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
6.06 ? 22% -5.4 0.71 ?110% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
5.99 ? 22% -5.3 0.70 ?110% perf-profile.calltrace.cycles-pp.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
6.24 ? 22% -5.2 1.05 ? 45% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND
5.50 ? 21% -4.8 0.74 ? 75% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg
5.36 ? 21% -4.6 0.73 ? 75% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
5.07 ? 21% -4.5 0.58 ?111% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
4.81 ? 23% -4.4 0.41 ?118% perf-profile.calltrace.cycles-pp.sctp_packet_pack.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
5.08 ? 21% -4.3 0.76 ? 55% perf-profile.calltrace.cycles-pp.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
4.74 ? 21% -4.2 0.55 ?111% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
4.32 ? 21% -3.9 0.44 ?116% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu
4.93 ? 5% -3.8 1.13 ? 31% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
4.91 ? 5% -3.8 1.13 ? 31% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
4.20 ? 5% -3.3 0.90 ? 51% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg
4.08 ? 5% -3.2 0.87 ? 51% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg
4.06 ? 5% -3.2 0.87 ? 51% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
0.49 ? 70% +0.5 0.95 ? 19% perf-profile.calltrace.cycles-pp.rcu_idle_exit.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.28 ?100% +0.5 0.76 ? 12% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
0.47 ? 70% +0.6 1.08 ? 21% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt
0.08 ?223% +0.6 0.72 ? 23% perf-profile.calltrace.cycles-pp.update_rq_clock.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
0.00 +0.7 0.68 ? 13% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.09 ?223% +0.7 0.77 ? 14% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu
0.00 +0.7 0.71 ? 13% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start
0.00 +0.7 0.74 ? 27% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.88 ? 17% +0.8 1.66 ? 12% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.8 0.79 ? 17% perf-profile.calltrace.cycles-pp.arch_scale_freq_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
0.50 ? 45% +0.8 1.31 ? 30% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.96 ? 13% +0.8 1.78 ? 15% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
0.83 ? 19% +1.0 1.85 ? 21% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.85 ? 51% +1.2 2.02 ? 17% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.51 ? 18% +1.2 2.72 ? 9% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.08 ?223% +1.2 1.30 ? 28% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.90 ? 51% +1.2 2.12 ? 17% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
1.51 ? 24% +1.3 2.78 ? 19% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.93 ? 26% +1.6 3.56 ? 17% perf-profile.calltrace.cycles-pp.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
7.52 ? 32% +7.5 15.02 ? 20% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
7.82 ? 33% +7.6 15.43 ? 21% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
35.11 +11.3 46.39 ? 13% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
11.67 ? 31% +11.3 22.99 ? 19% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
35.38 ? 2% +11.5 46.88 ? 13% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
14.93 ? 25% +15.2 30.10 ? 12% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
51.69 ? 9% +27.4 79.13 ? 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
52.49 ? 9% +27.8 80.28 ? 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
59.18 ? 10% +30.3 89.51 ? 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
59.30 ? 10% +30.4 89.69 ? 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.secondary_startup_64_no_verify
59.70 ? 10% +30.9 90.62 ? 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
57.21 ? 10% +31.0 88.18 ? 2% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
37.96 ? 16% -30.0 7.96 ? 32% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
37.73 ? 16% -29.8 7.89 ? 32% perf-profile.children.cycles-pp.do_syscall_64
28.26 ? 22% -23.2 5.06 ? 35% perf-profile.children.cycles-pp.main
28.26 ? 22% -23.2 5.06 ? 35% perf-profile.children.cycles-pp.__libc_start_main
27.68 ? 22% -22.8 4.83 ? 37% perf-profile.children.cycles-pp.sendmsg
26.83 ? 22% -22.2 4.66 ? 37% perf-profile.children.cycles-pp.__sys_sendmsg
26.65 ? 22% -22.0 4.62 ? 37% perf-profile.children.cycles-pp.___sys_sendmsg
25.54 ? 22% -21.1 4.43 ? 37% perf-profile.children.cycles-pp.____sys_sendmsg
24.54 ? 22% -20.3 4.24 ? 38% perf-profile.children.cycles-pp.sock_sendmsg
24.50 ? 22% -20.3 4.24 ? 38% perf-profile.children.cycles-pp.sctp_sendmsg
17.28 ? 22% -14.3 3.02 ? 39% perf-profile.children.cycles-pp.sctp_do_sm
17.15 ? 22% -14.1 3.00 ? 39% perf-profile.children.cycles-pp.sctp_cmd_interpreter
14.68 ? 22% -12.1 2.57 ? 39% perf-profile.children.cycles-pp.sctp_outq_flush
13.92 ? 21% -11.6 2.35 ? 39% perf-profile.children.cycles-pp.sctp_sendmsg_to_asoc
13.02 ? 22% -10.7 2.31 ? 40% perf-profile.children.cycles-pp.sctp_packet_transmit
12.39 ? 23% -10.2 2.22 ? 38% perf-profile.children.cycles-pp.sctp_assoc_bh_rcv
11.02 ? 5% -8.5 2.56 ? 31% perf-profile.children.cycles-pp.recvmsg
10.25 ? 5% -7.9 2.36 ? 31% perf-profile.children.cycles-pp.__sys_recvmsg
9.56 ? 23% -7.8 1.72 ? 36% perf-profile.children.cycles-pp.release_sock
10.13 ? 5% -7.8 2.33 ? 31% perf-profile.children.cycles-pp.___sys_recvmsg
9.38 ? 23% -7.7 1.68 ? 37% perf-profile.children.cycles-pp.__release_sock
9.35 ? 23% -7.7 1.68 ? 37% perf-profile.children.cycles-pp.sctp_backlog_rcv
9.48 ? 5% -7.3 2.18 ? 32% perf-profile.children.cycles-pp.____sys_recvmsg
9.19 ? 5% -7.1 2.13 ? 31% perf-profile.children.cycles-pp.sctp_recvmsg
9.05 ? 5% -7.0 2.09 ? 31% perf-profile.children.cycles-pp.inet_recvmsg
8.32 ? 21% -6.9 1.40 ? 41% perf-profile.children.cycles-pp.sctp_primitive_SEND
7.23 ? 22% -5.9 1.33 ? 42% perf-profile.children.cycles-pp.__ip_queue_xmit
7.08 ? 22% -5.8 1.30 ? 41% perf-profile.children.cycles-pp.ip_finish_output2
6.82 ? 22% -5.6 1.26 ? 40% perf-profile.children.cycles-pp.__local_bh_enable_ip
6.97 ? 9% -5.5 1.46 ? 30% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
6.71 ? 22% -5.5 1.24 ? 41% perf-profile.children.cycles-pp.do_softirq
6.50 ? 22% -5.3 1.21 ? 41% perf-profile.children.cycles-pp.net_rx_action
6.44 ? 22% -5.2 1.20 ? 41% perf-profile.children.cycles-pp.__napi_poll
6.43 ? 22% -5.2 1.20 ? 42% perf-profile.children.cycles-pp.process_backlog
6.30 ? 22% -5.1 1.18 ? 41% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
6.10 ? 22% -5.0 1.14 ? 42% perf-profile.children.cycles-pp.ip_local_deliver_finish
6.07 ? 22% -4.9 1.14 ? 42% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
6.02 ? 22% -4.9 1.13 ? 42% perf-profile.children.cycles-pp.sctp_rcv
5.09 ? 21% -4.2 0.84 ? 35% perf-profile.children.cycles-pp.sctp_datamsg_from_user
5.02 ? 22% -4.2 0.85 ? 38% perf-profile.children.cycles-pp.sctp_packet_pack
8.21 ? 13% -4.1 4.10 ? 19% perf-profile.children.cycles-pp.__softirqentry_text_start
4.68 ? 23% -3.9 0.79 ? 39% perf-profile.children.cycles-pp.memcpy_erms
4.93 ? 5% -3.8 1.13 ? 31% perf-profile.children.cycles-pp.skb_copy_datagram_iter
4.92 ? 5% -3.8 1.13 ? 31% perf-profile.children.cycles-pp.__skb_datagram_iter
4.20 ? 5% -3.2 0.98 ? 31% perf-profile.children.cycles-pp._copy_to_iter
4.09 ? 5% -3.1 0.96 ? 31% perf-profile.children.cycles-pp.copyout
3.15 ? 22% -2.6 0.58 ? 43% perf-profile.children.cycles-pp.sctp_ulpq_tail_data
2.36 ? 21% -2.0 0.40 ? 36% perf-profile.children.cycles-pp.sctp_user_addto_chunk
2.20 ? 25% -1.8 0.38 ? 38% perf-profile.children.cycles-pp.sctp_outq_sack
2.12 ? 22% -1.8 0.34 ? 35% perf-profile.children.cycles-pp.sctp_make_datafrag_empty
2.11 ? 21% -1.8 0.34 ? 33% perf-profile.children.cycles-pp._copy_from_iter
2.18 ? 21% -1.8 0.42 ? 40% perf-profile.children.cycles-pp.sctp_chunk_put
2.12 ? 23% -1.8 0.37 ? 36% perf-profile.children.cycles-pp.__alloc_skb
1.97 ? 21% -1.7 0.32 ? 31% perf-profile.children.cycles-pp.copyin
1.93 ? 22% -1.6 0.33 ? 36% perf-profile.children.cycles-pp._sctp_make_chunk
1.93 ? 20% -1.6 0.35 ? 40% perf-profile.children.cycles-pp.sctp_ulpq_tail_event
1.83 ? 20% -1.5 0.34 ? 41% perf-profile.children.cycles-pp.sctp_data_ready
1.63 ? 20% -1.3 0.31 ? 39% perf-profile.children.cycles-pp.__wake_up_common_lock
1.83 ? 6% -1.3 0.52 ? 19% perf-profile.children.cycles-pp.__schedule
1.58 ? 20% -1.3 0.30 ? 39% perf-profile.children.cycles-pp.__wake_up_common
1.62 ? 7% -1.3 0.34 ? 34% perf-profile.children.cycles-pp.sctp_skb_recv_datagram
1.52 ? 21% -1.3 0.25 ? 31% perf-profile.children.cycles-pp.sctp_outq_flush_data
1.49 ? 24% -1.2 0.26 ? 34% perf-profile.children.cycles-pp.kmalloc_reserve
1.44 ? 25% -1.2 0.25 ? 34% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
1.41 ? 20% -1.1 0.27 ? 41% perf-profile.children.cycles-pp.autoremove_wake_function
1.42 ? 20% -1.1 0.29 ? 40% perf-profile.children.cycles-pp.try_to_wake_up
1.34 ? 25% -1.1 0.23 ? 37% perf-profile.children.cycles-pp.kmalloc_large_node
1.34 ? 13% -1.1 0.26 ? 29% perf-profile.children.cycles-pp._copy_from_user
1.31 ? 21% -1.1 0.24 ? 48% perf-profile.children.cycles-pp.consume_skb
1.30 ? 15% -1.0 0.30 ? 25% perf-profile.children.cycles-pp.kmem_cache_free
1.20 ? 13% -1.0 0.24 ? 41% perf-profile.children.cycles-pp.skb_release_data
1.15 ? 24% -0.9 0.23 ? 30% perf-profile.children.cycles-pp.__alloc_pages
1.06 ? 24% -0.9 0.20 ? 46% perf-profile.children.cycles-pp.sctp_ulpevent_make_rcvmsg
1.04 ? 20% -0.9 0.18 ? 35% perf-profile.children.cycles-pp.sendmsg_copy_msghdr
0.99 ? 7% -0.8 0.21 ? 34% perf-profile.children.cycles-pp.schedule_timeout
0.91 ? 24% -0.7 0.18 ? 32% perf-profile.children.cycles-pp.get_page_from_freelist
0.89 ? 11% -0.7 0.18 ? 31% perf-profile.children.cycles-pp.__check_object_size
0.87 ? 14% -0.7 0.17 ? 37% perf-profile.children.cycles-pp.__copy_msghdr_from_user
0.94 ? 6% -0.7 0.25 ? 24% perf-profile.children.cycles-pp.schedule_idle
0.93 ? 6% -0.7 0.27 ? 17% perf-profile.children.cycles-pp.schedule
0.79 ? 14% -0.6 0.14 ? 38% perf-profile.children.cycles-pp.skb_release_head_state
0.80 ? 4% -0.6 0.18 ? 40% perf-profile.children.cycles-pp.kfree_skb_reason
0.67 ? 21% -0.6 0.11 ? 30% perf-profile.children.cycles-pp.sctp_outq_select_transport
0.68 ? 12% -0.5 0.13 ? 32% perf-profile.children.cycles-pp.import_iovec
0.66 ? 12% -0.5 0.13 ? 31% perf-profile.children.cycles-pp.__import_iovec
0.69 ? 7% -0.5 0.19 ? 36% perf-profile.children.cycles-pp.sctp_ulpevent_free
0.61 ? 20% -0.5 0.12 ? 49% perf-profile.children.cycles-pp.__rhashtable_lookup
0.60 ? 20% -0.5 0.11 ? 32% perf-profile.children.cycles-pp.sctp_packet_config
0.58 ? 18% -0.5 0.10 ? 25% perf-profile.children.cycles-pp.kfree
0.57 ? 9% -0.5 0.11 ? 48% perf-profile.children.cycles-pp.iovec_from_user
0.59 ? 22% -0.5 0.14 ? 36% perf-profile.children.cycles-pp.kmem_cache_alloc
0.53 ? 20% -0.4 0.10 ? 79% perf-profile.children.cycles-pp.free_unref_page
0.50 ? 24% -0.4 0.08 ? 60% perf-profile.children.cycles-pp.sctp_chunkify
0.51 ? 24% -0.4 0.10 ? 36% perf-profile.children.cycles-pp.rmqueue
0.49 ? 21% -0.4 0.10 ? 57% perf-profile.children.cycles-pp.ttwu_do_activate
0.44 ? 23% -0.4 0.06 ? 81% perf-profile.children.cycles-pp.__skb_clone
0.47 ? 20% -0.4 0.09 ? 61% perf-profile.children.cycles-pp.enqueue_task_fair
0.50 ? 7% -0.4 0.13 ? 46% perf-profile.children.cycles-pp.pick_next_task_fair
0.49 ? 15% -0.4 0.12 ? 38% perf-profile.children.cycles-pp.lock_sock_nested
0.50 ? 19% -0.4 0.14 ? 36% perf-profile.children.cycles-pp.__might_resched
0.44 ? 17% -0.4 0.08 ? 33% perf-profile.children.cycles-pp.__list_del_entry_valid
0.40 ? 9% -0.4 0.05 ?104% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.44 ? 7% -0.4 0.08 ? 32% perf-profile.children.cycles-pp.simple_copy_to_iter
0.42 ? 23% -0.3 0.08 ? 72% perf-profile.children.cycles-pp.sctp_hash_cmp
0.38 ? 22% -0.3 0.04 ?102% perf-profile.children.cycles-pp.__dev_queue_xmit
0.40 ? 25% -0.3 0.06 ? 79% perf-profile.children.cycles-pp.__sctp_rcv_lookup
0.38 ? 25% -0.3 0.06 ? 81% perf-profile.children.cycles-pp.sctp_addrs_lookup_transport
0.37 ? 23% -0.3 0.05 ? 74% perf-profile.children.cycles-pp.sctp_packet_transmit_chunk
0.36 ? 26% -0.3 0.04 ?100% perf-profile.children.cycles-pp.sctp_inq_pop
0.42 ? 8% -0.3 0.11 ? 25% perf-profile.children.cycles-pp.__slab_free
0.40 ? 21% -0.3 0.08 ? 57% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.38 ? 17% -0.3 0.06 ? 55% perf-profile.children.cycles-pp.ipv4_dst_check
0.36 ? 20% -0.3 0.05 ? 75% perf-profile.children.cycles-pp.sctp_addto_chunk
0.35 ? 16% -0.3 0.06 ? 79% perf-profile.children.cycles-pp.sctp_endpoint_lookup_assoc
0.33 ? 25% -0.3 0.04 ?101% perf-profile.children.cycles-pp.sctp_wfree
0.33 ? 27% -0.3 0.04 ? 72% perf-profile.children.cycles-pp.sctp_chunk_free
0.36 ? 23% -0.3 0.08 ? 64% perf-profile.children.cycles-pp.enqueue_entity
0.33 ? 18% -0.3 0.04 ? 73% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.32 ? 25% -0.3 0.03 ?103% perf-profile.children.cycles-pp.__copy_skb_header
0.32 ? 19% -0.3 0.04 ? 74% perf-profile.children.cycles-pp.sctp_sf_eat_data_6_2
0.33 ? 15% -0.3 0.06 ? 77% perf-profile.children.cycles-pp.sctp_epaddr_lookup_transport
0.32 ? 21% -0.3 0.04 ? 75% perf-profile.children.cycles-pp.sctp_packet_append_chunk
0.34 ? 9% -0.3 0.07 ? 81% perf-profile.children.cycles-pp.dequeue_task_fair
0.31 ? 18% -0.3 0.04 ? 73% perf-profile.children.cycles-pp.memset_erms
0.34 ? 17% -0.3 0.08 ? 25% perf-profile.children.cycles-pp.memcg_slab_free_hook
0.34 ? 21% -0.3 0.08 ? 28% perf-profile.children.cycles-pp.__might_fault
0.34 ? 7% -0.3 0.08 ? 50% perf-profile.children.cycles-pp.set_next_entity
0.31 ? 10% -0.3 0.06 ? 82% perf-profile.children.cycles-pp.dequeue_entity
0.32 ? 15% -0.2 0.08 ? 60% perf-profile.children.cycles-pp.__entry_text_start
0.30 ? 16% -0.2 0.05 ? 50% perf-profile.children.cycles-pp.move_addr_to_kernel
0.28 ? 21% -0.2 0.04 ? 75% perf-profile.children.cycles-pp.sctp_eat_data
0.30 ? 23% -0.2 0.07 ? 54% perf-profile.children.cycles-pp.__might_sleep
0.28 ? 23% -0.2 0.04 ? 77% perf-profile.children.cycles-pp.sock_kmalloc
0.28 ? 8% -0.2 0.05 ? 71% perf-profile.children.cycles-pp.dst_release
0.29 ? 26% -0.2 0.05 ? 77% perf-profile.children.cycles-pp.skb_set_owner_w
0.31 ? 17% -0.2 0.08 ? 68% perf-profile.children.cycles-pp.update_load_avg
0.26 ? 19% -0.2 0.04 ?103% perf-profile.children.cycles-pp.free_pcp_prepare
0.27 ? 21% -0.2 0.06 ? 13% perf-profile.children.cycles-pp.select_task_rq
0.24 ? 14% -0.2 0.03 ?111% perf-profile.children.cycles-pp.aa_sk_perm
0.24 ? 23% -0.2 0.03 ?101% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.23 ? 27% -0.2 0.03 ?101% perf-profile.children.cycles-pp.sctp_check_transmitted
0.25 ? 18% -0.2 0.05 ? 82% perf-profile.children.cycles-pp.sctp_gen_sack
0.23 ? 25% -0.2 0.04 ? 71% perf-profile.children.cycles-pp.__zone_watermark_ok
0.24 ? 19% -0.2 0.05 ? 85% perf-profile.children.cycles-pp.sctp_make_sack
0.24 ? 18% -0.2 0.05 ? 73% perf-profile.children.cycles-pp.__cond_resched
0.22 ? 25% -0.2 0.03 ?101% perf-profile.children.cycles-pp.sockfd_lookup_light
0.22 ? 22% -0.2 0.05 ? 49% perf-profile.children.cycles-pp.select_task_rq_fair
0.20 ? 13% -0.2 0.03 ? 70% perf-profile.children.cycles-pp.__virt_addr_valid
0.21 ? 26% -0.2 0.04 ?112% perf-profile.children.cycles-pp.sctp_ulpevent_receive_data
0.20 ? 11% -0.2 0.03 ?100% perf-profile.children.cycles-pp.__list_add_valid
0.18 ? 23% -0.2 0.03 ?100% perf-profile.children.cycles-pp.kmem_cache_alloc_trace
0.20 ? 18% -0.1 0.06 ? 30% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.16 ? 15% -0.1 0.02 ? 99% perf-profile.children.cycles-pp.__free_pages_ok
0.16 ? 26% -0.1 0.03 ?100% perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
0.17 ? 13% -0.1 0.05 ? 50% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.12 ? 17% -0.1 0.07 ? 51% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.09 ? 10% +0.0 0.14 ? 12% perf-profile.children.cycles-pp.sched_clock
0.02 ?142% +0.1 0.08 ? 24% perf-profile.children.cycles-pp.error_return
0.00 +0.1 0.06 ? 11% perf-profile.children.cycles-pp.unmap_page_range
0.06 ? 21% +0.1 0.12 ? 28% perf-profile.children.cycles-pp.cpuidle_reflect
0.04 ? 71% +0.1 0.10 ? 10% perf-profile.children.cycles-pp.can_stop_idle_tick
0.11 ? 6% +0.1 0.17 ? 21% perf-profile.children.cycles-pp.arch_cpu_idle_exit
0.00 +0.1 0.06 ? 19% perf-profile.children.cycles-pp.vm_mmap_pgoff
0.00 +0.1 0.06 ? 19% perf-profile.children.cycles-pp.do_mmap
0.00 +0.1 0.06 ? 19% perf-profile.children.cycles-pp.mmap_region
0.05 ? 51% +0.1 0.12 ? 20% perf-profile.children.cycles-pp.read_counters
0.05 ? 51% +0.1 0.12 ? 20% perf-profile.children.cycles-pp.process_interval
0.00 +0.1 0.07 ? 14% perf-profile.children.cycles-pp.unmap_vmas
0.05 ? 51% +0.1 0.12 ? 19% perf-profile.children.cycles-pp.cmd_stat
0.05 ? 51% +0.1 0.12 ? 19% perf-profile.children.cycles-pp.dispatch_events
0.00 +0.1 0.07 ? 21% perf-profile.children.cycles-pp.seq_read_iter
0.00 +0.1 0.07 ? 27% perf-profile.children.cycles-pp._dl_rtld_di_serinfo
0.00 +0.1 0.07 ? 36% perf-profile.children.cycles-pp.exit_mm
0.01 ?223% +0.1 0.08 ? 24% perf-profile.children.cycles-pp.path_openat
0.00 +0.1 0.08 ? 22% perf-profile.children.cycles-pp.new_sync_read
0.01 ?223% +0.1 0.09 ? 26% perf-profile.children.cycles-pp.do_filp_open
0.02 ?142% +0.1 0.10 ? 19% perf-profile.children.cycles-pp.hrtimer_forward
0.10 ? 33% +0.1 0.18 ? 10% perf-profile.children.cycles-pp._find_next_bit
0.06 ? 9% +0.1 0.14 ? 20% perf-profile.children.cycles-pp.__handle_mm_fault
0.13 ? 12% +0.1 0.22 ? 27% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.01 ?223% +0.1 0.09 ? 19% perf-profile.children.cycles-pp.fork
0.06 ? 6% +0.1 0.15 ? 17% perf-profile.children.cycles-pp.handle_mm_fault
0.18 ? 17% +0.1 0.27 ? 24% perf-profile.children.cycles-pp.call_cpuidle
0.02 ? 99% +0.1 0.11 ? 23% perf-profile.children.cycles-pp.bprm_execve
0.02 ?141% +0.1 0.10 ? 26% perf-profile.children.cycles-pp.do_sys_openat2
0.11 ? 33% +0.1 0.20 ? 8% perf-profile.children.cycles-pp.cpumask_next_and
0.07 ? 12% +0.1 0.16 ? 20% perf-profile.children.cycles-pp.exc_page_fault
0.07 ? 12% +0.1 0.16 ? 20% perf-profile.children.cycles-pp.do_user_addr_fault
0.00 +0.1 0.09 ? 35% perf-profile.children.cycles-pp.__get_cpu_features
0.02 ?141% +0.1 0.11 ? 27% perf-profile.children.cycles-pp.__x64_sys_openat
0.00 +0.1 0.09 ? 24% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.00 +0.1 0.09 ? 24% perf-profile.children.cycles-pp.do_group_exit
0.00 +0.1 0.09 ? 24% perf-profile.children.cycles-pp.do_exit
0.05 ? 47% +0.1 0.15 ? 19% perf-profile.children.cycles-pp.vfs_read
0.07 ? 18% +0.1 0.17 ? 18% perf-profile.children.cycles-pp.read
0.00 +0.1 0.10 ? 20% perf-profile.children.cycles-pp.load_elf_binary
0.05 ? 46% +0.1 0.15 ? 18% perf-profile.children.cycles-pp.__x64_sys_execve
0.05 ? 46% +0.1 0.15 ? 18% perf-profile.children.cycles-pp.do_execveat_common
0.05 ? 46% +0.1 0.15 ? 18% perf-profile.children.cycles-pp.execve
0.05 ? 46% +0.1 0.16 ? 16% perf-profile.children.cycles-pp.ksys_read
0.05 ? 76% +0.1 0.15 ? 33% perf-profile.children.cycles-pp.note_gp_changes
0.00 +0.1 0.10 ? 18% perf-profile.children.cycles-pp.exec_binprm
0.00 +0.1 0.10 ? 18% perf-profile.children.cycles-pp.search_binary_handler
0.09 ? 15% +0.1 0.20 ? 19% perf-profile.children.cycles-pp.asm_exc_page_fault
0.00 +0.1 0.12 ? 19% perf-profile.children.cycles-pp.mmput
0.00 +0.1 0.12 ? 19% perf-profile.children.cycles-pp.exit_mmap
0.09 ? 33% +0.1 0.22 ? 24% perf-profile.children.cycles-pp.rcu_core
0.18 ? 9% +0.1 0.32 ? 8% perf-profile.children.cycles-pp.rcu_dynticks_inc
0.07 ? 6% +0.1 0.21 ? 19% perf-profile.children.cycles-pp._dl_catch_error
0.60 ? 9% +0.2 0.75 ? 11% perf-profile.children.cycles-pp._raw_spin_lock
0.19 ? 9% +0.2 0.35 ? 19% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.13 ? 42% +0.2 0.29 ? 22% perf-profile.children.cycles-pp.tick_sched_do_timer
0.17 ? 25% +0.2 0.34 ? 21% perf-profile.children.cycles-pp.error_entry
0.15 ? 37% +0.3 0.41 ? 47% perf-profile.children.cycles-pp._raw_spin_trylock
0.26 ? 17% +0.3 0.53 ? 12% perf-profile.children.cycles-pp.update_sg_lb_stats
0.48 ? 26% +0.3 0.79 ? 11% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.41 ? 23% +0.3 0.74 ? 22% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.64 ? 22% +0.3 0.98 ? 18% perf-profile.children.cycles-pp.rcu_idle_exit
0.42 ? 14% +0.4 0.77 ? 14% perf-profile.children.cycles-pp.update_sd_lb_stats
0.45 ? 13% +0.4 0.83 ? 14% perf-profile.children.cycles-pp.find_busiest_group
0.58 ? 19% +0.4 1.01 ? 17% perf-profile.children.cycles-pp.irqtime_account_irq
0.37 ? 16% +0.4 0.80 ? 18% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.24 ? 7% +0.4 0.69 ? 12% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.32 ? 20% +0.5 0.78 ? 26% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.78 ? 13% +0.5 1.26 ? 6% perf-profile.children.cycles-pp.native_sched_clock
0.65 ? 21% +0.5 1.14 ? 21% perf-profile.children.cycles-pp.load_balance
0.39 ? 41% +0.5 0.92 ? 33% perf-profile.children.cycles-pp.start_kernel
0.93 ? 12% +0.6 1.54 ? 5% perf-profile.children.cycles-pp.sched_clock_cpu
0.76 ? 6% +0.6 1.37 ? 21% perf-profile.children.cycles-pp.read_tsc
0.60 ? 11% +0.8 1.39 ? 31% perf-profile.children.cycles-pp.lapic_next_deadline
0.89 ? 17% +0.8 1.69 ? 12% perf-profile.children.cycles-pp.rebalance_domains
1.02 ? 15% +0.9 1.87 ? 14% perf-profile.children.cycles-pp.tick_nohz_next_event
0.73 ? 10% +0.9 1.64 ? 30% perf-profile.children.cycles-pp.native_irq_return_iret
0.86 ? 18% +1.1 1.92 ? 20% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.96 ? 34% +1.1 2.09 ? 18% perf-profile.children.cycles-pp.tick_irq_enter
0.99 ? 33% +1.2 2.14 ? 17% perf-profile.children.cycles-pp.irq_enter_rcu
1.55 ? 17% +1.3 2.81 ? 9% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.97 ? 25% +1.7 3.66 ? 17% perf-profile.children.cycles-pp.__irq_exit_rcu
1.56 ? 18% +2.4 3.99 ? 41% perf-profile.children.cycles-pp.ktime_get
7.68 ? 32% +7.6 15.26 ? 20% perf-profile.children.cycles-pp.hrtimer_interrupt
7.97 ? 33% +7.7 15.64 ? 21% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
35.41 +11.3 46.71 ? 13% perf-profile.children.cycles-pp.mwait_idle_with_hints
11.89 ? 31% +11.5 23.35 ? 19% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
35.63 +11.5 47.14 ? 13% perf-profile.children.cycles-pp.intel_idle
13.80 ? 27% +13.6 27.37 ? 15% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
52.77 ? 9% +28.2 80.99 ? 2% perf-profile.children.cycles-pp.cpuidle_enter_state
52.87 ? 9% +28.3 81.17 ? 2% perf-profile.children.cycles-pp.cpuidle_enter
59.70 ? 10% +30.9 90.62 ? 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
59.70 ? 10% +30.9 90.62 ? 2% perf-profile.children.cycles-pp.cpu_startup_entry
59.68 ? 10% +30.9 90.61 ? 2% perf-profile.children.cycles-pp.do_idle
57.68 ? 10% +31.6 89.24 ? 3% perf-profile.children.cycles-pp.cpuidle_idle_call
6.92 ? 9% -5.5 1.45 ? 30% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
4.66 ? 23% -3.9 0.78 ? 39% perf-profile.self.cycles-pp.memcpy_erms
0.62 ? 22% -0.5 0.12 ? 29% perf-profile.self.cycles-pp.kmem_cache_free
0.57 ? 13% -0.5 0.12 ? 38% perf-profile.self.cycles-pp.__check_object_size
0.43 ? 18% -0.4 0.05 ? 80% perf-profile.self.cycles-pp.sctp_datamsg_from_user
0.45 ? 19% -0.3 0.12 ? 37% perf-profile.self.cycles-pp.__might_resched
0.36 ? 8% -0.3 0.04 ?114% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.39 ? 17% -0.3 0.06 ? 50% perf-profile.self.cycles-pp.__list_del_entry_valid
0.42 ? 8% -0.3 0.10 ? 23% perf-profile.self.cycles-pp.__slab_free
0.41 ? 10% -0.3 0.10 ? 33% perf-profile.self.cycles-pp.sctp_recvmsg
0.39 ? 21% -0.3 0.08 ? 55% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.37 ? 17% -0.3 0.06 ? 55% perf-profile.self.cycles-pp.ipv4_dst_check
0.33 ? 20% -0.3 0.04 ? 74% perf-profile.self.cycles-pp.kfree
0.31 ? 25% -0.3 0.03 ?103% perf-profile.self.cycles-pp.__copy_skb_header
0.32 ? 19% -0.3 0.06 ? 49% perf-profile.self.cycles-pp.memcg_slab_free_hook
0.32 ? 18% -0.3 0.06 ? 53% perf-profile.self.cycles-pp.sctp_chunk_put
0.30 ? 18% -0.3 0.04 ? 73% perf-profile.self.cycles-pp.memset_erms
0.29 ? 25% -0.2 0.05 ? 82% perf-profile.self.cycles-pp.kmem_cache_alloc
0.28 ? 27% -0.2 0.04 ?102% perf-profile.self.cycles-pp.skb_set_owner_w
0.31 ? 5% -0.2 0.08 ? 23% perf-profile.self.cycles-pp.__schedule
0.28 ? 8% -0.2 0.05 ? 71% perf-profile.self.cycles-pp.dst_release
0.25 ? 25% -0.2 0.03 ?101% perf-profile.self.cycles-pp.sctp_outq_flush_data
0.26 ? 17% -0.2 0.04 ?103% perf-profile.self.cycles-pp.free_pcp_prepare
0.28 ? 14% -0.2 0.06 ? 48% perf-profile.self.cycles-pp.__skb_datagram_iter
0.25 ? 18% -0.2 0.03 ?101% perf-profile.self.cycles-pp.sctp_cmd_interpreter
0.25 ? 22% -0.2 0.04 ? 75% perf-profile.self.cycles-pp.__might_sleep
0.26 ? 9% -0.2 0.06 ? 49% perf-profile.self.cycles-pp.set_next_entity
0.24 ? 8% -0.2 0.04 ? 71% perf-profile.self.cycles-pp.sctp_skb_recv_datagram
0.22 ? 25% -0.2 0.03 ?100% perf-profile.self.cycles-pp.__zone_watermark_ok
0.24 ? 13% -0.2 0.06 ? 76% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.19 ? 17% -0.2 0.02 ? 99% perf-profile.self.cycles-pp._copy_from_user
0.16 ? 13% -0.1 0.03 ?100% perf-profile.self.cycles-pp.__list_add_valid
0.14 ? 26% -0.1 0.02 ? 99% perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook
0.12 ? 19% -0.0 0.07 ? 52% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.02 ?142% +0.1 0.08 ? 12% perf-profile.self.cycles-pp.can_stop_idle_tick
0.08 ? 16% +0.1 0.15 ? 19% perf-profile.self.cycles-pp.hrtimer_next_event_without
0.00 +0.1 0.07 ? 10% perf-profile.self.cycles-pp.irqentry_exit
0.17 ? 15% +0.1 0.25 ? 25% perf-profile.self.cycles-pp.call_cpuidle
0.12 ? 16% +0.1 0.20 ? 26% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.13 ? 15% +0.1 0.21 ? 16% perf-profile.self.cycles-pp.sched_clock_cpu
0.09 ? 54% +0.1 0.17 ? 12% perf-profile.self.cycles-pp._find_next_bit
0.08 ? 25% +0.1 0.16 ? 11% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.1 0.08 ? 20% perf-profile.self.cycles-pp.hrtimer_forward
0.00 +0.1 0.09 ? 18% perf-profile.self.cycles-pp.cpuidle_reflect
0.18 ? 11% +0.1 0.31 ? 9% perf-profile.self.cycles-pp.rcu_dynticks_inc
0.09 ? 64% +0.1 0.22 ? 24% perf-profile.self.cycles-pp.tick_sched_do_timer
0.56 ? 11% +0.1 0.70 ? 11% perf-profile.self.cycles-pp._raw_spin_lock
0.25 ? 20% +0.1 0.39 ? 19% perf-profile.self.cycles-pp.tick_nohz_next_event
0.15 ? 29% +0.2 0.31 ? 24% perf-profile.self.cycles-pp.error_entry
0.24 ? 4% +0.2 0.42 ? 18% perf-profile.self.cycles-pp.rcu_idle_exit
0.17 ? 13% +0.2 0.37 ? 12% perf-profile.self.cycles-pp.update_sg_lb_stats
0.30 ? 30% +0.2 0.51 ? 29% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.22 ? 9% +0.2 0.44 ? 25% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
0.24 ? 15% +0.2 0.46 ? 8% perf-profile.self.cycles-pp.intel_idle
0.13 ? 17% +0.3 0.41 ? 47% perf-profile.self.cycles-pp._raw_spin_trylock
0.46 ? 18% +0.3 0.75 ? 13% perf-profile.self.cycles-pp.cpuidle_idle_call
0.20 ? 27% +0.3 0.53 ? 33% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.36 ? 16% +0.4 0.80 ? 18% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.24 ? 7% +0.4 0.69 ? 12% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.73 ? 12% +0.5 1.20 ? 7% perf-profile.self.cycles-pp.native_sched_clock
0.74 ? 6% +0.6 1.32 ? 21% perf-profile.self.cycles-pp.read_tsc
0.60 ? 11% +0.8 1.38 ? 31% perf-profile.self.cycles-pp.lapic_next_deadline
0.73 ? 10% +0.9 1.64 ? 30% perf-profile.self.cycles-pp.native_irq_return_iret
0.89 ? 33% +2.0 2.85 ? 51% perf-profile.self.cycles-pp.ktime_get
2.60 ? 10% +2.9 5.47 ? 13% perf-profile.self.cycles-pp.cpuidle_enter_state
35.37 +11.3 46.62 ? 13% perf-profile.self.cycles-pp.mwait_idle_with_hints



***************************************************************************************************
lkp-icl-2sp6: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
gcc-11/performance/1HDD/4K/nfsv4/xfs/1x/x86_64-rhel-8.3/1fpd/1t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-icl-2sp6/50M/fsmark/0xd000331

commit:
4033d677de ("Merge branch 'for-5.19/io_uring-socket' into sock-nolock.2")
5aa32dd9bd ("net: add support for fast socket release")

4033d677de27560d 5aa32dd9bdb57e4a26143216df5
---------------- ---------------------------
%stddev %change %stddev
\ | \
10335 -3.1% 10012 proc-vmstat.nr_shmem
96.15 +0.9% 96.98 iostat.cpu.idle
3.40 ? 3% -24.8% 2.56 ? 19% iostat.cpu.system
3.19 ? 3% -0.8 2.42 ? 18% mpstat.cpu.all.irq%
0.03 ? 2% -0.0 0.02 ? 6% mpstat.cpu.all.sys%
0.01 ? 4% -0.0 0.01 ? 4% mpstat.cpu.all.usr%
10.89 ? 34% -77.7% 2.43 ?122% sched_debug.cfs_rq:/.MIN_vruntime.avg
724.26 ? 11% -73.5% 192.06 ?114% sched_debug.cfs_rq:/.MIN_vruntime.max
82.06 ? 16% -76.0% 19.70 ?115% sched_debug.cfs_rq:/.MIN_vruntime.stddev
10.89 ? 34% -77.7% 2.43 ?122% sched_debug.cfs_rq:/.max_vruntime.avg
724.26 ? 11% -73.5% 192.06 ?114% sched_debug.cfs_rq:/.max_vruntime.max
82.06 ? 16% -76.0% 19.70 ?115% sched_debug.cfs_rq:/.max_vruntime.stddev
24.45 ? 2% +9.9% 26.87 ? 3% sched_debug.cfs_rq:/.runnable_avg.avg
24.37 ? 2% +9.9% 26.79 ? 3% sched_debug.cfs_rq:/.util_avg.avg
5.52 ? 3% -1.4 4.08 ? 21% turbostat.Busy%
16.33 ? 47% +55.7 72.07 ? 48% turbostat.C1E%
78.99 ? 9% -54.5 24.53 ?138% turbostat.C6%
19.49 ? 40% +275.3% 73.16 ? 45% turbostat.CPU%c1
74.99 ? 10% -69.7% 22.76 ?141% turbostat.CPU%c6
48.33 ? 3% +7.6% 52.00 ? 3% turbostat.CoreTmp
22.51 ? 19% -70.5% 6.63 ?142% turbostat.Pkg%pc2
10.15 ? 39% -78.4% 2.20 ?193% turbostat.Pkg%pc6
48.67 ? 3% +7.5% 52.33 ? 3% turbostat.PkgTmp
181.48 ? 5% +11.4% 202.20 ? 4% turbostat.PkgWatt
60.39 ? 2% +4.5% 63.11 ? 2% turbostat.RAMWatt
11.29 -6.7 4.56 ?102% perf-stat.i.branch-miss-rate%
23636197 -60.3% 9387443 ?102% perf-stat.i.branch-misses
1.20 -0.8 0.39 ?136% perf-stat.i.dTLB-load-miss-rate%
3477591 -67.4% 1133867 ?136% perf-stat.i.dTLB-load-misses
0.35 -0.2 0.12 ?127% perf-stat.i.dTLB-store-miss-rate%
544620 -65.2% 189537 ?127% perf-stat.i.dTLB-store-misses
11.20 -6.7 4.54 ?102% perf-stat.overall.branch-miss-rate%
1.19 -0.8 0.39 ?136% perf-stat.overall.dTLB-load-miss-rate%
0.35 -0.2 0.12 ?127% perf-stat.overall.dTLB-store-miss-rate%
23620071 -60.3% 9381070 ?102% perf-stat.ps.branch-misses
3475219 -67.4% 1133091 ?136% perf-stat.ps.dTLB-load-misses
544246 -65.2% 189406 ?127% perf-stat.ps.dTLB-store-misses
8.50 ? 8% -2.3 6.19 ? 20% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
95.43 -1.9 93.52 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
95.59 -1.9 93.70 perf-profile.calltrace.cycles-pp.cpu_startup_entry.secondary_startup_64_no_verify
93.89 -1.5 92.37 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.secondary_startup_64_no_verify
5.25 ? 5% -0.8 4.42 ? 14% perf-profile.calltrace.cycles-pp.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
4.22 ? 5% -0.7 3.48 ? 16% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
3.04 -0.6 2.49 ? 15% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
1.27 ? 2% -0.3 0.96 ? 15% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt
1.83 -0.2 1.60 ? 9% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.87 ? 9% -0.2 0.71 ? 10% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu
0.81 ? 11% -0.2 0.66 ? 10% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start
1.88 ? 7% +1.8 3.72 ? 39% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
8.67 ? 8% -2.3 6.34 ? 20% perf-profile.children.cycles-pp.menu_select
5.37 ? 5% -0.8 4.56 ? 13% perf-profile.children.cycles-pp.__irq_exit_rcu
4.37 ? 6% -0.7 3.67 ? 15% perf-profile.children.cycles-pp.__softirqentry_text_start
0.88 -0.4 0.48 ? 57% perf-profile.children.cycles-pp.timerqueue_add
0.77 -0.3 0.42 ? 51% perf-profile.children.cycles-pp.timerqueue_del
0.70 ? 3% -0.3 0.38 ? 42% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
1.39 ? 3% -0.3 1.11 ? 13% perf-profile.children.cycles-pp.load_balance
0.52 ? 7% -0.3 0.25 ? 65% perf-profile.children.cycles-pp.irqentry_enter
1.86 ? 2% -0.2 1.62 ? 9% perf-profile.children.cycles-pp.rebalance_domains
0.41 ? 4% -0.2 0.24 ? 47% perf-profile.children.cycles-pp.rb_insert_color
0.26 ? 5% -0.2 0.10 ? 93% perf-profile.children.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.62 ? 3% -0.2 0.46 ? 21% perf-profile.children.cycles-pp.update_irq_load_avg
0.28 ? 10% -0.2 0.12 ? 84% perf-profile.children.cycles-pp.rcu_nmi_enter
0.22 ? 12% -0.1 0.10 ? 59% perf-profile.children.cycles-pp.pm_qos_read_value
0.25 ? 11% -0.1 0.14 ? 42% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.25 -0.1 0.14 ? 40% perf-profile.children.cycles-pp.get_cpu_device
0.19 ? 22% -0.1 0.08 ? 96% perf-profile.children.cycles-pp.rcu_nmi_exit
0.20 ? 18% -0.1 0.12 ? 50% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.13 ? 10% -0.1 0.05 ?101% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
0.09 ? 13% +0.0 0.14 ? 21% perf-profile.children.cycles-pp.load_elf_binary
0.09 ? 13% +0.1 0.14 ? 21% perf-profile.children.cycles-pp.exec_binprm
0.09 ? 13% +0.1 0.14 ? 21% perf-profile.children.cycles-pp.search_binary_handler
0.07 ? 23% +0.1 0.13 ? 27% perf-profile.children.cycles-pp.exit_mmap
0.26 ? 4% +0.1 0.33 ? 16% perf-profile.children.cycles-pp.schedule
0.28 ? 10% +0.1 0.36 ? 14% perf-profile.children.cycles-pp.rcu_dynticks_inc
0.43 ? 11% +0.1 0.52 ? 10% perf-profile.children.cycles-pp.__schedule
0.07 ? 23% +0.1 0.16 ? 34% perf-profile.children.cycles-pp._IO_str_seekoff
0.16 ? 5% +0.1 0.26 ? 28% perf-profile.children.cycles-pp.do_user_addr_fault
0.28 ? 4% +0.1 0.39 ? 19% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.20 ? 10% +0.1 0.33 ? 28% perf-profile.children.cycles-pp.asm_exc_page_fault
0.09 ? 18% +0.2 0.25 ? 58% perf-profile.children.cycles-pp.devkmsg_write.cold
0.09 ? 18% +0.2 0.25 ? 58% perf-profile.children.cycles-pp.devkmsg_emit
0.61 ? 8% +0.3 0.92 ? 25% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.15 ? 6% +0.5 1.66 ? 20% perf-profile.children.cycles-pp.read_tsc
0.42 ? 14% +0.7 1.10 ? 39% perf-profile.children.cycles-pp.tick_nohz_irq_exit
1.97 ? 8% +1.8 3.81 ? 39% perf-profile.children.cycles-pp.clockevents_program_event
2.22 ? 6% +2.5 4.72 ? 39% perf-profile.children.cycles-pp.ktime_get
4.77 ? 15% -2.1 2.63 ? 41% perf-profile.self.cycles-pp.menu_select
0.51 ? 7% -0.2 0.27 ? 54% perf-profile.self.cycles-pp.timerqueue_add
0.40 ? 4% -0.2 0.22 ? 52% perf-profile.self.cycles-pp.rb_insert_color
0.61 ? 2% -0.2 0.44 ? 23% perf-profile.self.cycles-pp.update_irq_load_avg
0.23 ? 7% -0.2 0.08 ?116% perf-profile.self.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.22 ? 10% -0.1 0.07 ?115% perf-profile.self.cycles-pp.pm_qos_read_value
0.29 ? 28% -0.1 0.14 ? 62% perf-profile.self.cycles-pp.rcu_eqs_enter
0.28 ? 11% -0.1 0.14 ? 43% perf-profile.self.cycles-pp.load_balance
0.23 ? 11% -0.1 0.10 ? 61% perf-profile.self.cycles-pp.irqentry_enter
0.25 ? 10% -0.1 0.13 ? 51% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.25 -0.1 0.13 ? 44% perf-profile.self.cycles-pp.get_cpu_device
0.24 ? 12% -0.1 0.14 ? 40% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt
0.30 ? 10% -0.1 0.20 ? 34% perf-profile.self.cycles-pp.tick_irq_enter
0.22 ? 9% -0.1 0.13 ? 37% perf-profile.self.cycles-pp.cpuidle_governor_latency_req
0.17 ? 25% -0.1 0.08 ? 78% perf-profile.self.cycles-pp.irq_work_needs_cpu
0.22 ? 9% -0.1 0.15 ? 18% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.16 ? 18% -0.1 0.10 ? 54% perf-profile.self.cycles-pp.menu_reflect
0.27 ? 10% +0.1 0.35 ? 15% perf-profile.self.cycles-pp.rcu_dynticks_inc
0.59 ? 8% +0.3 0.88 ? 25% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.12 ? 6% +0.5 1.60 ? 20% perf-profile.self.cycles-pp.read_tsc
5.11 ? 2% +1.2 6.35 ? 14% perf-profile.self.cycles-pp.cpuidle_enter_state
1.24 ? 11% +2.1 3.38 ? 50% perf-profile.self.cycles-pp.ktime_get





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


--
0-DAY CI Kernel Test Service
https://01.org/lkp



Attachments:
(No filename) (139.72 kB)
config-5.18.0-rc4-00071-g5aa32dd9bdb5 (165.11 kB)
job-script (8.44 kB)
job.yaml (5.79 kB)
reproduce (3.54 kB)
Download all attachments