LinuxLists.cc - [PATCH v2] fs/dax: Fix pmd vs pte conflict detection

2019-10-21 16:47:13

Subject: [PATCH v2] fs/dax: Fix pmd vs pte conflict detection

Users reported a v5.3 performance regression and inability to establish
huge page mappings. A revised version of the ndctl "dax.sh" huge page
unit test identifies commit 23c84eb78375 "dax: Fix missed wakeup with
PMD faults" as the source.

Update get_unlocked_entry() to check for NULL entries before checking
the entry order, otherwise NULL is misinterpreted as a present pte
conflict. The 'order' check needs to happen before the locked check as
an unlocked entry at the wrong order must fallback to lookup the correct
order.

Reported-by: Jeff Smits <[email protected]>
Reported-by: Doug Nelson <[email protected]>
Cc: <[email protected]>
Fixes: 23c84eb78375 ("dax: Fix missed wakeup with PMD faults")
Reviewed-by: Jan Kara <[email protected]>
Cc: Jeff Moyer <[email protected]>
Cc: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Dan Williams <[email protected]>
---
Changes in v2:
- Update the changelog to reflect the user visible effects of the bug
(Jeff)

fs/dax.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/dax.c b/fs/dax.c
index 6bf81f931de3..2cc43cd914eb 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -220,10 +220,11 @@ static void *get_unlocked_entry(struct xa_state *xas, unsigned int order)

for (;;) {
entry = xas_find_conflict(xas);
+ if (!entry || WARN_ON_ONCE(!xa_is_value(entry)))
+ return entry;
if (dax_entry_order(entry) < order)
return XA_RETRY_ENTRY;
- if (!entry || WARN_ON_ONCE(!xa_is_value(entry)) ||
- !dax_is_locked(entry))
+ if (!dax_is_locked(entry))
return entry;

wq = dax_entry_waitqueue(xas, entry, &ewait.key);

2019-10-22 07:27:57

by Johannes Thumshirn

[permalink] [raw]

Subject: Re: [PATCH v2] fs/dax: Fix pmd vs pte conflict detection

Looks good,
Reviewed-by: Johannes Thumshirn <[email protected]>

--
Johannes Thumshirn SUSE Labs Filesystems
[email protected] +49 911 74053 689
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5
90409 Nürnberg
Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

2019-10-27 05:19:06

by Chen, Rong A

[permalink] [raw]

Subject: [fs/dax] a70e8083a9: fio.read_bw_MBps 523.2% improvement

Greeting,

FYI, we

commit: url:
in testcase: fio-basic
on test with following parameters:

disk: 2pmem
fs: ext4
mount_option: dax
runtime: 200s
nr_task: 50%
time_based: tb
rw: rw
bs: 4k
ioengine: mmap
test_size: 200G
ucode: 0x5000021
cpufreq_governor: performance

test-description: test-url: In addition
+---------------- | testcase: | test machine | test parameters | bs=2M | | | | | | | | | | | +----------------

Details are as below:
-----------------

To reproduce:

git clone bin/lkp run job.yaml

================= bs/compiler/cpufr 4k/gcc-7/perfor
commit:
v5.4-rc4
a70e8083a9
---------------- fail:runs | 1:5 %stddev \ 0.04 ±103% 0.40 ± 17% 42.23 ± 3% 0.04 ± 48% 0.03 ± 36% 26.56 ± 5% 6660 19072 19993 23475 7134 11158 ± 17% 1704992 227.61 227.61 4.724e+08 4224 8989 626.08 ± 2% 23802 6.821e+08 6662 23526 25779 32665 ± 6% 20398 9815 ± 15% 1705684 40917 ± 70% 41.15 2.87 ± 2% 56.36 40.80 2.85 ± 2% 56.00 2.00 2143726 1156106 ± 19% 1162440 ± 17% 6344 ±193% 916775 ± 23% 941710 ± 21% 498777 7644983 1178141 498777 681978 40679 1246 44.75 56.00 56.20 245.28 128.78 4275 ± 14% 142255 ± 5% 52355 ± 21% 1367388 ± 23% 1360094 ± 23% 150783 ± 7% 72319 ± 16% 738.20 ± 8% 1699 ± 8% 1699 ± 8% 760144 13576 760334 13576 560.40 ± 7% 209548 ± 21% 17189 ± 13% 3707244 ± 4% 572232 ± 5% 209548 ± 21% 308536 ± 16% 289299 ± 15% 3939263 ± 3% 606602 ± 7% 289299 ± 15% 373513 ± 13% 5108 64785 86.00 ± 13% 1028576 2059669 10368676 95519 84.00 ± 2% 96766 292164 124646 45797 5108 95519 84.00 ± 2% 2127714 2096428 3112 ± 7% 2268500 4.731e+08 2172870 666825 129.10 ± 4% 792591 343702 ± 3% 1.00 0.40 ± 2% 791828 343573 ± 3% 549.67 ± 4% 597408 365969 2.73 ± 4% 2.73 ± 4% 1.00 19380 ± 22% 14002 ± 23% 2363 ± 15% 6961 ± 23% 23.40 ± 3% 1191 ± 14% 51.70 ± 4% 441.41 30.50 ± 4% 193.20 ± 2% 0.00 24.20 ±183% 22.00 ±130% 162.60 ±113% 10.40 ±143% 4.40 ± 51% 84.00 ±180% 81.80 ±172% 2.40 ± 62% 3.20 ± 93% 2840 ± 12% 4.00 ± 54% +14525.0% 2.20 ± 66% 2.40 ± 97% 3.20 ± 66% 2.20 ±109% 0.00 11.60 ± 82% 6.20 ± 73% 9.20 ± 67% 13.00 ± 27% 7.60 ± 62% 14.80 ±141% 8.20 ± 56% 9.40 ± 63% 14.80 ± 83% 5.60 ± 96% 8.20 ± 64% 10.20 ± 50% 36.60 ± 50% 8.60 ±136% 3.60 ±132% 7.80 ± 98% 17.40 ± 80% 193.20 ± 2% 14.71 ± 3% 5.704e+09 0.45 ± 8% 23774744 ± 2% 54.74 2.382e+08 ± 3% 4.015e+08 ± 2% 4.39 1.195e+11 896.52 ± 7% 0.05 ± 12% 3096297 6.8e+09 0.24 ± 11% 7425754 ± 11% 2.807e+09 68.54 12855481 ± 3% 2.552e+10 2195 ± 3% 0.27 ± 4% 2077653 31404609 ± 27% 32711606 ± 20% 26979202 ± 24% 25469597 ± 22% 2077670 15.73 ± 2% 0.42 ± 2% 59.33 4.68 501.88 ± 2% 0.05 0.26 ± 11% 70.67 1988 ± 4% 0.21 8446 5.679e+09 23670395 ± 2% 2.372e+08 ± 3% 3.998e+08 ± 2% 1.189e+11 3082720 6.77e+09 7393213 ± 11% 2.795e+09 12799067 ± 3% 2.541e+10 2068549 31266968 ± 27% 32567992 ± 20% 26860938 ± 24% 25357836 ± 22% 2068565 5.762e+12 7535 ± 26% 6126 ± 2% 6381 ± 7% 6087 ± 5% 6287 ± 2% 7342 ± 23% 5947 ± 3% 6407 ± 11% 6614 ± 6% 7221 ± 7% 6926 ± 9% 6566 ± 4% 7679 ± 28% 6611 ± 3% 6726 ± 4% 6571 ± 4% 7565 ± 19% 8353 ± 19% 6097 ± 7% 5926 ± 7% 6096 ± 7% 5783 ± 10% 5998 ± 7% 10929 ± 12% 6860 ± 36% 5811 ± 7% 6107 ± 5% 6072 ± 6% 6014 ± 3% 6002 ± 4% 6323 ± 9% 5893 ± 2% 5894 ± 3% 6129 ± 3% 6155 ± 9% 6324 ± 11% 5909 ± 4% 6131 ± 7% 5907 ± 2% 6020 ± 7% 5859 ± 4% 5877 ± 5% 5826 ± 3% 7221 ± 27% 20592 ± 38% 6200 ± 8% 6992 ± 17% 5929 ± 6% 6678 ± 8% 6522 ± 12% 5935 ± 8% 5873 ± 8% 5938 ± 6% 5727 ± 8% 5878 ± 7% 5787 ± 6% 5925 ± 6% 6262 ± 4% 6107 ± 8% 6101 ± 13% 6414 ± 16% 5835 ± 7% 6121 ± 7% 6790 ± 29% 6139 ± 8% 6002 ± 5% 5766 ± 8% 5780 ± 7% 24983 ± 37% 6216 ± 8% 5838 ± 5% 6131 ± 4% 5992 ± 2% 6138 ± 5% 5888 ± 3% 5961 ± 4% 6367 ± 17% 6033 ± 7% 6364 ± 6% 5892 ± 4% 6732 ± 10% 5826 ± 3% 5906 ± 4% 6215 ± 10% 5892 ± 3% 5738 ± 7% 6017 ± 12% 5960 ± 5% 6085 ± 12% 5808 ± 6% 5887 ± 5% 5755 ± 7% 5865 ± 6% 5962 ± 5% 5914 ± 5% 5940 ± 9% 5715 ± 3% 5820 ± 4% 600723 ± 3% 1744186 ± 3% 66.51 ± 5% 66.47 ± 5% 66.41 ± 5% 66.15 ± 5% 66.02 ± 5% 57.80 ± 5% 55.85 ± 5% 55.26 ± 5% 46.70 ± 6% 46.68 ± 6% 43.98 ± 6% 41.96 ± 6% 41.55 ± 6% 19.15 ± 5% 19.17 ± 5% 17.32 ± 5% 16.42 ± 5% 16.26 ± 5% 0.00 0.00 0.36 ± 81% 0.36 ± 81% 0.37 ± 81% 0.37 ± 81% 0.37 ± 81% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 30.84 ± 13% 30.84 ± 13% 30.98 ± 12% 30.92 ± 13% 30.92 ± 13% 30.92 ± 13% 31.29 ± 12% 66.53 ± 5% 66.42 ± 5% 66.47 ± 5% 66.16 ± 5% 66.03 ± 5% 65.85 ± 5% 61.30 ± 5% 58.37 ± 5% 57.80 ± 5% 57.80 ± 5% 56.37 ± 5% 55.28 ± 5% 46.70 ± 6% 19.17 ± 5% 2.76 ± 13% 2.67 ± 13% 2.24 ± 14% 1.71 ± 9% 1.32 ± 11% 1.21 ± 11% 1.73 ± 3% 1.70 ± 3% 1.71 ± 3% 1.07 ± 15% 0.79 ± 14% 0.77 ± 15% 0.55 ± 3% 0.27 ± 2% 0.20 ± 6% 0.13 ± 3% 0.10 ± 3% 0.13 ± 5% 0.13 ± 8% 0.11 ± 4% 0.12 ± 6% 0.08 ± 5% 0.16 ± 6% 0.07 ± 5% 0.05 ± 9% 0.01 ±200% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 ±200% 0.01 ±200% 0.00 0.01 ±200% 0.01 ±200% 0.00 0.00 0.00 0.00 0.00 0.08 ± 19% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.32 ± 17% 0.00 0.00 0.38 ± 18% 0.01 ±200% 0.00 0.00 0.00 0.02 ±122% 0.02 ±122% 0.05 0.02 ±123% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.10 ± 17% 0.03 ±123% 0.04 ± 51% 0.00 0.00 0.00 0.06 ± 59% 0.04 ± 84% 0.00 0.03 ± 82% 0.07 ± 32% 0.07 ± 32% 0.07 ± 32% 0.00 0.00 0.05 ± 84% 0.07 ± 15% 0.00 0.00 0.09 ± 17% 0.00 0.09 ± 21% 0.09 ± 21% 0.01 ±200% 0.16 ± 17% 0.16 ± 17% 0.37 ± 81% 0.19 ± 15% 0.00 0.06 ± 14% 0.24 ± 14% 0.07 ± 6% 0.08 ± 9% 0.06 ± 12% 0.45 ± 20% 0.00 0.53 ± 26% 0.58 ± 25% 0.27 ± 9% 0.16 ± 7% 0.26 ± 4% 0.00 0.09 ± 4% 30.98 ± 12% 30.92 ± 13% 31.21 ± 12% 31.21 ± 12% 31.29 ± 12% 31.29 ± 12% 31.29 ± 12% 54.97 ± 5% 1.38 ± 16% 1.70 ± 9% 1.18 ± 11% 1.19 ± 3% 1.09 ± 3% 0.73 ± 15% 0.27 ± 2% 0.15 ± 20% 0.20 ± 6% 0.10 ± 3% 0.08 0.04 ± 50% 0.00 0.00 0.00 0.00 0.00 0.00 0.01 ±200% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.04 ± 50% 0.04 ± 51% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 ±122% 0.00 0.07 ± 6% 0.08 ± 9% 0.06 ± 12% 0.27 ± 9% 0.15 ± 7% 0.25 ± 6% 0.00 0.09 ± 8% 30.98 ± 12%

50000 | O 45000 +-+ O 40000 O-+ O O | 35000 +-+ 30000 +-+ | 25000 +-+ 20000 +-+ | 15000 +-+ 10000 +-+ |..+..+..+. .+..+.. 5000 1.2e+07 1.1e+07 +-+ O O O O 1e+07 +-+ 9e+06 +-+ 8e+06 +-+ 7e+06 +-+ | 6e+06 +-+ 5e+06 +-+ 4e+06 +-+ 3e+06 +-+ | 2e+06 1e+06 8000 +-+------- | 7000 +-++..+..+..+.+..+. | 6000 +-+ | 5000 +-+ | 4000 +-+ | 3000 +-+ | 2000 O-+ O O O O | O 1000 +-+------- 16000 | + 14000 +-+ : : 12000 +-+ : : | : : 10000 +-++.. : | + 8000 +-+ | 6000 +-+ 4000 +-+ | 2000 O-+ | O O 0 20000 18000 +-+ 16000 +-+ | 14000 +-+ 12000 +-+ | 10000 +-+ 8000 +-+ | 6000 +-+ 4000 +-+ O 2000 22000 20000 +-++..+.. | 18000 +-+ 16000 +-+ | 14000 +-+ 12000 +-+ 10000 +-+ | 8000 +-+ 6000 +-+ | 4000 O-+ 2000 24000 22000 +-+ | 20000 +-+ 18000 +-+ 16000 +-+ 14000 +-+ | 12000 +-+ 10000 +-+ 8000 +-+ 6000 +-+ O 4000 +-+O 2000 50000 | O 45000 +-+ O 40000 O-+ O O | 35000 +-+ 30000 +-+ | 25000 +-+ 20000 +-+ | 15000 +-+ 10000 +-+ |..+..+..+. .+..+.. 5000 1.2e+07 1.1e+07 +-+ O O O O 1e+07 +-+ 9e+06 +-+ 8e+06 +-+ 7e+06 +-+ | 6e+06 +-+ 5e+06 +-+ 4e+06 +-+ 3e+06 +-+ | 2e+06 1e+06 22000 20000 | 18000 +-+ 16000 +-+ 14000 +-+ 12000 +-+ | 10000 +-+ 8000 +-+ 6000 +-+ 4000 +-+ | 2000 O-+O 0 25000 | 20000 +-+ | | 15000 +-+ | 10000 +-+ | | 5000 +-+ | O O O O O O 0 30000 | 25000 | | 20000 +-+ | 15000 +-+ | 10000 +-+ | | 5000 +-+ O O O O O O 0 40000 | 35000 +-+ + 30000 +-++..+..+ | 25000 +-+ | 20000 +-+ | 15000 +-+ 10000 +-+ | 5000 O-+O | 0 70 +-+O-------- | O 60 +-+ O O O 50 +-+ | 40 +-+ | 30 +-+ | 20 +-+ | 10 +-+ | 0 +-+--------- 50 +-+--------- 45 +-+ |..+..+..+.. .+. 40 +-+ 35 +-+ | 30 +-+ 25 +-+ 20 +-+ | 15 +-+ 10 +-+ | 5 +-+ 0 O-+O--O--O-- 35 +-+--------- | 30 +-+ |.. .. 25 +-++..+..+ | 20 +-+ | 15 +-+ | 10 +-+ | 5 +-+ | 0 O-+O--O--O-- 5e+09 | O 4.5e+09 +-+ O 4e+09 O-+ | 3.5e+09 +-+ 3e+09 +-+ | 2.5e+09 +-+ 2e+09 +-+ | 1.5e+09 +-+ 1e+09 +-+ 5e+08 10000 9000 O-+O | 8000 +-+ 7000 +-+ | 6000 +-+ 5000 +-+ 4000 +-+ | 3000 +-+ 2000 +-+ | 1000 0 10000 9000 +-++..+..+.+..+..+.. | 8000 +-+ 7000 +-+ | 6000 +-+ 5000 +-+ 4000 +-+ | 3000 +-+ 2000 +-+ | 1000 +-+ 0 4600 +-+------- O 4550 +-+ 4500 +-+ | 4450 +-+ | 4400 +-+ | 4350 +-+ 4300 +-+ | 4250 +-+ |..+..+..+..+.+ 4200 +-+------- 228 +-+-------- | +. 226 +-+ 224 +-+ | 222 +-+ 220 +-+ | 218 +-+ 216 +-+ | 214 +-+ 212 +-+ | 210 O-+O--O--O- 228 +-+-------- | +. 226 +-+ 224 +-+ | 222 +-+ 220 +-+ | 218 +-+ 216 +-+ | 214 +-+ 212 +-+ | 210 O-+O--O--O- 5e+08 4.5e+08 +-+ 4e+08 +-+ | 3.5e+08 +-+ 3e+08 +-+ | 2.5e+08 +-+ 2e+08 +-+ | 1.5e+08 +-+ 1e+08 +-+ 5e+07 24000 23500 +-+ | | 23000 +-+ | 22500 +-+ | 22000 +-+ | | O 21500 O-+ O O | 21000 [*] bisect-good sample
[O] bisect-bad sample

***************** lkp-csl-2sp6: ================= bs/compiler/cpufr 2M/gcc-7/perfor
commit:
v5.4-rc4
a70e8083a9
---------------- fail:runs | 1:5 %stddev \ 22.58 ± 15% 77.40 ± 4% 0.01 ± 81% 0.01 ± 50% 227.24 227.24 4.65e+08 4235 9021 603.69 ± 2% 25150 ± 2% 907948 9079 11180441 11285299 11652300 10493648 755702 ± 11% 4539 23285 ± 5% 41.24 2.76 ± 2% 56.37 40.89 2.74 ± 2% 1179347 ± 15% 1204191 ± 13% 1384986 ± 12% 1391284 ± 11% 56.40 2.00 2102090 1602 504545 10737953 1591563 504545 685303 57985 1244 44.66 54.06 ± 2% 54.80 56.40 ± 2% 247.09 137.84 2176 ± 12% 2176 ± 12% 1818 ± 5% 1818 ± 5% 15953 ± 2% 16088 ± 2% 789.20 ± 5% 768474 13726 768688 13726 310305 ± 18% 17976285 ± 6% 843639 ± 4% 310305 ± 18% 408879 ± 14% 80122 ±172% 79981 ±173% 193866 ± 29% 84665 ±166% 749499 ± 4% 193866 ± 29% 276049 ± 21% 4495963 ± 6% 210021 ± 4% 77599 ± 18% 1659023 ± 8% 1633793 ± 8% 19906 ±173% 21108 ±166% 186480 ± 4% 48450 ± 29% 19906 ±173% 1299939 ± 11% 1137570 ± 11% 162369 ± 7% 777339 5072 742845 1387 951491 1905310 9596723 95152 96870 396937 ± 2% 125996 777339 5072 95152 2621128 2589980 19962 ± 21% 56091 ± 24% 3543684 4.656e+08 3468918 19962 ± 21% 907948 58.57 ± 16% 5269 ± 9% 551.52 ± 11% 129.89 ± 7% 58.59 ± 16% 5271 ± 9% 551.73 ± 11% 0.02 ± 13% 545.21 ± 3% 602620 47781 ± 17% 366457 2.94 ± 4% 2.94 ± 4% 0.00 ± 4% 16092 ± 17% 83.25 ± 8% 711.37 ± 2% 8020 ± 17% 22.80 ± 2% 8084 ± 19% 50.55 ± 7% 1295 ± 9% 448.69 ± 2% 30.35 ± 6% 945.06 ± 20% 5.20 ± 80% 2.20 ± 72% 95.60 ± 85% 0.80 ± 93% +31775.0% 1.20 ± 81% +56733.3% 2.00 ± 44% 4.40 ± 71% +15718.2% 7.60 ±121% 32.00 ±181% 1.20 ± 97% +45858.3% 15.40 ± 80% 4.60 ±168% 7.60 ±104% 4.60 ±168% +10590.2% 8.40 ±194% 7.60 ±193% 3.80 ±187% 2530 3.40 ±171% 2.80 ±165% 5.60 ±200% 2.80 ±182% 2.80 ±182% 1.80 ±123% 2540 ± 35% 2540 ± 35% 7.80 ±112% 2918 ± 31% 2918 ± 31% 5.00 ±111% 2499 ± 33% 2499 ± 33% 2930 ± 30% 2930 ± 30% 2864 ± 31% 2864 ± 31% 2880 ± 33% 2880 ± 33% 6.60 ±118% 2516 ± 34% 2516 ± 34% 19.60 ± 99% 45.80 ± 57% 2480 ± 35% 2480 ± 35% 2745 ± 25% 2745 ± 25% 15.80 ± 62% 2496 ± 34% 2496 ± 34% 14.60 ± 61% 46.20 ± 52% 2512 ± 34% 2512 ± 34% 20.40 ± 67% 506.40 ± 27% 26.37 ± 5% 4.044e+09 ± 4% 0.36 ± 5% 13139077 ± 2% 40.81 ± 5% 2.185e+08 ± 6% 5.04e+08 ± 4% 1545 6.27 ± 3% 1.19e+11 890.35 ± 10% 2315604 ± 3% 4.693e+09 ± 3% 2.76 ± 3% 56407157 ± 3% 1.824e+09 75.83 11955091 3255737 1.772e+10 ± 3% 1717 ± 5% 0.22 ± 2% 2041764 46.19 ± 7% 12550181 ± 15% 17032624 ± 3% 39521035 ± 10% 22609084 ± 19% 2041781 28.49 ± 6% 0.33 ± 2% 6.72 ± 3% 547.41 ± 7% 0.05 ± 2% 3.00 ± 3% 78.60 1482 ± 4% 0.15 ± 3% 42.15 ± 8% 4422813 ± 3% 4.027e+09 ± 4% 13081765 ± 2% 2.175e+08 ± 6% 5.018e+08 ± 4% 1538 1.185e+11 2305544 ± 3% 4.673e+09 ± 3% 56162654 ± 3% 1.816e+09 11903152 3241524 1.764e+10 ± 3% 2032916 12495695 ± 15% 16958923 ± 3% 39349478 ± 10% 22511418 ± 19% 2032932 4.015e+12 ± 3% 7276 ± 14% 7428 ± 27% 90558 ± 18% 7372 ± 31% 6380 ± 5% 26925 ± 21% 95764 ± 17% 6839 ± 21% 6195 ± 4% 6379 ± 7% 6234 ± 6% 6744 ± 4% 96371 ± 16% 6560 ± 3% 6456 ± 2% 21824 ± 40% 7235 ± 23% 7523 ± 27% 7973 ± 26% 6831 ± 4% 24201 ± 37% 6985 ± 8% 6729 ± 9% 7647 ± 7% 6613 ± 8% 6228 ± 5% 6370 ± 4% 5960 ± 5% 6476 ± 8% 8136 ± 21% 6270 ± 10% 6267 ± 5% 5940 ± 2% 6015 ± 2% 5833 ± 3% 7405 ± 32% 5908 ± 5% 6216 ± 2% 5960 5806 ± 5% 6711 ± 8% 5746 ± 4% 5945 ± 3% 5935 ± 2% 5880 5908 5996 ± 2% 6450 ± 7% 5566 ± 12% 6942 ± 12% 6225 ± 4% 6666 ± 21% 25268 ± 38% 6271 ± 8% 5995 ± 7% 7059 ± 31% 6309 ± 6% 6369 ± 3% 6443 ± 13% 6359 ± 14% 6309 ± 5% 6342 ± 11% 6249 ± 6% 6260 ± 10% 6397 ± 15% 6411 ± 9% 6225 ± 7% 6169 ± 7% 6528 ± 7% 6488 ± 12% 6408 ± 7% 6431 ± 5% 6289 ± 7% 6651 ± 9% 6370 ± 13% 27083 ± 16% 106068 ± 14% 6287 ± 7% 7080 ± 15% 6229 ± 13% 5633 ± 6% 5720 ± 8% 5614 ± 7% 5672 ± 4% 5717 ± 6% 5741 ± 5% 5634 ± 6% 6579 ± 7% 5515 ± 5% 29701 ± 5% 6523 ± 25% 5764 ± 3% 5590 ± 4% 5667 ± 5% 5880 ± 9% 5775 ± 3% 6126 ± 10% 5728 ± 3% 5617 ± 2% 6054 ± 7% 29041 ± 5% 5813 ± 5% 6071 ± 12% 6124 ± 5% 5891 ± 8% 6384 ± 26% 5795 ± 9% 606775 ± 3% 1736128 ± 3% 64.54 ± 7% 64.51 ± 7% 64.68 ± 7% 64.79 ± 7% 65.11 ± 7% 65.02 ± 7% 65.07 ± 7% 54.60 ± 10% 49.90 ± 12% 49.36 ± 12% 49.36 ± 12% 47.65 ± 12% 47.09 ± 12% 6.22 ± 21% 6.11 ± 21% 0.70 ± 24% 0.70 ± 24% 0.70 ± 24% 0.70 ± 24% 0.70 ± 24% 0.00 0.00 0.00 0.00 0.00 0.10 ±200% 33.25 ± 15% 32.74 ± 15% 32.75 ± 15% 32.84 ± 15% 32.84 ± 15% 32.84 ± 15% 33.54 ± 15% 64.54 ± 7% 64.54 ± 7% 64.68 ± 7% 64.80 ± 7% 65.02 ± 7% 65.13 ± 7% 65.08 ± 7% 54.61 ± 10% 49.90 ± 12% 49.36 ± 12% 49.36 ± 12% 48.14 ± 12% 47.11 ± 12% 6.40 ± 20% 6.52 ± 19% 4.07 ± 21% 3.11 ± 17% 2.94 ± 17% 2.65 ± 16% 1.90 ± 23% 1.78 ± 20% 1.75 ± 20% 1.52 ± 19% 1.51 ± 8% 1.53 ± 8% 1.53 ± 8% 0.14 ± 7% 0.11 ± 6% 0.09 ± 9% 0.15 ± 13% 0.07 ± 5% 0.09 ± 11% 0.07 ± 10% 0.02 ±123% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±122% 0.02 ±123% 0.02 ±123% 0.00 0.00 0.00 0.00 0.01 ±200% 0.00 0.00 0.00 0.16 ± 21% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 ±200% 0.00 0.00 0.00 0.00 0.00 0.10 ± 7% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.07 ± 9% 0.00 0.05 0.00 0.00 0.00 0.00 0.42 ± 9% 0.00 0.00 0.11 ± 10% 0.00 0.00 0.14 ± 9% 0.05 ± 7% 0.05 ± 52% 0.05 ± 52% 0.05 ± 52% 0.05 ± 55% 0.02 ±122% 0.02 ±123% 0.05 ± 52% 0.02 ±122% 0.04 ± 83% 0.05 ± 55% 0.05 ± 52% 0.00 0.00 0.01 ±200% 0.08 ± 18% 0.01 ±200% 0.09 ± 16% 0.09 ± 16% 0.09 ± 16% 0.61 ± 10% 0.45 ± 9% 0.13 ± 11% 0.08 ± 23% 0.63 ± 10% 0.57 ± 10% 0.00 0.10 ± 27% 0.06 ± 54% 0.70 ± 24% 0.19 ± 36% 0.11 ± 11% 0.19 ± 36% 0.11 ± 11% 0.22 ± 32% 0.26 ± 29% 0.00 0.42 ± 23% 0.52 ± 26% 0.56 ± 25% 0.46 ± 8% 33.24 ± 15% 33.45 ± 15% 33.45 ± 15% 32.84 ± 15% 33.54 ± 15% 33.54 ± 15% 33.55 ± 15% 46.84 ± 12% 3.44 ± 21% 2.89 ± 17% 2.63 ± 16% 1.70 ± 20% 1.52 ± 20% 1.02 ± 6% 1.06 ± 9% 0.08 ± 17% 0.15 ± 11% 0.06 ± 7% 0.00 0.00 0.04 ± 50% 0.06 ± 8% 0.00 0.00 0.00 0.00 0.05 0.42 ± 9% 0.00 0.00 0.45 ± 9% 33.24 ± 15%

Disclaimer:
Results for informational design or

Thanks,
Rong Chen

noticed a 523.2% improvement of fio.read_bw_MBps due to commit:
a70e8083a91b17a7c77012f7746dbf29b5050e66 ("[PATCH v2] fs/dax: Fix pmd vs pte conflict detection")
href="https://github.com/0day-ci/linux/commits/Dan-Williams/fs-dax-Fix-pmd-vs-pte-conflict-detection/20191022-045925">https://github.com/0day-ci/linux/commits/Dan-Williams/fs-dax-Fix-pmd-vs-pte-conflict-detection/20191022-045925
machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
href="https://github.com/axboe/fio">https://github.com/axboe/fio
to that, the commit also has significant impact on the following tests:
--+----------------------------------------------------------------------+
change | fio-basic: fio.write_bw_MBps 363.7% improvement |
| 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
|
| cpufreq_governor=performance |
| disk=2pmem |
| fs=ext4 |
| ioengine=mmap |
| mount_option=dax |
| nr_task=50% |
| runtime=200s |
| rw=write |
| test_size=200G |
| time_based=tb |
| ucode=0x5000021 |
--+----------------------------------------------------------------------+
--------------------------------------------------------------------------------->
href="https://github.com/intel/lkp-tests.git">https://github.com/intel/lkp-tests.git
bin/lkp install job.yaml # job file is attached in this email
========================================================================
eq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
mance/2pmem/ext4/mmap/x86_64-rhel-7.6/dax/50%/debian-x86_64-2019-05-14.cgz/200s/rw/lkp-csl-2sp6/200G/fio-basic/tb/0x5000021
("fs/dax: Fix pmd vs pte conflict detection")
v5.4-rc4 a70e8083a91b17a7c77012f7746
---------------------------
%reproduction fail:runs
| |
-19% 0:4 perf-profile.children.cycles-pp.error_entry
%change %stddev
| \
-0.0 0.01 fio.latency_100us%
+0.7 1.09 ± 11% fio.latency_10us%
-42.0 0.24 ± 30% fio.latency_20us%
+58.8 58.84 ± 7% fio.latency_2us%
+25.4 25.47 ± 21% fio.latency_4us%
-26.3 0.24 ± 13% fio.latency_50us%
+523.2% 41507 ± 4% fio.read_bw_MBps
-86.6% 2560 ± 4% fio.read_clat_90%_us
-85.8% 2840 ± 4% fio.read_clat_95%_us
-79.6% 4784 ± 4% fio.read_clat_99%_us
-73.5% 1892 ± 5% fio.read_clat_mean_us
-84.4% 1738 ± 6% fio.read_clat_stddev
+523.2% 10625819 ± 4% fio.read_iops
-7.6% 210.23 fio.time.elapsed_time
-7.6% 210.23 fio.time.elapsed_time.max
-85.0% 70736715 ± 3% fio.time.minor_page_faults
+8.3% 4573 fio.time.percent_of_cpu_this_job_got
-96.8% 288.96 ± 4% fio.time.system_time
+1389.9% 9327 fio.time.user_time
-9.7% 21483 fio.time.voluntary_context_switches
+523.1% 4.251e+09 ± 4% fio.workload
+523.1% 41513 ± 4% fio.write_bw_MBps
-90.9% 2150 ± 3% fio.write_clat_90%_us
-90.5% 2448 ± 4% fio.write_clat_95%_us
-83.9% 5248 ± 5% fio.write_clat_99%_us
-92.1% 1612 ± 3% fio.write_clat_mean_us
-76.4% 2315 ± 8% fio.write_clat_stddev
+523.1% 10627503 ± 4% fio.write_iops
-77.1% 9387 ± 57% cpuidle.POLL.usage
-39.5 1.63 ± 4% mpstat.cpu.all.sys%
+43.9 46.79 mpstat.cpu.all.usr%
-7.7% 52.03 iostat.cpu.idle
-96.0% 1.62 ± 4% iostat.cpu.system
+1528.8% 46.35 iostat.cpu.user
-7.1% 52.00 vmstat.cpu.id
+2200.0% 46.00 vmstat.cpu.us
-19.7% 1720555 vmstat.memory.cache
-65.7% 396915 ± 49% numa-numastat.node0.local_node
-64.5% 412613 ± 44% numa-numastat.node0.numa_hit
+147.5% 15702 ± 98% numa-numastat.node0.other_node
-55.2% 410953 ± 48% numa-numastat.node1.local_node
-54.7% 426566 ± 43% numa-numastat.node1.numa_hit
-80.2% 98859 meminfo.KReclaimable
-18.7% 6215641 meminfo.Memused
-87.3% 149626 ± 3% meminfo.PageTables
-80.2% 98859 meminfo.SReclaimable
-58.1% 285710 meminfo.Slab
-23.8% 30984 meminfo.max_used_kB
+10.7% 1379 turbostat.Avg_MHz
+4.7 49.43 turbostat.Busy%
+11.2% 62.25 turbostat.CoreTmp
+10.3% 62.00 turbostat.PkgTmp
+18.0% 289.40 turbostat.PkgWatt
+45.3% 187.11 turbostat.RAMWatt
+1135.7% 52830 ± 90% numa-vmstat.node0.nr_mapped
-86.4% 19293 ± 49% numa-vmstat.node0.nr_page_table_pages
-74.2% 13489 ± 13% numa-vmstat.node0.nr_slab_reclaimable
-32.4% 924713 ± 22% numa-vmstat.node0.numa_hit
-33.2% 908947 ± 22% numa-vmstat.node0.numa_local
-88.0% 18133 ± 58% numa-vmstat.node1.nr_page_table_pages
-84.5% 11218 ± 17% numa-vmstat.node1.nr_slab_reclaimable
+13.8% 840.00 ± 5% slabinfo.blkdev_ioc.num_objs
-44.5% 942.75 ± 8% slabinfo.dquot.active_objs
-44.5% 942.75 ± 8% slabinfo.dquot.num_objs
-92.4% 57717 slabinfo.radix_tree_node.active_objs
-92.4% 1031 slabinfo.radix_tree_node.active_slabs
-92.4% 57771 slabinfo.radix_tree_node.num_objs
-92.4% 1031 slabinfo.radix_tree_node.num_slabs
+14.2% 640.25 ± 3% slabinfo.skbuff_ext_cache.active_objs
-74.2% 53962 ± 13% numa-meminfo.node0.KReclaimable
+1129.1% 211277 ± 90% numa-meminfo.node0.Mapped
-13.1% 3221190 ± 3% numa-meminfo.node0.MemUsed
-86.5% 77122 ± 49% numa-meminfo.node0.PageTables
-74.2% 53962 ± 13% numa-meminfo.node0.SReclaimable
-48.4% 159181 ± 7% numa-meminfo.node0.Slab
-84.5% 44861 ± 17% numa-meminfo.node1.KReclaimable
-24.0% 2994130 ± 4% numa-meminfo.node1.MemUsed
-88.0% 72494 ± 58% numa-meminfo.node1.PageTables
-84.5% 44861 ± 17% numa-meminfo.node1.SReclaimable
-66.1% 126490 ± 9% numa-meminfo.node1.Slab
+4.4% 5331 ± 2% proc-vmstat.nr_active_file
+1.3% 65642 proc-vmstat.nr_anon_pages
+41.6% 121.75 ± 28% proc-vmstat.nr_dirtied
+3.4% 1064023 proc-vmstat.nr_dirty_background_threshold
+3.4% 2130649 proc-vmstat.nr_dirty_threshold
+3.4% 10723419 proc-vmstat.nr_free_pages
+8.2% 103375 proc-vmstat.nr_inactive_anon
+25.6% 105.50 ± 11% proc-vmstat.nr_inactive_file
+8.7% 105168 proc-vmstat.nr_mapped
-87.2% 37319 ± 3% proc-vmstat.nr_page_table_pages
-80.2% 24716 proc-vmstat.nr_slab_reclaimable
+2.0% 46711 proc-vmstat.nr_slab_unreclaimable
+4.4% 5331 ± 2% proc-vmstat.nr_zone_active_file
+8.2% 103375 proc-vmstat.nr_zone_inactive_anon
+25.6% 105.50 ± 11% proc-vmstat.nr_zone_inactive_file
-59.4% 864384 proc-vmstat.numa_hit
-60.3% 833061 proc-vmstat.numa_local
+16.8% 3637 ± 9% proc-vmstat.numa_pte_updates
-60.9% 886227 proc-vmstat.pgalloc_normal
-84.9% 71307545 ± 3% proc-vmstat.pgfault
-62.9% 806246 proc-vmstat.pgfree
-88.8% 74786 ± 3% proc-vmstat.thp_fault_fallback
+134.2% 302.32 ± 13% sched_debug.cfs_rq:/.exec_clock.min
+49.3% 1183484 ± 10% sched_debug.cfs_rq:/.load.max
+16.1% 399036 ± 2% sched_debug.cfs_rq:/.load.stddev
+31.2% 1.31 ± 8% sched_debug.cfs_rq:/.nr_running.max
+9.3% 0.44 sched_debug.cfs_rq:/.nr_running.stddev
+49.5% 1183484 ± 10% sched_debug.cfs_rq:/.runnable_weight.max
+16.1% 399045 ± 2% sched_debug.cfs_rq:/.runnable_weight.stddev
-38.6% 337.58 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.avg
+36.4% 814808 sched_debug.cpu.avg_idle.avg
-38.5% 225103 ± 2% sched_debug.cpu.avg_idle.stddev
+236.1% 9.18 ± 14% sched_debug.cpu.clock.stddev
+236.1% 9.18 ± 14% sched_debug.cpu.clock_task.stddev
+31.2% 1.31 ± 8% sched_debug.cpu.nr_running.max
+78.8% 34650 ± 35% sched_debug.cpu.nr_switches.max
+95.1% 27315 ± 33% sched_debug.cpu.sched_count.max
+40.3% 3316 ± 23% sched_debug.cpu.sched_count.stddev
+95.7% 13623 ± 34% sched_debug.cpu.sched_goidle.max
-71.7% 6.62 ± 13% sched_debug.cpu.sched_goidle.min
+39.6% 1663 ± 23% sched_debug.cpu.sched_goidle.stddev
+51.0% 78.06 ± 24% sched_debug.cpu.ttwu_count.min
-12.4% 386.45 ± 2% sched_debug.cpu.ttwu_local.avg
+28.1% 39.06 ± 17% sched_debug.cpu.ttwu_local.min
-8.1% 177.50 ± 3% interrupts.37:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
+6.6e+103% 66.25 ±159% interrupts.88:PCI-MSI.31981619-edge.i40e-eth0-TxRx-50
+3928.9% 975.00 ±126% interrupts.CPU10.RES:Rescheduling_interrupts
+284.1% 84.50 ± 59% interrupts.CPU12.RES:Rescheduling_interrupts
+408.8% 827.25 ± 98% interrupts.CPU2.RES:Rescheduling_interrupts
+1779.8% 195.50 ± 84% interrupts.CPU26.RES:Rescheduling_interrupts
+3985.2% 179.75 ±107% interrupts.CPU27.RES:Rescheduling_interrupts
+7481.5% 6368 ±163% interrupts.CPU30.RES:Rescheduling_interrupts
+368.2% 383.00 ± 98% interrupts.CPU31.RES:Rescheduling_interrupts
+6097.9% 148.75 ±145% interrupts.CPU32.RES:Rescheduling_interrupts
+4689.1% 153.25 ±111% interrupts.CPU33.RES:Rescheduling_interrupts
-18.2% 2324 ± 5% interrupts.CPU35.CAL:Function_call_interrupts
585.00 ±158% interrupts.CPU38.RES:Rescheduling_interrupts
+2422.7% 55.50 ± 56% interrupts.CPU39.RES:Rescheduling_interrupts
+7660.4% 186.25 ±133% interrupts.CPU42.RES:Rescheduling_interrupts
+1642.2% 55.75 ± 88% interrupts.CPU44.RES:Rescheduling_interrupts
+4604.5% 103.50 ±102% interrupts.CPU45.RES:Rescheduling_interrupts
+6.6e+103% 66.00 ±160% interrupts.CPU50.88:PCI-MSI.31981619-edge.i40e-eth0-TxRx-50
+684.5% 91.00 ± 65% interrupts.CPU50.RES:Rescheduling_interrupts
+1545.2% 102.00 ±104% interrupts.CPU51.RES:Rescheduling_interrupts
+1416.3% 139.50 ± 61% interrupts.CPU54.RES:Rescheduling_interrupts
+503.8% 78.50 ± 73% interrupts.CPU55.RES:Rescheduling_interrupts
+962.5% 80.75 ± 83% interrupts.CPU56.RES:Rescheduling_interrupts
+416.9% 76.50 ± 41% interrupts.CPU57.RES:Rescheduling_interrupts
+1445.7% 126.75 ± 88% interrupts.CPU60.RES:Rescheduling_interrupts
+817.6% 86.25 ± 28% interrupts.CPU61.RES:Rescheduling_interrupts
+513.2% 90.75 ± 32% interrupts.CPU62.RES:Rescheduling_interrupts
+1400.0% 84.00 ± 51% interrupts.CPU63.RES:Rescheduling_interrupts
+750.6% 69.75 ± 33% interrupts.CPU65.RES:Rescheduling_interrupts
+581.4% 69.50 ± 28% interrupts.CPU68.RES:Rescheduling_interrupts
+150.0% 91.50 ± 69% interrupts.CPU70.TLB:TLB_shootdowns
+594.8% 59.75 ± 51% interrupts.CPU76.RES:Rescheduling_interrupts
+1497.2% 57.50 ± 73% interrupts.CPU78.RES:Rescheduling_interrupts
+1582.7% 131.25 ± 91% interrupts.CPU8.RES:Rescheduling_interrupts
+293.7% 68.50 ± 60% interrupts.CPU8.TLB:TLB_shootdowns
-8.1% 177.50 ± 3% interrupts.CPU9.37:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
+81.5% 26.70 perf-stat.i.MPKI
+69.4% 9.665e+09 ± 4% perf-stat.i.branch-instructions
+0.1 0.54 ± 4% perf-stat.i.branch-miss-rate%
+111.4% 50251745 ± 3% perf-stat.i.branch-misses
+26.7 81.45 perf-stat.i.cache-miss-rate%
+367.4% 1.113e+09 ± 4% perf-stat.i.cache-misses
+229.1% 1.321e+09 ± 4% perf-stat.i.cache-references
-39.4% 2.66 ± 4% perf-stat.i.cpi
+8.0% 1.291e+11 perf-stat.i.cpu-cycles
-66.4% 301.06 ± 17% perf-stat.i.cycles-between-cache-misses
-0.0 0.01 ± 52% perf-stat.i.dTLB-load-miss-rate%
-78.6% 663585 ± 3% perf-stat.i.dTLB-load-misses
+97.8% 1.345e+10 ± 5% perf-stat.i.dTLB-loads
-0.2 0.01 ± 7% perf-stat.i.dTLB-store-miss-rate%
-88.6% 844034 ± 3% perf-stat.i.dTLB-store-misses
+185.8% 8.022e+09 ± 4% perf-stat.i.dTLB-stores
-30.1 38.46 ± 5% perf-stat.i.iTLB-load-miss-rate%
-75.9% 3100419 ± 4% perf-stat.i.iTLB-load-misses
+89.4% 4.834e+10 ± 4% perf-stat.i.instructions
+618.7% 15776 ± 8% perf-stat.i.instructions-per-iTLB-miss
+41.6% 0.38 ± 4% perf-stat.i.ipc
-83.5% 342553 ± 3% perf-stat.i.minor-faults
+545.5% 2.027e+08 ± 22% perf-stat.i.node-load-misses
+625.0% 2.372e+08 ± 26% perf-stat.i.node-loads
+446.5% 1.474e+08 ± 18% perf-stat.i.node-store-misses
+515.9% 1.569e+08 ± 22% perf-stat.i.node-stores
-83.5% 342558 ± 3% perf-stat.i.page-faults
+73.7% 27.33 perf-stat.overall.MPKI
+0.1 0.52 perf-stat.overall.branch-miss-rate%
+24.9 84.27 perf-stat.overall.cache-miss-rate%
-42.8% 2.68 ± 4% perf-stat.overall.cpi
-76.9% 116.15 ± 4% perf-stat.overall.cycles-between-cache-misses
-0.0 0.00 ± 5% perf-stat.overall.dTLB-load-miss-rate%
-0.3 0.01 ± 4% perf-stat.overall.dTLB-store-miss-rate%
-32.5 38.22 ± 6% perf-stat.overall.iTLB-load-miss-rate%
+686.7% 15642 ± 7% perf-stat.overall.instructions-per-iTLB-miss
+75.3% 0.37 ± 4% perf-stat.overall.ipc
-72.1% 2357 perf-stat.overall.path-length
+69.4% 9.622e+09 ± 4% perf-stat.ps.branch-instructions
+111.4% 50029998 ± 3% perf-stat.ps.branch-misses
+367.4% 1.109e+09 ± 4% perf-stat.ps.cache-misses
+229.1% 1.316e+09 ± 4% perf-stat.ps.cache-references
+8.1% 1.285e+11 perf-stat.ps.cpu-cycles
-78.6% 660373 ± 3% perf-stat.ps.dTLB-load-misses
+97.8% 1.339e+10 ± 5% perf-stat.ps.dTLB-loads
-88.6% 840338 ± 3% perf-stat.ps.dTLB-store-misses
+185.8% 7.987e+09 ± 4% perf-stat.ps.dTLB-stores
-75.9% 3086339 ± 4% perf-stat.ps.iTLB-load-misses
+89.4% 4.813e+10 ± 4% perf-stat.ps.instructions
-83.5% 341065 ± 3% perf-stat.ps.minor-faults
+545.7% 2.019e+08 ± 22% perf-stat.ps.node-load-misses
+625.0% 2.361e+08 ± 26% perf-stat.ps.node-loads
+446.6% 1.468e+08 ± 18% perf-stat.ps.node-store-misses
+515.9% 1.562e+08 ± 22% perf-stat.ps.node-stores
-83.5% 341071 ± 3% perf-stat.ps.page-faults
+73.9% 1.002e+13 ± 4% perf-stat.total.instructions
+516.5% 46458 ± 13% softirqs.CPU0.RCU
+575.9% 41409 ± 15% softirqs.CPU1.RCU
+611.1% 45378 ± 12% softirqs.CPU10.RCU
+641.3% 45123 ± 9% softirqs.CPU11.RCU
+619.4% 45231 ± 8% softirqs.CPU12.RCU
+521.9% 45667 ± 10% softirqs.CPU13.RCU
+659.0% 45142 ± 8% softirqs.CPU14.RCU
+606.4% 45259 ± 9% softirqs.CPU15.RCU
+647.9% 49467 ± 10% softirqs.CPU16.RCU
+553.1% 47167 ± 9% softirqs.CPU17.RCU
+578.4% 46989 ± 9% softirqs.CPU18.RCU
+634.3% 48218 ± 9% softirqs.CPU19.RCU
+457.5% 42812 ± 18% softirqs.CPU2.RCU
+628.6% 48168 ± 10% softirqs.CPU20.RCU
+605.7% 47472 ± 13% softirqs.CPU21.RCU
+638.5% 48528 ± 10% softirqs.CPU22.RCU
+541.6% 48542 ± 10% softirqs.CPU23.RCU
+520.9% 51863 ± 8% softirqs.CPU24.RCU
+632.8% 44682 ± 14% softirqs.CPU25.RCU
+703.2% 47600 ± 7% softirqs.CPU26.RCU
+665.0% 46641 ± 10% softirqs.CPU27.RCU
+719.1% 47372 ± 6% softirqs.CPU28.RCU
+693.9% 47619 ± 7% softirqs.CPU29.RCU
+326.6% 46623 ± 11% softirqs.CPU3.RCU
+587.4% 47155 ± 7% softirqs.CPU30.RCU
+658.9% 44099 ± 7% softirqs.CPU31.RCU
+677.7% 47497 ± 6% softirqs.CPU32.RCU
+640.9% 44990 ± 5% softirqs.CPU33.RCU
+695.4% 47835 ± 6% softirqs.CPU34.RCU
+679.8% 46801 ± 10% softirqs.CPU35.RCU
+644.9% 47101 ± 11% softirqs.CPU36.RCU
+708.9% 47666 ± 6% softirqs.CPU37.RCU
+702.9% 47322 ± 5% softirqs.CPU38.RCU
+674.8% 47490 ± 5% softirqs.CPU39.RCU
+618.3% 44214 ± 13% softirqs.CPU4.RCU
+646.7% 47224 ± 5% softirqs.CPU40.RCU
+659.6% 44884 ± 7% softirqs.CPU41.RCU
+680.5% 47855 ± 5% softirqs.CPU42.RCU
+724.0% 48673 ± 7% softirqs.CPU43.RCU
+650.2% 45159 ± 8% softirqs.CPU44.RCU
+728.0% 48510 ± 8% softirqs.CPU45.RCU
+697.1% 46851 ± 5% softirqs.CPU46.RCU
+645.7% 43449 ± 2% softirqs.CPU47.RCU
+562.9% 47870 ± 4% softirqs.CPU48.RCU
-63.5% 7517 ± 65% softirqs.CPU48.SCHED
+597.3% 43232 ± 9% softirqs.CPU49.RCU
+532.4% 44217 ± 8% softirqs.CPU5.RCU
+686.3% 46619 ± 9% softirqs.CPU50.RCU
+558.3% 43964 ± 4% softirqs.CPU51.RCU
+623.5% 47186 ± 8% softirqs.CPU52.RCU
+680.5% 46325 ± 4% softirqs.CPU53.RCU
+674.1% 45468 ± 5% softirqs.CPU54.RCU
+673.8% 45947 ± 3% softirqs.CPU55.RCU
+693.2% 45435 ± 3% softirqs.CPU56.RCU
+684.8% 46133 ± 4% softirqs.CPU57.RCU
+651.3% 43475 ± 8% softirqs.CPU58.RCU
+673.9% 45855 ± 4% softirqs.CPU59.RCU
+619.3% 45045 ± 10% softirqs.CPU6.RCU
+619.2% 43923 ± 6% softirqs.CPU60.RCU
+638.9% 45082 ± 5% softirqs.CPU61.RCU
+582.2% 43759 ± 3% softirqs.CPU62.RCU
+712.7% 47419 ± 5% softirqs.CPU63.RCU
+694.5% 48633 ± 3% softirqs.CPU64.RCU
+605.1% 47880 ± 10% softirqs.CPU65.RCU
+678.5% 47797 ± 9% softirqs.CPU66.RCU
+681.1% 46887 ± 4% softirqs.CPU67.RCU
+720.3% 47306 ± 9% softirqs.CPU68.RCU
+802.4% 52160 ± 4% softirqs.CPU69.RCU
-59.8% 10032 ±110% softirqs.CPU69.SCHED
+592.8% 43066 ± 8% softirqs.CPU7.RCU
+732.2% 48584 ± 7% softirqs.CPU70.RCU
+695.4% 48767 ± 7% softirqs.CPU71.RCU
+688.5% 47249 ± 10% softirqs.CPU72.RCU
+666.4% 47042 ± 10% softirqs.CPU73.RCU
+692.0% 46641 ± 11% softirqs.CPU74.RCU
+649.4% 44673 ± 12% softirqs.CPU75.RCU
+632.7% 46652 ± 13% softirqs.CPU76.RCU
+675.3% 46776 ± 11% softirqs.CPU77.RCU
+634.7% 46759 ± 13% softirqs.CPU78.RCU
+650.1% 44198 ± 15% softirqs.CPU79.RCU
+558.4% 44328 ± 9% softirqs.CPU8.RCU
+659.8% 44272 ± 9% softirqs.CPU80.RCU
+663.5% 45095 ± 7% softirqs.CPU81.RCU
+621.3% 44831 ± 9% softirqs.CPU82.RCU
+636.1% 43371 ± 9% softirqs.CPU83.RCU
+690.5% 45364 ± 6% softirqs.CPU84.RCU
+634.4% 44190 ± 8% softirqs.CPU85.RCU
+644.9% 44398 ± 9% softirqs.CPU86.RCU
+635.9% 44782 ± 9% softirqs.CPU87.RCU
+662.2% 44277 ± 9% softirqs.CPU88.RCU
+634.6% 43246 ± 8% softirqs.CPU89.RCU
+666.3% 44105 ± 10% softirqs.CPU9.RCU
+663.9% 44804 ± 8% softirqs.CPU90.RCU
+660.7% 45354 ± 9% softirqs.CPU91.RCU
+624.6% 42854 ± 10% softirqs.CPU92.RCU
+646.3% 44333 ± 9% softirqs.CPU93.RCU
+673.2% 44188 ± 8% softirqs.CPU94.RCU
+659.7% 44215 ± 10% softirqs.CPU95.RCU
+635.8% 4420294 ± 3% softirqs.RCU
-11.1% 1551273 softirqs.SCHED
-62.8 3.73 ± 14% perf-profile.calltrace.cycles-pp.page_fault
-62.8 3.71 ± 14% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
-62.7 3.66 ± 15% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
-62.7 3.43 ± 15% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
-62.7 3.33 ± 15% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
-57.5 0.30 ±173% perf-profile.calltrace.cycles-pp.lookup_memtype.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault
-55.9 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.lookup_memtype.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault
-55.3 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.lookup_memtype.track_pfn_insert.__vm_insert_mixed
-45.8 0.89 ± 24% perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
-45.8 0.88 ± 23% perf-profile.calltrace.cycles-pp.ext4_dax_huge_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
-43.2 0.79 ± 24% perf-profile.calltrace.cycles-pp.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault.__handle_mm_fault.handle_mm_fault
-41.8 0.18 ±173% perf-profile.calltrace.cycles-pp.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault.__handle_mm_fault
-41.4 0.17 ±173% perf-profile.calltrace.cycles-pp.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault
-18.3 0.81 ± 27% perf-profile.calltrace.cycles-pp.ext4_dax_huge_fault.do_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
-18.3 0.82 ± 27% perf-profile.calltrace.cycles-pp.do_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
-17.0 0.36 ±100% perf-profile.calltrace.cycles-pp.dax_iomap_pte_fault.ext4_dax_huge_fault.do_wp_page.__handle_mm_fault.handle_mm_fault
-16.3 0.14 ±173% perf-profile.calltrace.cycles-pp.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.do_wp_page.__handle_mm_fault
-16.1 0.13 ±173% perf-profile.calltrace.cycles-pp.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.do_wp_page
+0.6 0.58 perf-profile.calltrace.cycles-pp.__get_io_u
+0.8 0.78 ± 14% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
+0.8 1.16 ± 24% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
+0.8 1.17 ± 24% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
+0.8 1.17 ± 24% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
+0.8 1.17 ± 24% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
+0.8 1.17 ± 24% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
+0.9 0.85 ± 11% perf-profile.calltrace.cycles-pp.dax_iomap_pmd_fault.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
+1.0 0.98 ± 10% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
+1.0 1.05 ± 17% perf-profile.calltrace.cycles-pp.io_u_mark_submit
+1.1 1.12 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
+1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt
+1.2 1.19 ± 2% perf-profile.calltrace.cycles-pp.utime_since_now
+1.2 1.20 ± 6% perf-profile.calltrace.cycles-pp.add_clat_sample
+1.2 1.21 ± 5% perf-profile.calltrace.cycles-pp.add_lat_sample
+1.2 1.22 ± 7% perf-profile.calltrace.cycles-pp.ramp_time_over
+1.5 1.51 ± 9% perf-profile.calltrace.cycles-pp.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
+3.6 3.64 ± 3% perf-profile.calltrace.cycles-pp.get_io_u
+4.0 4.02 ± 3% perf-profile.calltrace.cycles-pp.io_queue_event
+4.5 4.46 ± 4% perf-profile.calltrace.cycles-pp.td_io_queue
+5.4 5.40 ± 4% perf-profile.calltrace.cycles-pp.fio_gettime
+9.7 9.71 ± 4% perf-profile.calltrace.cycles-pp.io_u_sync_complete
+25.9 56.78 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
+26.0 56.80 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
+26.1 57.08 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
+26.3 57.20 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
+26.3 57.21 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
+26.3 57.21 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
+27.1 58.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64
-62.8 3.76 ± 14% perf-profile.children.cycles-pp.page_fault
-62.7 3.69 ± 14% perf-profile.children.cycles-pp.__do_page_fault
-62.7 3.74 ± 14% perf-profile.children.cycles-pp.do_page_fault
-62.7 3.46 ± 15% perf-profile.children.cycles-pp.handle_mm_fault
-62.7 3.35 ± 15% perf-profile.children.cycles-pp.__handle_mm_fault
-62.7 3.20 ± 15% perf-profile.children.cycles-pp.ext4_dax_huge_fault
-59.9 1.37 ± 25% perf-profile.children.cycles-pp.dax_iomap_pte_fault
-57.5 0.86 ± 28% perf-profile.children.cycles-pp.__vm_insert_mixed
-56.6 1.17 ± 22% perf-profile.children.cycles-pp.lookup_memtype
-56.6 1.17 ± 22% perf-profile.children.cycles-pp.track_pfn_insert
-56.0 0.40 ± 24% perf-profile.children.cycles-pp._raw_spin_lock
-55.2 0.07 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
-45.8 0.89 ± 23% perf-profile.children.cycles-pp.__do_fault
-18.3 0.82 ± 27% perf-profile.children.cycles-pp.do_wp_page
-2.5 0.28 ± 13% perf-profile.children.cycles-pp.jbd2__journal_start
-2.5 0.21 ± 11% perf-profile.children.cycles-pp.start_this_handle
-1.9 0.35 ± 16% perf-profile.children.cycles-pp.ext4_iomap_begin
-1.2 0.48 ± 24% perf-profile.children.cycles-pp._raw_read_lock
-1.1 0.18 ± 14% perf-profile.children.cycles-pp.__ext4_journal_stop
-1.0 0.17 ± 13% perf-profile.children.cycles-pp.jbd2_journal_stop
-1.0 0.71 ± 21% perf-profile.children.cycles-pp.pat_pagerange_is_ram
-1.0 0.69 ± 22% perf-profile.children.cycles-pp.find_next_iomem_res
-1.0 0.71 ± 21% perf-profile.children.cycles-pp.walk_system_ram_range
-1.0 0.08 ± 24% perf-profile.children.cycles-pp.jbd2_transaction_committed
-0.7 0.10 ± 14% perf-profile.children.cycles-pp.__ext4_journal_start_sb
-0.7 0.10 ± 17% perf-profile.children.cycles-pp.ext4_journal_check_start
-0.5 0.03 ±100% perf-profile.children.cycles-pp.insert_pfn
-0.1 0.13 ± 23% perf-profile.children.cycles-pp.sync_regs
-0.1 0.11 ± 14% perf-profile.children.cycles-pp.rbt_memtype_lookup
-0.1 0.05 ± 61% perf-profile.children.cycles-pp.dax_insert_entry
-0.1 0.04 ± 57% perf-profile.children.cycles-pp._raw_spin_lock_irq
-0.1 0.07 ± 22% perf-profile.children.cycles-pp.dax_unlock_entry
-0.1 0.08 ± 27% perf-profile.children.cycles-pp.ext4_es_lookup_extent
-0.0 0.07 ± 22% perf-profile.children.cycles-pp.dax_iomap_pfn
-0.0 0.10 ± 8% perf-profile.children.cycles-pp.xas_store
+0.0 0.10 ± 5% perf-profile.children.cycles-pp.___perf_sw_event
+0.0 0.19 ± 6% perf-profile.children.cycles-pp.grab_mapping_entry
+0.0 0.11 ± 7% perf-profile.children.cycles-pp.get_unlocked_entry
+0.0 0.10 ± 8% perf-profile.children.cycles-pp.xas_find_conflict
+0.1 0.06 ± 6% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
+0.1 0.05 ± 8% perf-profile.children.cycles-pp.lapic_next_deadline
+0.1 0.05 ± 8% perf-profile.children.cycles-pp.hrtimer_active
+0.1 0.06 ± 9% perf-profile.children.cycles-pp.td_set_runstate
+0.1 0.06 ± 14% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
+0.1 0.06 ± 11% perf-profile.children.cycles-pp.__run_perf_stat
+0.1 0.06 ± 11% perf-profile.children.cycles-pp.process_interval
+0.1 0.06 ± 11% perf-profile.children.cycles-pp.read_counters
+0.1 0.07 ± 10% perf-profile.children.cycles-pp.vmacache_find
+0.1 0.07 ± 17% perf-profile.children.cycles-pp.console_unlock
+0.1 0.06 ± 17% perf-profile.children.cycles-pp.cmd_stat
+0.1 0.07 ± 11% perf-profile.children.cycles-pp.find_vma
+0.1 0.08 ± 10% perf-profile.children.cycles-pp.irq_work_run_list
+0.1 0.07 ± 12% perf-profile.children.cycles-pp.queue_full
+0.1 0.07 ± 16% perf-profile.children.cycles-pp.log_io_u
+0.1 0.07 ± 12% perf-profile.children.cycles-pp.ext4_data_block_valid_rcu
+0.1 0.07 ± 28% perf-profile.children.cycles-pp.io_serial_in
+0.1 0.07 ± 10% perf-profile.children.cycles-pp.__ext4_get_inode_loc
+0.1 0.15 ± 9% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty
+0.1 0.08 ± 27% perf-profile.children.cycles-pp.__x64_sys_poll
+0.1 0.08 ± 27% perf-profile.children.cycles-pp.do_sys_poll
+0.1 0.08 ± 14% perf-profile.children.cycles-pp.get_start_offset
+0.1 0.08 perf-profile.children.cycles-pp.lock_file
+0.1 0.08 ± 15% perf-profile.children.cycles-pp.vfs_read
+0.1 0.08 ± 30% perf-profile.children.cycles-pp.rcu_core
+0.1 0.08 ± 10% perf-profile.children.cycles-pp.ksys_read
+0.1 0.09 ± 21% perf-profile.children.cycles-pp.perf_evlist__poll
+0.1 0.09 ± 21% perf-profile.children.cycles-pp.poll
+0.1 0.09 ± 4% perf-profile.children.cycles-pp.flow_threshold_exceeded
+0.1 0.10 ± 14% perf-profile.children.cycles-pp.ext4_reserve_inode_write
+0.1 0.10 ± 33% perf-profile.children.cycles-pp.update_curr
+0.1 0.11 ± 21% perf-profile.children.cycles-pp.native_write_msr
+0.1 0.43 ± 7% perf-profile.children.cycles-pp.ext4_dirty_inode
+0.1 0.11 ± 7% perf-profile.children.cycles-pp.get_file
+0.1 0.11 ± 19% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
+0.1 0.50 ± 8% perf-profile.children.cycles-pp.generic_update_time
+0.1 0.14 ± 60% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
+0.1 0.13 ± 6% perf-profile.children.cycles-pp.put_file_log
+0.1 0.13 ± 40% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
+0.1 0.13 ± 23% perf-profile.children.cycles-pp.update_load_avg
+0.1 0.16 ± 37% perf-profile.children.cycles-pp.__generic_file_write_iter
+0.1 0.16 ± 37% perf-profile.children.cycles-pp.generic_perform_write
+0.1 0.19 ± 23% perf-profile.children.cycles-pp.dax_wake_entry
+0.1 0.17 ± 37% perf-profile.children.cycles-pp.generic_file_write_iter
+0.1 0.17 ± 38% perf-profile.children.cycles-pp.new_sync_write
+0.1 0.17 ± 37% perf-profile.children.cycles-pp.__GI___libc_write
+0.2 0.18 ± 36% perf-profile.children.cycles-pp.record__pushfn
+0.2 0.18 ± 36% perf-profile.children.cycles-pp.writen
+0.2 0.18 ± 34% perf-profile.children.cycles-pp.ksys_write
+0.2 0.18 ± 34% perf-profile.children.cycles-pp.vfs_write
+0.2 0.18 ± 36% perf-profile.children.cycles-pp.perf_mmap__push
+0.2 0.27 ± 8% perf-profile.children.cycles-pp.ext4_mark_inode_dirty
+0.2 0.19 ± 35% perf-profile.children.cycles-pp.record__mmap_read_evlist
+0.2 0.21 ± 5% perf-profile.children.cycles-pp.unlock_file
+0.2 0.17 ± 4% perf-profile.children.cycles-pp.utime_since
+0.2 0.20 ± 26% perf-profile.children.cycles-pp.rcu_sched_clock_irq
+0.2 0.20 ± 2% perf-profile.children.cycles-pp.in_ramp_time
+0.2 0.27 ± 31% perf-profile.children.cycles-pp.cmd_record
+0.2 0.28 ± 56% perf-profile.children.cycles-pp.menu_select
+0.2 0.24 ± 4% perf-profile.children.cycles-pp.put_file
+0.2 0.28 ± 57% perf-profile.children.cycles-pp.__softirqentry_text_start
+0.3 0.33 ± 25% perf-profile.children.cycles-pp.__libc_start_main
+0.3 0.33 ± 25% perf-profile.children.cycles-pp.main
+0.3 0.33 ± 25% perf-profile.children.cycles-pp.run_builtin
+0.3 0.30 perf-profile.children.cycles-pp.td_io_prep
+0.3 0.31 ± 2% perf-profile.children.cycles-pp.ntime_since
+0.3 0.36 ± 54% perf-profile.children.cycles-pp.irq_exit
+0.3 0.40 ± 26% perf-profile.children.cycles-pp.task_tick_fair
+0.4 0.35 ± 11% perf-profile.children.cycles-pp.io_u_mark_depth
+0.4 0.38 ± 13% perf-profile.children.cycles-pp.vmf_insert_pfn_pmd
+0.4 0.51 ± 19% perf-profile.children.cycles-pp.scheduler_tick
+0.5 0.46 ± 8% perf-profile.children.cycles-pp.io_u_mark_complete
+0.5 0.56 ± 16% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
+0.5 0.56 ± 16% perf-profile.children.cycles-pp.do_syscall_64
+0.6 0.58 perf-profile.children.cycles-pp.__get_io_u
+0.7 0.91 ± 10% perf-profile.children.cycles-pp.update_process_times
+0.8 0.94 ± 9% perf-profile.children.cycles-pp.tick_sched_handle
+0.8 1.17 ± 24% perf-profile.children.cycles-pp.start_kernel
+0.8 1.01 ± 9% perf-profile.children.cycles-pp.tick_sched_timer
+0.9 0.86 ± 10% perf-profile.children.cycles-pp.dax_iomap_pmd_fault
+1.0 1.05 ± 16% perf-profile.children.cycles-pp.io_u_mark_submit
+1.0 1.26 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
+1.1 1.19 ± 2% perf-profile.children.cycles-pp.utime_since_now
+1.1 1.21 ± 5% perf-profile.children.cycles-pp.add_lat_sample
+1.1 1.20 ± 6% perf-profile.children.cycles-pp.add_clat_sample
+1.2 1.66 ± 5% perf-profile.children.cycles-pp.hrtimer_interrupt
+1.2 1.23 ± 7% perf-profile.children.cycles-pp.ramp_time_over
+1.6 2.09 ± 14% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
+1.6 2.17 ± 14% perf-profile.children.cycles-pp.apic_timer_interrupt
+3.5 3.78 ± 3% perf-profile.children.cycles-pp.get_io_u
+4.9 5.06 ± 3% perf-profile.children.cycles-pp.td_io_queue
+5.2 5.49 ± 4% perf-profile.children.cycles-pp.fio_gettime
+5.5 5.45 ± 4% perf-profile.children.cycles-pp.io_queue_event
+8.8 8.92 ± 4% perf-profile.children.cycles-pp.io_u_sync_complete
+26.1 57.08 ± 2% perf-profile.children.cycles-pp.intel_idle
+26.3 57.21 ± 3% perf-profile.children.cycles-pp.start_secondary
+26.8 57.96 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
+26.8 57.97 ± 3% perf-profile.children.cycles-pp.cpuidle_enter
+27.1 58.38 ± 3% perf-profile.children.cycles-pp.do_idle
+27.1 58.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64
+27.1 58.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry
-54.9 0.07 ± 11% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
-1.3 0.08 ± 11% perf-profile.self.cycles-pp.start_this_handle
-1.2 0.48 ± 25% perf-profile.self.cycles-pp._raw_read_lock
-1.1 0.10 ± 12% perf-profile.self.cycles-pp.jbd2_journal_stop
-0.8 0.38 ± 16% perf-profile.self.cycles-pp.find_next_iomem_res
-0.7 0.34 ± 26% perf-profile.self.cycles-pp._raw_spin_lock
-0.6 0.08 ± 13% perf-profile.self.cycles-pp.ext4_journal_check_start
-0.1 0.13 ± 23% perf-profile.self.cycles-pp.sync_regs
-0.1 0.04 ± 58% perf-profile.self.cycles-pp.ext4_iomap_begin
-0.1 0.11 ± 16% perf-profile.self.cycles-pp.rbt_memtype_lookup
-0.1 0.04 ± 57% perf-profile.self.cycles-pp._raw_spin_lock_irq
-0.0 0.04 ± 59% perf-profile.self.cycles-pp.handle_mm_fault
+0.0 0.08 ± 5% perf-profile.self.cycles-pp.xas_find_conflict
+0.1 0.05 ± 8% perf-profile.self.cycles-pp.hrtimer_interrupt
+0.1 0.05 ± 8% perf-profile.self.cycles-pp.log_io_u
+0.1 0.05 ± 8% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
+0.1 0.05 ± 8% perf-profile.self.cycles-pp.hrtimer_active
+0.1 0.06 ± 9% perf-profile.self.cycles-pp.td_set_runstate
+0.1 0.06 ± 11% perf-profile.self.cycles-pp.io_serial_in
+0.1 0.07 ± 10% perf-profile.self.cycles-pp.vmacache_find
+0.1 0.06 ± 6% perf-profile.self.cycles-pp.lock_file
+0.1 0.07 ± 17% perf-profile.self.cycles-pp.ext4_data_block_valid_rcu
+0.1 0.07 ± 12% perf-profile.self.cycles-pp.queue_full
+0.1 0.08 ± 11% perf-profile.self.cycles-pp.ext4_mark_iloc_dirty
+0.1 0.08 ± 14% perf-profile.self.cycles-pp.get_start_offset
+0.1 0.08 ± 10% perf-profile.self.cycles-pp.get_file
+0.1 0.09 ± 4% perf-profile.self.cycles-pp.flow_threshold_exceeded
+0.1 0.09 ± 40% perf-profile.self.cycles-pp.menu_select
+0.1 0.09 ± 15% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
+0.1 0.10 ± 18% perf-profile.self.cycles-pp.native_write_msr
+0.1 0.10 ± 4% perf-profile.self.cycles-pp.put_file_log
+0.1 0.18 ± 18% perf-profile.self.cycles-pp.dax_wake_entry
+0.2 0.20 ± 6% perf-profile.self.cycles-pp.unlock_file
+0.2 0.17 ± 4% perf-profile.self.cycles-pp.utime_since
+0.2 0.19 ± 29% perf-profile.self.cycles-pp.rcu_sched_clock_irq
+0.2 0.20 ± 3% perf-profile.self.cycles-pp.in_ramp_time
+0.2 0.20 ± 10% perf-profile.self.cycles-pp.io_u_mark_depth
+0.2 0.22 ± 5% perf-profile.self.cycles-pp.put_file
+0.2 0.24 ± 2% perf-profile.self.cycles-pp.td_io_prep
+0.3 0.31 ± 2% perf-profile.self.cycles-pp.ntime_since
+0.4 0.45 ± 8% perf-profile.self.cycles-pp.io_u_mark_complete
+0.6 0.57 ± 2% perf-profile.self.cycles-pp.__get_io_u
+0.7 0.70 ± 15% perf-profile.self.cycles-pp.io_u_mark_submit
+0.9 0.91 ± 8% perf-profile.self.cycles-pp.ramp_time_over
+1.1 1.14 ± 2% perf-profile.self.cycles-pp.utime_since_now
+1.1 1.20 ± 5% perf-profile.self.cycles-pp.add_lat_sample
+1.1 1.18 ± 5% perf-profile.self.cycles-pp.add_clat_sample
+3.4 3.70 ± 3% perf-profile.self.cycles-pp.get_io_u
+4.8 4.92 ± 3% perf-profile.self.cycles-pp.td_io_queue
+5.1 5.37 ± 4% perf-profile.self.cycles-pp.fio_gettime
+5.4 5.40 ± 4% perf-profile.self.cycles-pp.io_queue_event
+7.3 7.38 ± 4% perf-profile.self.cycles-pp.io_u_sync_complete
+26.1 57.07 ± 2% perf-profile.self.cycles-pp.intel_idle

fio.read_bw_MBps

+-+-----------------------------------------------------------------+
|
O O O |
O O O O O O O O O |
O O O O O |
O |
|
|
|
|
|
|
|
.+.+..+..+..+..+..+.+..+..+.. .+..+.+..+..|
+-+-----------------------------------------------------------------+

fio.read_iops

+-+O--------------------------------------------------------------+
O O O |
O O O O O O O |
O O O O O O |
O O |
|
|
|
|
|
|
|
|
+-++..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..|
+-+---------------------------------------------------------------+

fio.read_clat_mean_us

-----------------------------------------------------------+
.+.. .+..+.. |
+..+..+..+..+ +..+..+..+..+..+.+..+..+..|
|
|
|
|
|
|
|
|
O |
O O O O O O O O O O O O O |
O O O |
-----------------------------------------------------------+

fio.read_clat_stddev

+-+-----------------------------------------------------------------+
+ + |
: : : : |
: : : :|
.+.. : : : :|
+..+..+.. .+.. .+. +..+..+.+.. : +..+..+. : |
+. +.+. + + |
|
|
|
|
|
O O O O O O O O O O O O |
O O O O O O O O |
+-+-----------------------------------------------------------------+

fio.read_clat_90__us

+-+-----------------------------------------------------------------+
|..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..|
|
|
|
|
|
|
|
|
|
|
|
O O O O O O O O O O O |
+-+O--O--O-O-------------------O-----------O-O-----O--O--O----O-----+

fio.read_clat_95__us

+-+-----------------------------------------------------------------+
.+..+.. .+..+..+.+..+.. .+..+.+.. |
+.+..+..+. +.+..+..+. +..+. +..|
|
|
|
|
|
|
|
|
|
|
O O O O O O O O O O O O O O O O O |
+-+O--O------------------------O-----------O----------O-------------+

fio.read_clat_99__us

+-+-----------------------------------------------------------------+
+.+..+..+..+. +.+..+..+..+. +..+. +..|
|
|
|
|
|
|
|
|
|
|
O O O O O O O O O O O O O O O O O O O O |
O |
+-+-----------------------------------------------------------------+

fio.write_bw_MBps

+-+-----------------------------------------------------------------+
|
O O O |
O O O O O O O O O |
O O O O O |
O |
|
|
|
|
|
|
|
.+.+..+..+..+..+..+.+..+..+.. .+..+.+..+..|
+-+-----------------------------------------------------------------+

fio.write_iops

+-+O--------------------------------------------------------------+
O O O |
O O O O O O O |
O O O O O O |
O O |
|
|
|
|
|
|
|
|
+-++..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..|
+-+---------------------------------------------------------------+

fio.write_clat_mean_us

+-+-----------------------------------------------------------------+
+-++..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..|
|
|
|
|
|
|
|
|
|
|
|
O O O O O O O O O O O O O O O O O O O O O |
+-+-----------------------------------------------------------------+

fio.write_clat_90__us

+-+-----------------------------------------------------------------+
|..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..|
|
|
|
|
|
|
|
|
|
|
O O |
O O O O O O O O O O O O O O O |
+-+-----------------------------------------------------------------+

fio.write_clat_95__us

+-+-----------------------------------------------------------------+
|
+-++..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..|
|
|
|
|
|
|
|
|
|
O O |
O O O O O O O O O O O O O O O |
+-+-----------------------------------------------------------------+

fio.write_clat_99__us

+-+-----------------------------------------------------------------+
+.. +.. |
.+.. .+.. .. ..|
+..+. +..+.+..+..+. +..+.+..+..+ +..+.+..+ |
|
|
|
|
|
|
|
|
O O O O O O O O O O O O O O O O O O O O O |
|
+-+-----------------------------------------------------------------+

fio.latency_2us_

-----------------------------------------------------------+
O O O |
O O |
O O O O O |
O O O O O |
O |
O |
|
O |
|
|
|
|
|
-----------------------------------------------------------+

fio.latency_20us_

-----------------------------------------------------------+
.+.. +..+..+..+ |
.. + .+..+..+..+..+.. .+..+..+..+..|
+. +..+ +. +. |
|
|
|
|
|
|
|
|
|
|
O--O--O--O--O--O--O--O--O-O--O--O--O--O--O--O--O--O--O-----+

fio.latency_50us_

-----------------------------------------------------------+
|
+.. +..+ +.. |
: + .+..+..+.. .. +.. ..|
+.. : + + +..+..+ +..+..+ |
: +..+.. .. |
+ + |
|
|
|
|
|
|
|
O--O--O--O--O--O--O--O--O-O--O--O--O--O--O--O--O--O--O-----+

fio.workload

+-+---------------------------------------------------------------+
O O O |
O O |
O O O O O O O O O O O O O |
O |
O |
|
|
|
|
|
|
|
|..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..+..+..+.+..|
+-+---------------------------------------------------------------+

fio.time.user_time

+-+-----------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O |
|
|
|
|
|
|
|
|
|
|
|
+-++..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..+..+..+..+.+..+..|
+-+-----------------------------------------------------------------+

fio.time.system_time

+-+-----------------------------------------------------------------+
.+..+.+..+..+..+.. .+.+..+..+..+..+..+.+..+..|
+. +. |
|
|
|
|
|
|
|
|
|
|
|
O-+O--O--O-O--O--O--O--O--O-O--O--O--O--O--O-O--O--O--O--O--O-O-----+

fio.time.percent_of_cpu_this_job_got

-----------------------------------------------------------+
O O O O O O O O O O O O O O O O O O O O O O |
|
|
|
|
|
|
|
|
|
+.. |
.. |
+..+..+..+..+..+.+..+..+..+..+..+..+..+.+..+..+..|
-----------------------------------------------------------+

fio.time.elapsed_time

-----------------------------------------------------------+
+ +..+..+. +. +. +. +. |
+ + |
+ |
|
|
|
|
|
|
|
|
|
|
-O--O--O-O--O--O--O--O--O--O--O--O--O--O--O-O--O--O--O-----+

fio.time.elapsed_time.max

-----------------------------------------------------------+
+ +..+..+. +. +. +. +. |
+ + |
+ |
|
|
|
|
|
|
|
|
|
|
-O--O--O-O--O--O--O--O--O--O--O--O--O--O--O-O--O--O--O-----+

fio.time.minor_page_faults

+-+---------------------------------------------------------------+
|..+..+.+..+..+. +..+..+..+ +..+..+.+..+..+..+.+..+..+..+.+..|
|
|
|
|
|
|
|
|
|
|
|
O O O O O O O O O O O O O O O O O O O O O |
+-+------------------------------O--O-----------------------------+

fio.time.voluntary_context_switches

+-+-----------------------------------------------------------------+
|..+..+..+.+..+..+..+..+..+ +..+.. .+..+.+..+..+..+..+..+.+..+..|
+. |
|
|
|
|
|
|
|
|
O |
O O O O O O O O O O O O |
O O O O O |
+-+-------------------------O---------------------------------------+

**********************************************************************************
96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
========================================================================
eq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
mance/2pmem/ext4/mmap/x86_64-rhel-7.6/dax/50%/debian-x86_64-2019-05-14.cgz/200s/write/lkp-csl-2sp6/200G/fio-basic/tb/0x5000021
("fs/dax: Fix pmd vs pte conflict detection")
v5.4-rc4 a70e8083a91b17a7c77012f7746
---------------------------
%reproduction fail:runs
| |
-12% 0:4 perf-profile.children.cycles-pp.error_entry
%change %stddev
| \
-21.6 0.94 ± 79% fio.latency_10ms%
-77.4 0.01 ± 24% fio.latency_20ms%
+60.9 60.94 ± 59% fio.latency_2ms%
+36.8 36.85 ± 97% fio.latency_4ms%
-7.4% 210.34 fio.time.elapsed_time
-7.4% 210.34 fio.time.elapsed_time.max
-90.4% 44651382 ± 6% fio.time.minor_page_faults
+8.1% 4577 fio.time.percent_of_cpu_this_job_got
-96.5% 314.08 ± 3% fio.time.system_time
+1443.2% 9315 fio.time.user_time
-10.0% 22623 fio.time.voluntary_context_switches
+363.7% 4210010 ± 15% fio.workload
+363.7% 42099 ± 15% fio.write_bw_MBps
-81.1% 2109440 ± 16% fio.write_clat_90%_us
-80.4% 2211840 ± 16% fio.write_clat_95%_us
-65.4% 4026368 ± 7% fio.write_clat_99%_us
-81.5% 1937158 ± 16% fio.write_clat_mean_us
-43.3% 428848 ± 12% fio.write_clat_stddev
+363.7% 21049 ± 15% fio.write_iops
-71.3% 6686 ± 15% cpuidle.POLL.usage
-39.4 1.83 ± 2% mpstat.cpu.all.sys%
+44.1 46.82 mpstat.cpu.all.usr%
-8.1% 51.80 iostat.cpu.idle
-95.6% 1.82 ± 2% iostat.cpu.system
+1590.9% 46.39 iostat.cpu.user
-63.3% 433356 ± 29% numa-numastat.node0.local_node
-62.7% 448980 ± 24% numa-numastat.node0.numa_hit
-71.9% 388829 ± 28% numa-numastat.node1.local_node
-70.9% 404502 ± 23% numa-numastat.node1.numa_hit
-8.2% 51.75 vmstat.cpu.id
+2200.0% 46.00 vmstat.cpu.us
-20.3% 1676188 vmstat.memory.cache
-5.4% 1516 vmstat.system.cs
-80.7% 97541 meminfo.KReclaimable
-14.9% 9139470 meminfo.Memused
-89.8% 162150 ± 5% meminfo.PageTables
-80.7% 97541 meminfo.SReclaimable
-59.1% 280065 meminfo.Slab
-21.0% 45801 meminfo.max_used_kB
+12.1% 1394 turbostat.Avg_MHz
+5.3 49.96 turbostat.Busy%
-9.3% 49.05 turbostat.CPU%c1
+12.2% 61.50 turbostat.CoreTmp
+9.5% 61.75 turbostat.PkgTmp
+17.2% 289.48 turbostat.PkgWatt
+36.8% 188.59 turbostat.RAMWatt
+19.1% 2592 ± 6% slabinfo.avc_xperms_data.active_objs
+19.1% 2592 ± 6% slabinfo.avc_xperms_data.num_objs
-35.8% 1167 ± 5% slabinfo.dquot.active_objs
-35.8% 1167 ± 5% slabinfo.dquot.num_objs
+8.3% 17277 ± 2% slabinfo.kmalloc-512.active_objs
+8.1% 17385 ± 3% slabinfo.kmalloc-512.num_objs
+11.3% 878.00 ± 6% slabinfo.pool_workqueue.active_objs
-92.6% 56580 slabinfo.radix_tree_node.active_objs
-92.6% 1010 slabinfo.radix_tree_node.active_slabs
-92.6% 56646 slabinfo.radix_tree_node.num_objs
-92.6% 1010 slabinfo.radix_tree_node.num_slabs
-82.3% 54895 ± 6% numa-meminfo.node0.KReclaimable
+9.4% 19667753 ± 7% numa-meminfo.node0.MemFree
-92.3% 64794 ± 44% numa-meminfo.node0.PageTables
-82.3% 54895 ± 6% numa-meminfo.node0.SReclaimable
-61.0% 159577 ± 2% numa-meminfo.node0.Slab
+150.1% 200350 ± 93% numa-meminfo.node1.Inactive
+150.3% 200180 ± 93% numa-meminfo.node1.Inactive(anon)
-78.0% 42668 ± 10% numa-meminfo.node1.KReclaimable
+146.3% 208493 ± 91% numa-meminfo.node1.Mapped
-87.0% 97358 ± 22% numa-meminfo.node1.PageTables
-78.0% 42668 ± 10% numa-meminfo.node1.SReclaimable
-56.3% 120510 ± 3% numa-meminfo.node1.Slab
+9.4% 4918393 ± 7% numa-vmstat.node0.nr_free_pages
-92.3% 16203 ± 44% numa-vmstat.node0.nr_page_table_pages
-82.3% 13719 ± 6% numa-vmstat.node0.nr_slab_reclaimable
-33.9% 1096193 ± 18% numa-vmstat.node0.numa_hit
-33.9% 1080213 ± 17% numa-vmstat.node0.numa_local
+151.2% 50007 ± 93% numa-vmstat.node1.nr_inactive_anon
+147.1% 52152 ± 91% numa-vmstat.node1.nr_mapped
-87.0% 24268 ± 22% numa-vmstat.node1.nr_page_table_pages
-78.0% 10661 ± 10% numa-vmstat.node1.nr_slab_reclaimable
+151.2% 50006 ± 93% numa-vmstat.node1.nr_zone_inactive_anon
-33.2% 868151 ± 24% numa-vmstat.node1.numa_hit
-38.8% 696149 ± 28% numa-vmstat.node1.numa_local
+5.9% 172002 ± 8% numa-vmstat.node1.numa_other
+6.2% 825713 proc-vmstat.nr_active_anon
+2.7% 5208 proc-vmstat.nr_active_file
+8.4% 805220 proc-vmstat.nr_anon_pages
+8.0% 1497 proc-vmstat.nr_anon_transparent_hugepages
+4.1% 990957 proc-vmstat.nr_dirty_background_threshold
+4.1% 1984338 proc-vmstat.nr_dirty_threshold
+4.1% 9991832 proc-vmstat.nr_free_pages
+8.9% 103606 proc-vmstat.nr_inactive_anon
+9.3% 105877 proc-vmstat.nr_mapped
-89.8% 40520 ± 5% proc-vmstat.nr_page_table_pages
-80.6% 24386 proc-vmstat.nr_slab_reclaimable
+6.2% 825713 proc-vmstat.nr_zone_active_anon
+2.7% 5208 proc-vmstat.nr_zone_active_file
+8.9% 103606 proc-vmstat.nr_zone_inactive_anon
-66.5% 879152 ± 2% proc-vmstat.numa_hit
-67.3% 847841 ± 2% proc-vmstat.numa_local
+16.8% 23324 ± 13% proc-vmstat.numa_pages_migrated
-35.2% 36320 ± 6% proc-vmstat.pgactivate
-52.0% 1702505 proc-vmstat.pgalloc_normal
-90.3% 45213914 ± 6% proc-vmstat.pgfault
-52.4% 1650770 proc-vmstat.pgfree
+16.8% 23324 ± 13% proc-vmstat.pgmigrate_success
-91.3% 78860 ± 5% proc-vmstat.thp_fault_fallback
+514.0% 359.57 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.avg
+435.2% 28206 ± 36% sched_debug.cfs_rq:/.MIN_vruntime.max
+473.4% 3162 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.stddev
+162.5% 340.93 ± 13% sched_debug.cfs_rq:/.exec_clock.min
+513.9% 359.64 ± 37% sched_debug.cfs_rq:/.max_vruntime.avg
+435.2% 28212 ± 36% sched_debug.cfs_rq:/.max_vruntime.max
+473.3% 3163 ± 37% sched_debug.cfs_rq:/.max_vruntime.stddev
+64.9% 0.03 ± 23% sched_debug.cfs_rq:/.nr_spread_over.avg
-14.2% 467.53 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.avg
+37.4% 827802 sched_debug.cpu.avg_idle.avg
+140.5% 114924 ± 60% sched_debug.cpu.avg_idle.min
-44.7% 202505 ± 4% sched_debug.cpu.avg_idle.stddev
+825.6% 27.20 ± 18% sched_debug.cpu.clock.stddev
+825.6% 27.20 ± 18% sched_debug.cpu.clock_task.stddev
+293.4% 0.00 sched_debug.cpu.next_balance.stddev
-33.4% 10717 ± 14% sched_debug.cpu.sched_count.max
+92.4% 160.19 ± 31% sched_debug.cpu.sched_count.min
-9.8% 641.54 sched_debug.cpu.sched_goidle.avg
-33.9% 5302 ± 14% sched_debug.cpu.sched_goidle.max
-72.3% 6.31 ± 13% sched_debug.cpu.sched_goidle.min
-38.5% 4968 ± 21% sched_debug.cpu.ttwu_count.max
+105.0% 103.62 ± 43% sched_debug.cpu.ttwu_count.min
-33.3% 864.58 ± 15% sched_debug.cpu.ttwu_count.stddev
-12.4% 392.94 sched_debug.cpu.ttwu_local.avg
+57.7% 47.88 ± 36% sched_debug.cpu.ttwu_local.min
-33.1% 632.26 ± 11% sched_debug.cpu.ttwu_local.stddev
+1823.1% 100.00 ± 81% interrupts.CPU12.RES:Rescheduling_interrupts
+3934.1% 88.75 ± 62% interrupts.CPU13.RES:Rescheduling_interrupts
-67.8% 30.75 ± 44% interrupts.CPU13.TLB:TLB_shootdowns
255.00 ±143% interrupts.CPU15.RES:Rescheduling_interrupts
682.00 ±162% interrupts.CPU21.RES:Rescheduling_interrupts
+3025.0% 62.50 ± 40% interrupts.CPU22.RES:Rescheduling_interrupts
696.00 ±135% interrupts.CPU23.RES:Rescheduling_interrupts
+2702.6% 213.00 ±148% interrupts.CPU27.RES:Rescheduling_interrupts
+476.6% 184.50 ± 57% interrupts.CPU3.RES:Rescheduling_interrupts
551.50 ±166% interrupts.CPU33.RES:Rescheduling_interrupts
+679.2% 120.00 ± 84% interrupts.CPU5.RES:Rescheduling_interrupts
+2150.0% 103.50 ± 57% interrupts.CPU50.RES:Rescheduling_interrupts
+1232.2% 101.25 ± 55% interrupts.CPU51.RES:Rescheduling_interrupts
491.75 ±136% interrupts.CPU53.RES:Rescheduling_interrupts
+2120.2% 186.50 ± 87% interrupts.CPU55.RES:Rescheduling_interrupts
+1011.8% 84.50 ± 24% interrupts.CPU58.RES:Rescheduling_interrupts
+3307.9% 129.50 ± 65% interrupts.CPU59.RES:Rescheduling_interrupts
+15.3% 2918 ± 8% interrupts.CPU60.CAL:Function_call_interrupts
+6841.2% 236.00 ±110% interrupts.CPU60.RES:Rescheduling_interrupts
+5132.1% 146.50 ± 82% interrupts.CPU61.RES:Rescheduling_interrupts
+1859.8% 109.75 ± 38% interrupts.CPU63.RES:Rescheduling_interrupts
+2739.3% 79.50 ± 24% interrupts.CPU64.RES:Rescheduling_interrupts
+3301.8% 95.25 ± 47% interrupts.CPU66.RES:Rescheduling_interrupts
+8622.2% 157.00 ±105% interrupts.CPU69.RES:Rescheduling_interrupts
+63.1% 4143 ± 13% interrupts.CPU7.NMI:Non-maskable_interrupts
+63.1% 4143 ± 13% interrupts.CPU7.PMI:Performance_monitoring_interrupts
+3980.1% 318.25 ±127% interrupts.CPU7.RES:Rescheduling_interrupts
+40.2% 4092 ± 18% interrupts.CPU79.NMI:Non-maskable_interrupts
+40.2% 4092 ± 18% interrupts.CPU79.PMI:Performance_monitoring_interrupts
+1120.0% 61.00 ±111% interrupts.CPU8.RES:Rescheduling_interrupts
+58.8% 3968 ± 19% interrupts.CPU80.NMI:Non-maskable_interrupts
+58.8% 3968 ± 19% interrupts.CPU80.PMI:Performance_monitoring_interrupts
+56.6% 4590 ± 36% interrupts.CPU81.NMI:Non-maskable_interrupts
+56.6% 4590 ± 36% interrupts.CPU81.PMI:Performance_monitoring_interrupts
+40.6% 4026 ± 21% interrupts.CPU85.NMI:Non-maskable_interrupts
+40.6% 4026 ± 21% interrupts.CPU85.PMI:Performance_monitoring_interrupts
+42.5% 4106 ± 20% interrupts.CPU86.NMI:Non-maskable_interrupts
+42.5% 4106 ± 20% interrupts.CPU86.PMI:Performance_monitoring_interrupts
+2365.9% 162.75 ± 41% interrupts.CPU9.RES:Rescheduling_interrupts
+60.4% 4037 ± 21% interrupts.CPU90.NMI:Non-maskable_interrupts
+60.4% 4037 ± 21% interrupts.CPU90.PMI:Performance_monitoring_interrupts
+185.7% 56.00 ± 64% interrupts.CPU90.RES:Rescheduling_interrupts
+110.2% 96.25 ± 39% interrupts.CPU90.TLB:TLB_shootdowns
+63.8% 4062 ± 21% interrupts.CPU91.NMI:Non-maskable_interrupts
+63.8% 4062 ± 21% interrupts.CPU91.PMI:Performance_monitoring_interrupts
+46.3% 4016 ± 20% interrupts.CPU92.NMI:Non-maskable_interrupts
+46.3% 4016 ± 20% interrupts.CPU92.PMI:Performance_monitoring_interrupts
+469.6% 90.00 ± 73% interrupts.CPU92.RES:Rescheduling_interrupts
+61.0% 4020 ± 20% interrupts.CPU93.NMI:Non-maskable_interrupts
+61.0% 4020 ± 20% interrupts.CPU93.PMI:Performance_monitoring_interrupts
+679.1% 113.75 ± 94% interrupts.CPU93.RES:Rescheduling_interrupts
+47.7% 68.25 ± 41% interrupts.CPU93.TLB:TLB_shootdowns
+61.5% 4056 ± 18% interrupts.CPU94.NMI:Non-maskable_interrupts
+61.5% 4056 ± 18% interrupts.CPU94.PMI:Performance_monitoring_interrupts
+221.1% 65.50 ± 69% interrupts.CPU94.RES:Rescheduling_interrupts
-36.5% 321.75 ± 12% interrupts.IWI:IRQ_work_interrupts
+1527.5% 429.21 ± 5% perf-stat.i.MPKI
-86.6% 5.422e+08 ± 6% perf-stat.i.branch-instructions
+0.4 0.71 ± 7% perf-stat.i.branch-miss-rate%
-71.0% 3808633 ± 2% perf-stat.i.branch-misses
+3.0 43.77 ± 3% perf-stat.i.cache-miss-rate%
+150.0% 5.462e+08 ± 10% perf-stat.i.cache-misses
+141.9% 1.219e+09 ± 13% perf-stat.i.cache-references
-6.2% 1449 perf-stat.i.context-switches
+638.0% 46.28 ± 8% perf-stat.i.cpi
+9.2% 1.299e+11 perf-stat.i.cpu-cycles
-42.9% 508.33 ± 11% perf-stat.i.cycles-between-cache-misses
-85.8% 329151 ± 4% perf-stat.i.dTLB-load-misses
-84.4% 7.32e+08 ± 6% perf-stat.i.dTLB-loads
-1.5 1.24 ± 3% perf-stat.i.dTLB-store-miss-rate%
-87.3% 7140595 ± 6% perf-stat.i.dTLB-store-misses
-68.8% 5.692e+08 ± 9% perf-stat.i.dTLB-stores
-20.8 55.02 ± 2% perf-stat.i.iTLB-load-miss-rate%
-73.7% 3145209 ± 7% perf-stat.i.iTLB-load-misses
-22.2% 2533427 ± 3% perf-stat.i.iTLB-loads
-83.7% 2.883e+09 ± 7% perf-stat.i.instructions
-44.3% 956.76 ± 3% perf-stat.i.instructions-per-iTLB-miss
-81.5% 0.04 ± 7% perf-stat.i.ipc
-89.4% 216527 ± 6% perf-stat.i.minor-faults
-41.3 4.92 ± 8% perf-stat.i.node-load-miss-rate%
-83.5% 2075631 ± 23% perf-stat.i.node-load-misses
+654.5% 1.285e+08 ± 27% perf-stat.i.node-loads
+177.3% 1.096e+08 ± 35% perf-stat.i.node-store-misses
+442.8% 1.227e+08 ± 41% perf-stat.i.node-stores
-89.4% 216529 ± 6% perf-stat.i.page-faults
+1379.4% 421.53 ± 5% perf-stat.overall.MPKI
+0.4 0.71 ± 7% perf-stat.overall.branch-miss-rate%
+575.2% 45.40 ± 8% perf-stat.overall.cpi
-56.1% 240.54 ± 10% perf-stat.overall.cycles-between-cache-misses
-0.0 0.05 ± 4% perf-stat.overall.dTLB-load-miss-rate%
-1.8 1.24 ± 2% perf-stat.overall.dTLB-store-miss-rate%
-23.3 55.31 ± 2% perf-stat.overall.iTLB-load-miss-rate%
-38.2% 916.91 ± 4% perf-stat.overall.instructions-per-iTLB-miss
-85.1% 0.02 ± 7% perf-stat.overall.ipc
-40.5 1.61 ± 11% perf-stat.overall.node-load-miss-rate%
-96.7% 143888 ± 7% perf-stat.overall.path-length
-86.6% 5.393e+08 ± 6% perf-stat.ps.branch-instructions
-71.0% 3788024 ± 2% perf-stat.ps.branch-misses
+150.1% 5.441e+08 ± 10% perf-stat.ps.cache-misses
+142.0% 1.214e+09 ± 13% perf-stat.ps.cache-references
-6.2% 1442 perf-stat.ps.context-switches
+9.2% 1.294e+11 perf-stat.ps.cpu-cycles
-85.8% 327568 ± 4% perf-stat.ps.dTLB-load-misses
-84.4% 7.286e+08 ± 6% perf-stat.ps.dTLB-loads
-87.3% 7112891 ± 6% perf-stat.ps.dTLB-store-misses
-68.8% 5.663e+08 ± 9% perf-stat.ps.dTLB-stores
-73.7% 3131396 ± 7% perf-stat.ps.iTLB-load-misses
-22.2% 2522758 ± 2% perf-stat.ps.iTLB-loads
-83.7% 2.868e+09 ± 7% perf-stat.ps.instructions
-89.4% 215676 ± 6% perf-stat.ps.minor-faults
-83.5% 2063340 ± 23% perf-stat.ps.node-load-misses
+655.0% 1.28e+08 ± 27% perf-stat.ps.node-loads
+177.4% 1.092e+08 ± 35% perf-stat.ps.node-store-misses
+442.9% 1.222e+08 ± 41% perf-stat.ps.node-stores
-89.4% 215678 ± 6% perf-stat.ps.page-faults
-85.1% 5.985e+11 ± 7% perf-stat.total.instructions
+461.8% 40876 ± 6% softirqs.CPU0.RCU
+499.5% 44535 ± 9% softirqs.CPU1.RCU
-19.2% 73203 ± 3% softirqs.CPU1.TIMER
+494.3% 43809 ± 5% softirqs.CPU10.RCU
+597.8% 44523 ± 6% softirqs.CPU11.RCU
-63.9% 9707 ±101% softirqs.CPU11.SCHED
-22.9% 73876 ± 6% softirqs.CPU11.TIMER
+532.9% 43287 ± 6% softirqs.CPU12.RCU
+608.2% 43878 ± 7% softirqs.CPU13.RCU
+542.5% 40985 softirqs.CPU14.RCU
+601.4% 43724 ± 6% softirqs.CPU15.RCU
+594.0% 46802 ± 9% softirqs.CPU16.RCU
-25.5% 71817 ± 4% softirqs.CPU16.TIMER
+608.8% 46502 ± 10% softirqs.CPU17.RCU
+662.1% 49208 ± 3% softirqs.CPU18.RCU
-80.9% 4168 ± 10% softirqs.CPU18.SCHED
+550.6% 47074 ± 10% softirqs.CPU19.RCU
+487.8% 44225 ± 9% softirqs.CPU2.RCU
+486.7% 46778 ± 10% softirqs.CPU20.RCU
+615.9% 48903 ± 6% softirqs.CPU21.RCU
-57.4% 10314 ± 95% softirqs.CPU21.SCHED
+525.8% 43713 ± 10% softirqs.CPU22.RCU
+509.6% 41024 ± 5% softirqs.CPU23.RCU
+561.8% 50609 ± 7% softirqs.CPU24.RCU
+607.1% 46768 ± 7% softirqs.CPU25.RCU
+607.8% 44084 ± 10% softirqs.CPU26.RCU
+590.8% 44004 ± 9% softirqs.CPU27.RCU
+668.2% 45791 ± 12% softirqs.CPU28.RCU
+589.5% 44652 ± 7% softirqs.CPU29.RCU
+464.6% 45941 ± 9% softirqs.CPU3.RCU
+634.2% 46042 ± 7% softirqs.CPU30.RCU
+605.2% 44200 ± 10% softirqs.CPU31.RCU
+656.6% 44941 ± 6% softirqs.CPU32.RCU
+640.3% 44535 ± 11% softirqs.CPU33.RCU
+620.7% 42042 ± 11% softirqs.CPU34.RCU
+512.2% 45332 ± 6% softirqs.CPU35.RCU
+663.2% 45091 ± 6% softirqs.CPU36.RCU
+650.7% 46666 ± 7% softirqs.CPU37.RCU
+637.4% 43954 ± 4% softirqs.CPU38.RCU
+670.3% 44725 ± 4% softirqs.CPU39.RCU
+561.7% 44406 ± 9% softirqs.CPU4.RCU
+677.9% 44698 ± 6% softirqs.CPU40.RCU
+658.4% 45090 ± 4% softirqs.CPU41.RCU
+680.0% 46293 ± 5% softirqs.CPU42.RCU
+676.0% 45628 ± 5% softirqs.CPU43.RCU
+669.4% 45463 ± 5% softirqs.CPU44.RCU
+656.0% 45334 ± 5% softirqs.CPU45.RCU
+598.1% 45026 ± 5% softirqs.CPU46.RCU
+603.2% 39142 ± 9% softirqs.CPU47.RCU
+574.0% 46795 ± 4% softirqs.CPU48.RCU
+602.6% 43742 ± 11% softirqs.CPU49.RCU
+565.3% 44356 ± 3% softirqs.CPU5.RCU
-60.3% 10034 ±105% softirqs.CPU5.SCHED
+596.6% 43690 ± 10% softirqs.CPU50.RCU
+631.4% 43848 ± 10% softirqs.CPU51.RCU
+494.0% 41931 ± 18% softirqs.CPU52.RCU
+541.8% 40493 ± 14% softirqs.CPU53.RCU
+577.4% 43144 ± 10% softirqs.CPU54.RCU
+544.3% 41520 ± 9% softirqs.CPU55.RCU
+559.3% 41923 ± 12% softirqs.CPU56.RCU
+601.7% 44273 ± 9% softirqs.CPU57.RCU
+585.7% 43490 ± 11% softirqs.CPU58.RCU
+547.2% 40445 ± 13% softirqs.CPU59.RCU
+609.4% 44414 ± 7% softirqs.CPU6.RCU
+582.1% 43638 ± 14% softirqs.CPU60.RCU
+613.7% 45759 ± 6% softirqs.CPU61.RCU
+602.2% 43716 ± 3% softirqs.CPU62.RCU
+592.9% 42749 ± 11% softirqs.CPU63.RCU
+605.1% 46029 ± 9% softirqs.CPU64.RCU
+601.3% 45497 ± 10% softirqs.CPU65.RCU
+506.0% 38834 ± 6% softirqs.CPU66.RCU
+601.7% 45124 ± 10% softirqs.CPU67.RCU
+627.6% 45763 ± 9% softirqs.CPU68.RCU
+530.1% 41916 ± 10% softirqs.CPU69.RCU
+593.1% 44148 ± 5% softirqs.CPU7.RCU
-52.0% 13004 ± 75% softirqs.CPU7.SCHED
-31.0% 73147 ± 4% softirqs.CPU7.TIMER
+667.9% 48279 ± 8% softirqs.CPU70.RCU
+610.7% 50320 ± 6% softirqs.CPU71.RCU
+584.6% 42645 ± 5% softirqs.CPU72.RCU
+697.9% 44944 ± 10% softirqs.CPU73.RCU
+681.2% 44689 ± 11% softirqs.CPU74.RCU
+704.3% 45154 ± 8% softirqs.CPU75.RCU
+696.8% 45199 ± 9% softirqs.CPU76.RCU
+620.1% 41171 ± 12% softirqs.CPU77.RCU
+631.2% 41981 ± 9% softirqs.CPU78.RCU
+712.8% 45795 ± 9% softirqs.CPU79.RCU
+547.3% 42586 ± 9% softirqs.CPU8.RCU
+688.6% 43491 ± 7% softirqs.CPU80.RCU
-47.5% 15599 ± 75% softirqs.CPU80.SCHED
+548.7% 42319 ± 6% softirqs.CPU81.RCU
+653.2% 43416 ± 11% softirqs.CPU82.RCU
+695.1% 44452 ± 8% softirqs.CPU83.RCU
+715.9% 46238 ± 14% softirqs.CPU84.RCU
+647.1% 43931 ± 8% softirqs.CPU85.RCU
+640.7% 42780 ± 11% softirqs.CPU86.RCU
+573.2% 41245 ± 11% softirqs.CPU87.RCU
+652.8% 43124 ± 7% softirqs.CPU88.RCU
+683.2% 43993 ± 9% softirqs.CPU89.RCU
+561.2% 40033 ± 3% softirqs.CPU9.RCU
-13.7% 25050 ± 13% softirqs.CPU9.SCHED
+667.4% 44609 ± 8% softirqs.CPU90.RCU
+620.2% 43727 ± 9% softirqs.CPU91.RCU
+615.8% 43837 ± 9% softirqs.CPU92.RCU
+626.8% 42815 ± 9% softirqs.CPU93.RCU
+594.5% 44342 ± 7% softirqs.CPU94.RCU
+666.8% 44438 ± 8% softirqs.CPU95.RCU
+600.3% 4249152 softirqs.RCU
-11.4% 1538649 softirqs.SCHED
-63.2 1.32 ± 19% perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
-63.2 1.30 ± 19% perf-profile.calltrace.cycles-pp.ext4_dax_huge_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
-60.8 3.91 ± 19% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
-60.7 4.05 ± 19% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
-60.6 4.46 ± 18% perf-profile.calltrace.cycles-pp.page_fault
-60.6 4.37 ± 18% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
-60.6 4.44 ± 18% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
-53.7 0.93 ± 18% perf-profile.calltrace.cycles-pp.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault.__handle_mm_fault.handle_mm_fault
-49.7 0.18 ±173% perf-profile.calltrace.cycles-pp.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault.__handle_mm_fault
-49.2 0.17 ±173% perf-profile.calltrace.cycles-pp.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault.__do_fault
-49.2 0.17 ±173% perf-profile.calltrace.cycles-pp.lookup_memtype.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault.ext4_dax_huge_fault
-47.7 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.lookup_memtype.track_pfn_insert.__vm_insert_mixed.dax_iomap_pte_fault
-47.1 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.lookup_memtype.track_pfn_insert.__vm_insert_mixed
-6.2 0.00 perf-profile.calltrace.cycles-pp.jbd2__journal_start.ext4_dax_huge_fault.__do_fault.__handle_mm_fault.handle_mm_fault
-6.1 0.00 perf-profile.calltrace.cycles-pp.start_this_handle.jbd2__journal_start.ext4_dax_huge_fault.__do_fault.__handle_mm_fault
+0.6 1.31 ± 35% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
+0.6 1.31 ± 35% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
+0.6 1.33 ± 36% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
+0.6 1.33 ± 36% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
+0.6 1.33 ± 36% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
+0.9 0.91 ± 28% perf-profile.calltrace.cycles-pp.__mark_inode_dirty.generic_update_time.file_update_time.ext4_dax_huge_fault.__handle_mm_fault
+0.9 0.91 ± 27% perf-profile.calltrace.cycles-pp.generic_update_time.file_update_time.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault
+0.9 0.95 ± 27% perf-profile.calltrace.cycles-pp.file_update_time.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
+1.2 1.17 ± 19% perf-profile.calltrace.cycles-pp.dax_iomap_pmd_fault.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
+2.4 2.39 ± 24% perf-profile.calltrace.cycles-pp.ext4_dax_huge_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
+21.7 21.78 ± 13% perf-profile.calltrace.cycles-pp.get_io_u
+34.9 68.17 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
+35.6 68.39 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
+35.7 68.42 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
+36.4 69.27 ± 4% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
+36.4 69.27 ± 4% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
+36.4 69.27 ± 4% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
+37.1 70.60 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64
-63.2 1.33 ± 19% perf-profile.children.cycles-pp.__do_fault
-60.8 3.70 ± 19% perf-profile.children.cycles-pp.ext4_dax_huge_fault
-60.7 3.95 ± 19% perf-profile.children.cycles-pp.__handle_mm_fault
-60.7 4.09 ± 19% perf-profile.children.cycles-pp.handle_mm_fault
-60.6 4.42 ± 18% perf-profile.children.cycles-pp.__do_page_fault
-60.6 4.53 ± 18% perf-profile.children.cycles-pp.page_fault
-60.6 4.50 ± 18% perf-profile.children.cycles-pp.do_page_fault
-53.7 0.93 ± 18% perf-profile.children.cycles-pp.dax_iomap_pte_fault
-49.4 0.52 ± 21% perf-profile.children.cycles-pp.__vm_insert_mixed
-48.4 0.93 ± 19% perf-profile.children.cycles-pp.lookup_memtype
-48.4 0.94 ± 19% perf-profile.children.cycles-pp.track_pfn_insert
-47.9 0.25 ± 20% perf-profile.children.cycles-pp._raw_spin_lock
-47.1 0.05 ± 59% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
-6.0 0.37 ± 39% perf-profile.children.cycles-pp.start_this_handle
-6.0 0.49 ± 35% perf-profile.children.cycles-pp.jbd2__journal_start
-3.7 0.40 ± 21% perf-profile.children.cycles-pp.ext4_iomap_begin
-2.8 0.30 ± 29% perf-profile.children.cycles-pp.__ext4_journal_stop
-2.7 0.28 ± 30% perf-profile.children.cycles-pp.jbd2_journal_stop
-2.2 0.47 ± 30% perf-profile.children.cycles-pp._raw_read_lock
-1.9 0.03 ±100% perf-profile.children.cycles-pp.jbd2_transaction_committed
-1.6 0.17 ± 19% perf-profile.children.cycles-pp.__ext4_journal_start_sb
-1.6 0.15 ± 19% perf-profile.children.cycles-pp.ext4_journal_check_start
-1.5 0.04 ±100% perf-profile.children.cycles-pp.add_transaction_credits
-0.9 0.61 ± 21% perf-profile.children.cycles-pp.find_next_iomem_res
-0.9 0.64 ± 21% perf-profile.children.cycles-pp.pat_pagerange_is_ram
-0.9 0.63 ± 21% perf-profile.children.cycles-pp.walk_system_ram_range
-0.1 0.06 ± 59% perf-profile.children.cycles-pp.dax_insert_entry
-0.1 0.04 ± 58% perf-profile.children.cycles-pp.dax_unlock_entry
-0.1 0.03 ±100% perf-profile.children.cycles-pp._raw_spin_lock_irq
-0.1 0.10 ± 15% perf-profile.children.cycles-pp.rbt_memtype_lookup
-0.0 0.03 ±100% perf-profile.children.cycles-pp.___might_sleep
+0.0 0.12 ± 13% perf-profile.children.cycles-pp.ext4_es_lookup_extent
+0.0 0.12 ± 21% perf-profile.children.cycles-pp.kmem_cache_alloc
+0.0 0.07 ± 19% perf-profile.children.cycles-pp.__sb_start_write
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.serial8250_console_putchar
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.wait_for_xmitr
+0.1 0.08 ± 11% perf-profile.children.cycles-pp.serial8250_console_write
+0.1 0.08 ± 11% perf-profile.children.cycles-pp.uart_console_write
+0.1 0.08 ± 10% perf-profile.children.cycles-pp.irq_work_run_list
+0.1 0.08 ± 10% perf-profile.children.cycles-pp.console_unlock
+0.1 0.06 ± 14% perf-profile.children.cycles-pp.read
+0.1 0.06 ± 14% perf-profile.children.cycles-pp.__libc_fork
+0.1 0.06 ± 11% perf-profile.children.cycles-pp.search_binary_handler
+0.1 0.06 ± 11% perf-profile.children.cycles-pp.load_elf_binary
+0.1 0.07 ± 31% perf-profile.children.cycles-pp.io_serial_in
+0.1 0.07 ± 17% perf-profile.children.cycles-pp.exit_mmap
+0.1 0.07 ± 19% perf-profile.children.cycles-pp.ext4_inode_csum_set
+0.1 0.07 ± 12% perf-profile.children.cycles-pp.mmput
+0.1 0.22 ± 14% perf-profile.children.cycles-pp.ext4_map_blocks
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.irq_work_interrupt
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.smp_irq_work_interrupt
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.irq_work_run
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.printk
+0.1 0.07 ± 5% perf-profile.children.cycles-pp.vprintk_emit
+0.1 0.07 ± 14% perf-profile.children.cycles-pp.__schedule
+0.1 0.08 ± 19% perf-profile.children.cycles-pp.__wake_up_common_lock
+0.1 0.10 ± 24% perf-profile.children.cycles-pp.ext4_data_block_valid
+0.1 0.09 ± 19% perf-profile.children.cycles-pp.native_write_msr
+0.1 0.09 ± 21% perf-profile.children.cycles-pp.kthread
+0.1 0.09 ± 12% perf-profile.children.cycles-pp.__do_execve_file
+0.1 0.09 ± 12% perf-profile.children.cycles-pp.execve
+0.1 0.09 ± 24% perf-profile.children.cycles-pp.ret_from_fork
+0.1 0.19 ± 15% perf-profile.children.cycles-pp.__perf_sw_event
+0.1 0.09 ± 13% perf-profile.children.cycles-pp.__x64_sys_execve
+0.1 0.09 ± 28% perf-profile.children.cycles-pp.vmacache_find
+0.1 0.09 ± 37% perf-profile.children.cycles-pp.__run_perf_stat
+0.1 0.09 ± 37% perf-profile.children.cycles-pp.cmd_stat
+0.1 0.09 ± 37% perf-profile.children.cycles-pp.process_interval
+0.1 0.09 ± 37% perf-profile.children.cycles-pp.read_counters
+0.1 0.10 ± 27% perf-profile.children.cycles-pp.ext4_data_block_valid_rcu
+0.1 0.10 ± 24% perf-profile.children.cycles-pp.find_vma
+0.1 0.10 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
+0.1 0.10 ± 29% perf-profile.children.cycles-pp.get_next_timer_interrupt
+0.1 0.18 ± 15% perf-profile.children.cycles-pp.___perf_sw_event
+0.1 0.11 ± 33% perf-profile.children.cycles-pp.do_wait
+0.1 0.18 ± 23% perf-profile.children.cycles-pp.dax_wake_entry
+0.1 0.13 ± 31% perf-profile.children.cycles-pp.__do_sys_wait4
+0.1 0.13 ± 31% perf-profile.children.cycles-pp.kernel_wait4
+0.1 0.13 ± 31% perf-profile.children.cycles-pp.__waitpid
+0.1 0.14 ± 28% perf-profile.children.cycles-pp.vfs_read
+0.1 0.56 ± 6% perf-profile.children.cycles-pp.native_irq_return_iret
+0.1 0.14 ± 36% perf-profile.children.cycles-pp.update_load_avg
+0.1 0.14 ± 29% perf-profile.children.cycles-pp.ksys_read
+0.1 0.25 ± 23% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty
+0.1 0.14 ± 23% perf-profile.children.cycles-pp.__ext4_get_inode_loc
+0.1 0.15 ± 30% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
+0.2 0.29 ± 16% perf-profile.children.cycles-pp.grab_mapping_entry
+0.2 0.21 ± 17% perf-profile.children.cycles-pp.get_unlocked_entry
+0.2 0.22 ± 49% perf-profile.children.cycles-pp.writen
+0.2 0.22 ± 49% perf-profile.children.cycles-pp.__GI___libc_write
+0.2 0.22 ± 47% perf-profile.children.cycles-pp.record__pushfn
+0.2 0.23 ± 47% perf-profile.children.cycles-pp.perf_mmap__push
+0.2 0.20 ± 47% perf-profile.children.cycles-pp.__generic_file_write_iter
+0.2 0.20 ± 47% perf-profile.children.cycles-pp.generic_perform_write
+0.2 0.23 ± 42% perf-profile.children.cycles-pp.vfs_write
+0.2 0.21 ± 48% perf-profile.children.cycles-pp.generic_file_write_iter
+0.2 0.22 ± 44% perf-profile.children.cycles-pp.new_sync_write
+0.2 0.24 ± 48% perf-profile.children.cycles-pp.record__mmap_read_evlist
+0.2 0.23 ± 43% perf-profile.children.cycles-pp.ksys_write
+0.2 0.20 ± 22% perf-profile.children.cycles-pp.ext4_reserve_inode_write
+0.2 0.20 ± 20% perf-profile.children.cycles-pp.xas_find_conflict
+0.2 0.23 ± 36% perf-profile.children.cycles-pp.tick_nohz_next_event
+0.2 0.31 ± 48% perf-profile.children.cycles-pp.cmd_record
+0.3 0.28 ± 35% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
+0.3 0.40 ± 31% perf-profile.children.cycles-pp.__libc_start_main
+0.3 0.40 ± 31% perf-profile.children.cycles-pp.main
+0.3 0.40 ± 31% perf-profile.children.cycles-pp.run_builtin
+0.3 0.94 ± 26% perf-profile.children.cycles-pp.__mark_inode_dirty
+0.3 0.78 ± 27% perf-profile.children.cycles-pp.ext4_dirty_inode
+0.3 0.47 ± 23% perf-profile.children.cycles-pp.ext4_mark_inode_dirty
+0.3 0.42 ± 38% perf-profile.children.cycles-pp.task_tick_fair
+0.4 0.98 ± 26% perf-profile.children.cycles-pp.file_update_time
+0.4 0.93 ± 27% perf-profile.children.cycles-pp.generic_update_time
+0.5 0.46 ± 20% perf-profile.children.cycles-pp.vmf_insert_pfn_pmd
+0.5 0.57 ± 40% perf-profile.children.cycles-pp.scheduler_tick
+0.5 0.58 ± 34% perf-profile.children.cycles-pp.menu_select
+0.6 1.33 ± 36% perf-profile.children.cycles-pp.start_kernel
+0.7 0.94 ± 42% perf-profile.children.cycles-pp.update_process_times
+0.8 0.91 ± 15% perf-profile.children.cycles-pp.do_syscall_64
+0.8 0.99 ± 42% perf-profile.children.cycles-pp.tick_sched_handle
+0.8 0.91 ± 15% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
+0.8 1.05 ± 42% perf-profile.children.cycles-pp.tick_sched_timer
+1.0 1.25 ± 38% perf-profile.children.cycles-pp.__hrtimer_run_queues
+1.2 1.17 ± 19% perf-profile.children.cycles-pp.dax_iomap_pmd_fault
+1.4 1.80 ± 40% perf-profile.children.cycles-pp.hrtimer_interrupt
+2.1 2.58 ± 41% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
+2.1 2.69 ± 40% perf-profile.children.cycles-pp.apic_timer_interrupt
+21.4 21.84 ± 13% perf-profile.children.cycles-pp.get_io_u
+34.9 68.17 ± 4% perf-profile.children.cycles-pp.intel_idle
+36.3 69.73 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
+36.3 69.74 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
+36.4 69.27 ± 4% perf-profile.children.cycles-pp.start_secondary
+37.1 70.60 ± 4% perf-profile.children.cycles-pp.secondary_startup_64
+37.1 70.60 ± 4% perf-profile.children.cycles-pp.cpu_startup_entry
+37.1 70.60 ± 4% perf-profile.children.cycles-pp.do_idle
-46.8 0.05 ± 59% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
-3.3 0.13 ± 47% perf-profile.self.cycles-pp.start_this_handle
-2.7 0.17 ± 39% perf-profile.self.cycles-pp.jbd2_journal_stop
-2.2 0.46 ± 31% perf-profile.self.cycles-pp._raw_read_lock
-1.6 0.14 ± 22% perf-profile.self.cycles-pp.ext4_journal_check_start
-1.5 0.04 ±100% perf-profile.self.cycles-pp.add_transaction_credits
-0.8 0.22 ± 23% perf-profile.self.cycles-pp._raw_spin_lock
-0.7 0.39 ± 18% perf-profile.self.cycles-pp.find_next_iomem_res
-0.1 0.03 ±100% perf-profile.self.cycles-pp.ext4_iomap_begin
-0.1 0.10 ± 15% perf-profile.self.cycles-pp.rbt_memtype_lookup
-0.0 0.03 ±100% perf-profile.self.cycles-pp.___might_sleep
+0.1 0.06 ± 11% perf-profile.self.cycles-pp.ext4_es_lookup_extent
+0.1 0.09 ± 19% perf-profile.self.cycles-pp.native_write_msr
+0.1 0.13 ± 29% perf-profile.self.cycles-pp.ext4_mark_iloc_dirty
+0.1 0.14 ± 18% perf-profile.self.cycles-pp.___perf_sw_event
+0.1 0.09 ± 24% perf-profile.self.cycles-pp.ext4_data_block_valid_rcu
+0.1 0.09 ± 28% perf-profile.self.cycles-pp.vmacache_find
+0.1 0.10 ± 22% perf-profile.self.cycles-pp.kmem_cache_alloc
+0.1 0.12 ± 27% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
+0.1 0.18 ± 23% perf-profile.self.cycles-pp.dax_wake_entry
+0.1 0.56 ± 6% perf-profile.self.cycles-pp.native_irq_return_iret
+0.2 0.16 ± 18% perf-profile.self.cycles-pp.xas_find_conflict
+0.2 0.18 ± 29% perf-profile.self.cycles-pp.menu_select
+21.1 21.52 ± 14% perf-profile.self.cycles-pp.get_io_u
+34.9 68.17 ± 4% perf-profile.self.cycles-pp.intel_idle
have been estimated based on internal Intel analysis and are provided
purposes only. Any difference in system hardware or software
configuration may affect actual performance.

Attachments:

(No filename) (137.48 kB)
config-5.4.0-rc4-00001-ga70e8083a91b1 (203.86 kB)
job-script (8.14 kB)
job.yaml (5.76 kB)
reproduce (878.00 B)
Download all attachments