LinuxLists.cc - [fsnotify] c738fbabb0: will-it-scale.per_process

2020-07-21 00:17:59

Subject: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

Greeting,

FYI, we

commit: https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
href="https://github.com/antonblanchard/will-it-scale">https://github.com/antonblanchard/will-it-scale
to that, the commit also has significant impact on the following tests:
--+---------------------------------------------------------------------------+
change | will-it-scale: will-it-scale.per_process_ops -9.6% regression |
| 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| cpufreq_governor=performance |
| mode=process |
| nr_task=16 |
| test=open2 |
| ucode=0x5002f01 |
--+---------------------------------------------------------------------------+
change | will-it-scale: will-it-scale.per_process_ops -9.8% regression |
| 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| cpufreq_governor=performance |
| mode=process |
| nr_task=50% |
| test=open2 |
| ucode=0x5002f01 |
--+---------------------------------------------------------------------------+
the issue, kindly add following tag
kernel test robot <[email protected]>
--------------------------------------------------------------------------------->
href="https://github.com/intel/lkp-tests.git">https://github.com/intel/lkp-tests.git
bin/lkp install job.yaml # job file is attached in this email
========================================================================
governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
ce/x86_64-rhel-8.3/process/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/open1/will-it-scale/0x5002f01
("fsnotify: Rearrange fast path to minimise overhead when there is no watcher")
("fsnotify: fold fsnotify() call into fsnotify_parent()")
c738fbabb0ff62d0f9a9572e56e
---------------------------
%change %stddev
| \
-9.5% 208520 will-it-scale.per_process_ops
-9.5% 3336327 will-it-scale.workload
-0.0 0.13 ± 3% mpstat.cpu.all.usr%
+1.3% 19175 vmstat.system.in
+30.6% 1732214 ± 8% cpuidle.C1.time
+84.9% 30624 ± 15% cpuidle.C1.usage
+86.7% 2.33 ± 20% sched_debug.cfs_rq:/.nr_spread_over.max
-31.7% 6.40 ± 11% sched_debug.cpu.clock.stddev
-24.3% 2487 ± 11% slabinfo.fsnotify_mark_connector.active_objs
-24.3% 2487 ± 11% slabinfo.fsnotify_mark_connector.num_objs
-1.5% 92776 proc-vmstat.nr_slab_unreclaimable
-8.2% 13486706 proc-vmstat.numa_hit
-8.2% 13486604 proc-vmstat.numa_local
-8.5% 51651910 proc-vmstat.pgalloc_normal
-8.5% 51766282 proc-vmstat.pgfree
+148.1% 2801 ± 32% numa-vmstat.node0.nr_inactive_anon
+148.1% 2801 ± 32% numa-vmstat.node0.nr_zone_inactive_anon
-31.9% 1508 ± 30% numa-vmstat.node2.nr_mapped
-82.5% 5004 ± 67% numa-vmstat.node3.nr_active_anon
-82.9% 4877 ± 70% numa-vmstat.node3.nr_anon_pages
-65.5% 91.75 ± 20% numa-vmstat.node3.nr_page_table_pages
-18.0% 18797 ± 10% numa-vmstat.node3.nr_slab_unreclaimable
-82.5% 5004 ± 67% numa-vmstat.node3.nr_zone_active_anon
-25.9% 481856 ± 11% numa-vmstat.node3.numa_hit
-34.1% 390815 ± 13% numa-vmstat.node3.numa_local
+143.3% 11414 ± 31% numa-meminfo.node0.Inactive
+148.0% 11207 ± 31% numa-meminfo.node0.Inactive(anon)
-32.3% 5911 ± 27% numa-meminfo.node2.Mapped
-82.1% 20520 ± 68% numa-meminfo.node3.Active
-82.5% 20016 ± 67% numa-meminfo.node3.Active(anon)
-84.3% 12813 ± 92% numa-meminfo.node3.AnonHugePages
-82.9% 19510 ± 70% numa-meminfo.node3.AnonPages
-19.5% 729240 ± 6% numa-meminfo.node3.MemUsed
-65.6% 368.25 ± 20% numa-meminfo.node3.PageTables
-18.0% 75192 ± 10% numa-meminfo.node3.SUnreclaim
-17.0% 99920 ± 9% numa-meminfo.node3.Slab
+926.2% 24739 ± 68% interrupts.CPU124.LOC:Local_timer_interrupts
-73.3% 5143 ±128% interrupts.CPU16.LOC:Local_timer_interrupts
-88.9% 1217 ±138% interrupts.CPU179.LOC:Local_timer_interrupts
+547.7% 3752 ± 91% interrupts.CPU20.LOC:Local_timer_interrupts
+1060.3% 250538 ± 26% interrupts.CPU3.LOC:Local_timer_interrupts
+1182.1% 4714 ±109% interrupts.CPU31.LOC:Local_timer_interrupts
+1195.1% 4821 ± 64% interrupts.CPU32.LOC:Local_timer_interrupts
+1211.7% 4971 ±100% interrupts.CPU33.LOC:Local_timer_interrupts
-73.3% 389.25 ± 9% interrupts.CPU45.LOC:Local_timer_interrupts
+399.6% 1770 ± 95% interrupts.CPU53.LOC:Local_timer_interrupts
+2367.8% 8754 ± 95% interrupts.CPU59.LOC:Local_timer_interrupts
+2115.8% 8370 ±134% interrupts.CPU68.LOC:Local_timer_interrupts
+597.9% 198364 ± 45% interrupts.CPU7.LOC:Local_timer_interrupts
-93.3% 305.75 ± 10% interrupts.CPU71.LOC:Local_timer_interrupts
-82.8% 791.75 ± 72% interrupts.CPU75.LOC:Local_timer_interrupts
-82.1% 51605 ±128% interrupts.CPU99.LOC:Local_timer_interrupts
-42.7% 10626 ± 44% softirqs.CPU105.RCU
-46.1% 58018 ± 39% softirqs.CPU105.TIMER
+301.0% 9431 ± 82% softirqs.CPU106.RCU
+259.2% 53325 ± 75% softirqs.CPU106.TIMER
-35.5% 8896 ± 10% softirqs.CPU179.TIMER
-18.0% 8578 ± 7% softirqs.CPU180.TIMER
+486.1% 16622 ± 25% softirqs.CPU3.RCU
+378.9% 81110 ± 23% softirqs.CPU3.TIMER
+23.7% 12080 ± 22% softirqs.CPU31.TIMER
+27.3% 13610 ± 14% softirqs.CPU48.TIMER
+140.9% 6274 ± 88% softirqs.CPU5.SCHED
+12.4% 10638 ± 12% softirqs.CPU52.TIMER
+40.8% 13320 ± 41% softirqs.CPU68.TIMER
+305.3% 13404 ± 38% softirqs.CPU7.RCU
+590.0% 17354 ± 84% softirqs.CPU7.SCHED
+299.2% 72406 ± 26% softirqs.CPU7.TIMER
-17.6% 9503 ± 2% softirqs.CPU75.TIMER
-12.3% 9649 ± 5% softirqs.CPU81.TIMER
-78.6% 4062 ± 91% softirqs.CPU99.RCU
-73.9% 23943 ± 79% softirqs.CPU99.TIMER

will-it-scale.per_process_ops

+------------------------------------------------------------------+
+..+.+..+.+.+.. |
: +.+.+..+ |
: : |
:.+.. .+ |
+ +.+..+ |
|
|
O O O O |
|
O O O O O |
O O O O O O O O |
O O O O O O |
O O O |
+------------------------------------------------------------------+

**********************************************************************************
192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
========================================================================
governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
ce/x86_64-rhel-8.3/process/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/open2/will-it-scale/0x5002f01
("fsnotify: Rearrange fast path to minimise overhead when there is no watcher")
("fsnotify: fold fsnotify() call into fsnotify_parent()")
c738fbabb0ff62d0f9a9572e56e
---------------------------
%change %stddev
| \
-9.6% 208858 will-it-scale.per_process_ops
-9.6% 3341737 will-it-scale.workload
-8.1% 12819132 numa-numastat.node0.local_node
-8.1% 12819137 numa-numastat.node0.numa_hit
+1048.3% 2253 ± 58% numa-vmstat.node0.nr_inactive_anon
+981.8% 2290 ± 56% numa-vmstat.node0.nr_shmem
+1048.3% 2253 ± 58% numa-vmstat.node0.nr_zone_inactive_anon
-42.0% 1605 ± 37% numa-vmstat.node2.nr_mapped
+24.0% 9916 ± 4% slabinfo.eventpoll_pwq.active_objs
+24.0% 9916 ± 4% slabinfo.eventpoll_pwq.num_objs
+10.4% 14044 slabinfo.shmem_inode_cache.active_objs
+10.1% 14134 slabinfo.shmem_inode_cache.num_objs
+882.8% 9088 ± 57% numa-meminfo.node0.Inactive
+1046.6% 9015 ± 58% numa-meminfo.node0.Inactive(anon)
+980.2% 9165 ± 56% numa-meminfo.node0.Shmem
+123.6% 98992 ± 76% numa-meminfo.node1.AnonHugePages
-42.0% 6422 ± 37% numa-meminfo.node2.Mapped
+3.9% 868.50 proc-vmstat.nr_page_table_pages
-8.0% 13532306 proc-vmstat.numa_hit
-8.0% 13531870 proc-vmstat.numa_local
-8.3% 51824913 proc-vmstat.pgalloc_normal
-8.2% 51934860 proc-vmstat.pgfree
-35.5% 0.18 ± 15% sched_debug.cfs_rq:/.nr_spread_over.avg
-50.7% 1.54 ± 24% sched_debug.cfs_rq:/.nr_spread_over.max
-50.1% 0.29 ± 29% sched_debug.cfs_rq:/.nr_spread_over.stddev
-28.4% 624.33 ± 25% sched_debug.cpu.nr_switches.min
-20.6% 65730 ± 13% sched_debug.cpu.sched_goidle.max
-90.7% 897.75 ± 3% softirqs.CPU104.RCU
+176.3% 110969 ± 25% softirqs.CPU107.TIMER
-89.9% 1573 ± 9% softirqs.CPU11.RCU
-84.7% 11155 ± 10% softirqs.CPU11.TIMER
+258.5% 8287 ±105% softirqs.CPU112.SCHED
+63.5% 15558 ± 32% softirqs.CPU121.TIMER
-47.0% 9170 ± 6% softirqs.CPU147.TIMER
-27.1% 9050 ± 12% softirqs.CPU179.TIMER
+43.7% 13310 ± 21% softirqs.CPU27.TIMER
+155.8% 59950 ± 56% softirqs.CPU7.TIMER
+42.5% 261827 ± 15% interrupts.CPU101.LOC:Local_timer_interrupts
+174.9% 300490 interrupts.CPU107.LOC:Local_timer_interrupts
-98.3% 3401 ± 74% interrupts.CPU11.LOC:Local_timer_interrupts
+342.6% 1331 ±115% interrupts.CPU117.LOC:Local_timer_interrupts
-97.1% 505.75 ± 69% interrupts.CPU147.LOC:Local_timer_interrupts
+346.2% 2120 ± 75% interrupts.CPU18.LOC:Local_timer_interrupts
-91.2% 258.00 ± 45% interrupts.CPU183.LOC:Local_timer_interrupts
+998.1% 3357 ± 93% interrupts.CPU30.LOC:Local_timer_interrupts
+160.7% 810.75 ± 63% interrupts.CPU34.LOC:Local_timer_interrupts
-89.7% 318.00 ± 7% interrupts.CPU37.LOC:Local_timer_interrupts
+52.4% 421.25 ± 19% interrupts.CPU42.LOC:Local_timer_interrupts
-43.5% 309.50 ± 7% interrupts.CPU44.LOC:Local_timer_interrupts
+476.4% 4043 ± 70% interrupts.CPU56.LOC:Local_timer_interrupts
+284.7% 175395 ± 67% interrupts.CPU7.LOC:Local_timer_interrupts
+637.5% 44.25 ± 94% interrupts.CPU72.RES:Rescheduling_interrupts
**********************************************************************************
192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
========================================================================
governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
ce/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap3/open2/will-it-scale/0x5002f01
("fsnotify: Rearrange fast path to minimise overhead when there is no watcher")
("fsnotify: fold fsnotify() call into fsnotify_parent()")
c738fbabb0ff62d0f9a9572e56e
---------------------------
%change %stddev
| \
-9.8% 23925 will-it-scale.per_process_ops
-9.8% 2296858 will-it-scale.workload
-0.1 0.43 ± 4% mpstat.cpu.all.soft%
-4.0% 38.28 ± 2% boot-time.boot
-4.9% 6180 ± 2% boot-time.idle
-27.3% 24087303 ± 17% cpuidle.C1E.time
-21.0% 75408 ± 12% cpuidle.C1E.usage
+39.3% 138.40 ± 17% sched_debug.cpu.clock.stddev
+44.7% 0.00 ± 28% sched_debug.cpu.next_balance.stddev
-21.7% 7522 sched_debug.cpu.ttwu_count.max
-15.2% 827.50 ± 6% sched_debug.cpu.ttwu_count.stddev
-24.4% 1281 ± 8% slabinfo.dmaengine-unmap-16.active_objs
-24.4% 1281 ± 8% slabinfo.dmaengine-unmap-16.num_objs
-18.7% 765.00 ± 7% slabinfo.skbuff_fclone_cache.active_objs
-18.7% 765.00 ± 7% slabinfo.skbuff_fclone_cache.num_objs
+123.5% 111141 ± 40% numa-meminfo.node1.Active
+127.9% 111137 ± 40% numa-meminfo.node1.Active(anon)
+183.7% 67649 ± 44% numa-meminfo.node1.AnonHugePages
+130.3% 110911 ± 40% numa-meminfo.node1.AnonPages
+11.6% 812918 ± 7% numa-meminfo.node1.MemUsed
-13.2% 2388164 ± 3% numa-numastat.node1.local_node
-13.2% 2388260 ± 3% numa-numastat.node1.numa_hit
-12.0% 2428082 ± 4% numa-numastat.node2.local_node
-12.0% 2428249 ± 4% numa-numastat.node2.numa_hit
-11.7% 2438785 ± 4% numa-numastat.node3.local_node
-11.7% 2438967 ± 4% numa-numastat.node3.numa_hit
-11.7% 9073752 proc-vmstat.numa_hit
-11.7% 9073178 proc-vmstat.numa_local
-76.3% 1379 ± 99% proc-vmstat.numa_pages_migrated
-90.3% 2912 ±137% proc-vmstat.numa_pte_updates
-12.3% 34068635 proc-vmstat.pgalloc_normal
-1.5% 1084378 proc-vmstat.pgfault
-12.3% 34116999 proc-vmstat.pgfree
-76.3% 1379 ± 99% proc-vmstat.pgmigrate_success
+900.3% 39258 ± 85% numa-vmstat.node0.numa_other
+128.0% 27807 ± 40% numa-vmstat.node1.nr_active_anon
+130.4% 27753 ± 40% numa-vmstat.node1.nr_anon_pages
+33.2% 1690 ± 33% numa-vmstat.node1.nr_mapped
+128.0% 27807 ± 40% numa-vmstat.node1.nr_zone_active_anon
-10.3% 1642592 ± 6% numa-vmstat.node2.numa_hit
-9.9% 1568287 ± 6% numa-vmstat.node2.numa_local
-9.8% 1643294 ± 5% numa-vmstat.node3.numa_hit
-10.4% 1551925 ± 6% numa-vmstat.node3.numa_local
-46.1% 66.75 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts
+15.3% 156593 ± 7% interrupts.CPU107.LOC:Local_timer_interrupts
-21.5% 117714 ± 5% interrupts.CPU111.LOC:Local_timer_interrupts
-13.8% 135670 ± 13% interrupts.CPU112.LOC:Local_timer_interrupts
+21.4% 589.00 ± 5% interrupts.CPU12.CAL:Function_call_interrupts
+58.4% 149451 ± 4% interrupts.CPU124.LOC:Local_timer_interrupts
+81.1% 162547 ± 8% interrupts.CPU138.LOC:Local_timer_interrupts
+57.9% 146526 ± 15% interrupts.CPU142.LOC:Local_timer_interrupts
+20.5% 167753 ± 3% interrupts.CPU147.LOC:Local_timer_interrupts
+40.6% 600.00 ± 29% interrupts.CPU148.CAL:Function_call_interrupts
-22.7% 437.75 ± 10% interrupts.CPU15.CAL:Function_call_interrupts
-30.8% 52.25 ± 13% interrupts.CPU15.RES:Rescheduling_interrupts
+30.5% 158362 ± 8% interrupts.CPU155.LOC:Local_timer_interrupts
+44.1% 179592 ± 10% interrupts.CPU156.LOC:Local_timer_interrupts
+17.7% 499.50 ± 12% interrupts.CPU16.CAL:Function_call_interrupts
-52.4% 32.75 ± 13% interrupts.CPU162.RES:Rescheduling_interrupts
-16.2% 420.50 ± 2% interrupts.CPU164.CAL:Function_call_interrupts
-45.0% 420.50 ± 3% interrupts.CPU165.CAL:Function_call_interrupts
-65.3% 31.25 ± 19% interrupts.CPU165.RES:Rescheduling_interrupts
+32.1% 176637 ± 5% interrupts.CPU167.LOC:Local_timer_interrupts
+50.7% 187215 ± 9% interrupts.CPU177.LOC:Local_timer_interrupts
+22.0% 171763 ± 9% interrupts.CPU180.LOC:Local_timer_interrupts
+33.0% 192841 ± 7% interrupts.CPU186.LOC:Local_timer_interrupts
-54.2% 35.25 ± 5% interrupts.CPU186.RES:Rescheduling_interrupts
+43.9% 665.00 ± 25% interrupts.CPU189.CAL:Function_call_interrupts
+14.2% 475.75 ± 9% interrupts.CPU19.CAL:Function_call_interrupts
+24.5% 179585 ± 6% interrupts.CPU190.LOC:Local_timer_interrupts
-10.4% 177947 ± 3% interrupts.CPU22.LOC:Local_timer_interrupts
-24.9% 190117 ± 5% interrupts.CPU28.LOC:Local_timer_interrupts
-26.8% 176499 ± 4% interrupts.CPU42.LOC:Local_timer_interrupts
-14.1% 419.00 interrupts.CPU44.CAL:Function_call_interrupts
-12.1% 180159 ± 4% interrupts.CPU51.LOC:Local_timer_interrupts
-15.1% 180387 ± 5% interrupts.CPU54.LOC:Local_timer_interrupts
-25.5% 71.75 ± 9% interrupts.CPU6.RES:Rescheduling_interrupts
-10.4% 172746 ± 7% interrupts.CPU72.LOC:Local_timer_interrupts
-9.1% 172040 ± 6% interrupts.CPU75.LOC:Local_timer_interrupts
-22.4% 174209 ± 4% interrupts.CPU78.LOC:Local_timer_interrupts
-27.0% 155382 ± 11% interrupts.CPU81.LOC:Local_timer_interrupts
-14.7% 167819 ± 4% interrupts.CPU83.LOC:Local_timer_interrupts
-15.3% 166862 ± 10% interrupts.CPU84.LOC:Local_timer_interrupts
-21.2% 150944 ± 8% interrupts.CPU90.LOC:Local_timer_interrupts
-16.0% 161414 ± 10% interrupts.CPU93.LOC:Local_timer_interrupts
-18.8% 160294 ± 7% interrupts.CPU94.LOC:Local_timer_interrupts
-39.1% 51.00 ± 8% interrupts.CPU97.RES:Rescheduling_interrupts
+13.1% 71681 ± 7% softirqs.CPU100.TIMER
+17.7% 39933 ± 6% softirqs.CPU107.RCU
+13.2% 65866 ± 7% softirqs.CPU107.TIMER
-21.7% 27835 ± 3% softirqs.CPU111.RCU
-17.7% 51841 ± 4% softirqs.CPU111.TIMER
-11.8% 58335 ± 11% softirqs.CPU112.TIMER
+14.2% 40140 ± 7% softirqs.CPU119.RCU
+110.0% 36984 ± 6% softirqs.CPU124.RCU
+45.5% 62274 ± 4% softirqs.CPU124.TIMER
+60.7% 33193 ± 16% softirqs.CPU130.RCU
+95.8% 40282 ± 7% softirqs.CPU138.RCU
+63.4% 66563 ± 7% softirqs.CPU138.TIMER
+42.0% 36043 ± 13% softirqs.CPU139.RCU
+13.5% 55672 softirqs.CPU14.RCU
+23.9% 40272 ± 5% softirqs.CPU147.RCU
+17.9% 68785 ± 2% softirqs.CPU147.TIMER
+17.9% 58621 softirqs.CPU15.RCU
+11.4% 90763 softirqs.CPU15.TIMER
+47.4% 38461 ± 13% softirqs.CPU155.RCU
+24.4% 65149 ± 7% softirqs.CPU155.TIMER
+36.8% 72800 ± 8% softirqs.CPU156.TIMER
-33.1% 5046 ± 4% softirqs.CPU162.SCHED
+42.8% 43295 ± 6% softirqs.CPU167.RCU
+17.5% 41681 ± 9% softirqs.CPU168.RCU
+50.6% 40701 ± 4% softirqs.CPU174.RCU
+57.1% 46613 ± 11% softirqs.CPU177.RCU
+42.8% 75671 ± 8% softirqs.CPU177.TIMER
+25.2% 42690 ± 7% softirqs.CPU179.RCU
+21.2% 70860 ± 6% softirqs.CPU179.TIMER
+30.2% 43553 ± 10% softirqs.CPU180.RCU
+20.2% 70753 ± 8% softirqs.CPU180.TIMER
+35.7% 47765 ± 7% softirqs.CPU186.RCU
+28.7% 77301 ± 6% softirqs.CPU186.TIMER
+18.0% 42852 ± 10% softirqs.CPU188.RCU
+11.7% 70261 ± 8% softirqs.CPU188.TIMER
+33.2% 44938 ± 7% softirqs.CPU190.RCU
+20.7% 72657 ± 6% softirqs.CPU190.TIMER
+18.1% 48502 ± 14% softirqs.CPU191.RCU
-8.0% 74547 ± 3% softirqs.CPU22.TIMER
-22.1% 77778 ± 4% softirqs.CPU28.TIMER
-32.4% 4619 ± 12% softirqs.CPU39.SCHED
-24.0% 72722 ± 3% softirqs.CPU42.TIMER
-13.2% 44920 ± 4% softirqs.CPU51.RCU
-13.5% 73979 ± 5% softirqs.CPU54.TIMER
-8.2% 71835 ± 6% softirqs.CPU72.TIMER
-7.6% 71394 ± 5% softirqs.CPU75.TIMER
-18.8% 72701 ± 3% softirqs.CPU78.TIMER
-23.8% 64917 ± 10% softirqs.CPU81.TIMER
-14.8% 42097 ± 5% softirqs.CPU83.RCU
-12.8% 69335 ± 3% softirqs.CPU83.TIMER
-18.1% 41342 ± 11% softirqs.CPU84.RCU
-13.3% 68951 ± 9% softirqs.CPU84.TIMER
-24.5% 36673 ± 9% softirqs.CPU90.RCU
-18.2% 63329 ± 7% softirqs.CPU90.TIMER
-15.6% 40405 ± 9% softirqs.CPU93.RCU
-13.3% 67512 ± 8% softirqs.CPU93.TIMER
-20.5% 39403 ± 9% softirqs.CPU94.RCU
-17.2% 66551 ± 6% softirqs.CPU94.TIMER
-78.3% 4842 ±122% softirqs.NET_RX
have been estimated based on internal Intel analysis and are provided
purposes only. Any difference in system hardware or software
configuration may affect actual performance.

Attachments:

(No filename) (28.17 kB)
config-5.8.0-rc4-00085-gc738fbabb0ff6 (160.83 kB)
job-script (7.50 kB)
job.yaml (5.12 kB)
reproduce (347.00 B)
Download all attachments

2020-07-21 16:00:19

by Amir Goldstein

[permalink] [raw]

Subject: Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On Tue, Jul 21, 2020 at 3:15 AM kernel test robot <[email protected]> wrote:
>
> Greeting,
>
> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops due to commit:
>
>
> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify() call into fsnotify_parent()")

Strange, that's a pretty dumb patch moving some inlined code from one
function to
another (assuming there are no fsnotify marks in this test).

Unless I am missing something the only thing that changes slightly is
an extra d_inode(file->f_path.dentry) deference.
I can get rid of it.

Is it possible to ask for a re-test with fix patch (attached)?

Thanks,
Amir.

Attachments:

fsnotify-pass-inode-to-fsnotify_parent.patch.txt (2.05 kB)

2020-07-24 02:48:41

by Chen, Rong A

[permalink] [raw]

Subject: Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On 7/21/20 11:59 PM, Amir Goldstein wrote:
> On Tue, Jul 21, 2020 at 3:15 AM kernel test robot <[email protected]> wrote:
>> Greeting,
>>
>> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops due to commit:
>>
>>
>> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify() call into fsnotify_parent()")
> Strange, that's a pretty dumb patch moving some inlined code from one
> function to
> another (assuming there are no fsnotify marks in this test).
>
> Unless I am missing something the only thing that changes slightly is
> an extra d_inode(file->f_path.dentry) deference.
> I can get rid of it.
>
> Is it possible to ask for a re-test with fix patch (attached)?

Hi Amir,

We failed to apply this patch, could you tell us the base commit or the
base branch?

Best Regards,
Rong Chen

2020-07-24 03:48:40

by Amir Goldstein

[permalink] [raw]

Subject: Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On Fri, Jul 24, 2020 at 5:45 AM Rong Chen <[email protected]> wrote:
>
>
>
> On 7/21/20 11:59 PM, Amir Goldstein wrote:
> > On Tue, Jul 21, 2020 at 3:15 AM kernel test robot <[email protected]> wrote:
> >> Greeting,
> >>
> >> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops due to commit:
> >>
> >>
> >> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify() call into fsnotify_parent()")
> > Strange, that's a pretty dumb patch moving some inlined code from one
> > function to
> > another (assuming there are no fsnotify marks in this test).
> >
> > Unless I am missing something the only thing that changes slightly is
> > an extra d_inode(file->f_path.dentry) deference.
> > I can get rid of it.
> >
> > Is it possible to ask for a re-test with fix patch (attached)?
>
> Hi Amir,
>
> We failed to apply this patch, could you tell us the base commit or the
> base branch?
>

Hi Rong,

The patch is applied on top of the reported offending commit:
c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify()
call into fsnotify_parent()")

I pushed it to my github:
https://github.com/amir73il/linux/commits/for_lkp

Thanks,
Amir.

2020-07-26 11:54:27

by Amir Goldstein

[permalink] [raw]

Subject: Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On Fri, Jul 24, 2020 at 6:47 AM Amir Goldstein <[email protected]> wrote:
>
> On Fri, Jul 24, 2020 at 5:45 AM Rong Chen <[email protected]> wrote:
> >
> >
> >
> > On 7/21/20 11:59 PM, Amir Goldstein wrote:
> > > On Tue, Jul 21, 2020 at 3:15 AM kernel test robot <[email protected]> wrote:
> > >> Greeting,
> > >>
> > >> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops due to commit:
> > >>
> > >>
> > >> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify() call into fsnotify_parent()")
> > > Strange, that's a pretty dumb patch moving some inlined code from one
> > > function to
> > > another (assuming there are no fsnotify marks in this test).
> > >
> > > Unless I am missing something the only thing that changes slightly is
> > > an extra d_inode(file->f_path.dentry) deference.
> > > I can get rid of it.
> > >
> > > Is it possible to ask for a re-test with fix patch (attached)?
> >
> > Hi Amir,
> >
> > We failed to apply this patch, could you tell us the base commit or the
> > base branch?
> >
>
> Hi Rong,
>
> The patch is applied on top of the reported offending commit:
> c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify()
> call into fsnotify_parent()")
>
> I pushed it to my github:
> https://github.com/amir73il/linux/commits/for_lkp
>

FWIW, I tried reproducing the reported regression on a local machine.

I ran the test twice on each of the branch commits:

26dc3d2bff62 fsnotify: pass inode to fsnotify_parent()
c738fbabb0ff fsnotify: fold fsnotify() call into fsnotify_parent()
71d734103edf fsnotify: Rearrange fast path to minimise overhead when
there is no watcher
47aaabdedf36 fanotify: Avoid softlockups when reading many events

Not only did I not observe a regression with the reported commit,
but there was a slight improvement. And then there yet was another
improvement with the fix commit on top of it.

But it could be that I am doing something wrong, because I have zero
millage with LKP.

=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-7/performance/defconfig/process/16/ubuntu/amir-lkp/open1/will-it-scale

commit:
47aaabdedf366ac5894c7fddec388832f0d8193e
71d734103edfa2b4c6657578a3082ee0e51d767e
c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a
26dc3d2bff623768cbbd0c8053ddd6390fd828d2

47aaabdedf366ac5 71d734103edfa2b4c6657578a30
c738fbabb0ff62d0f9a9572e56e 26dc3d2bff623768cbbd0c8053d
---------------- ---------------------------
--------------------------- ---------------------------
fail:runs %reproduction fail:runs %reproduction
fail:runs %reproduction fail:runs
| | | | |
| |
45:2 -555% 34:2 -807% 29:2
-996% 25:2 dmesg.timestamp:last
45:2 -555% 34:2 -807% 29:2
-996% 25:2 kmsg.timestamp:last
%stddev %change %stddev %change
%stddev %change %stddev
\ | \ | \
| \
1097404 +1.7% 1116452 +2.7% 1126533
+3.5% 1135663 will-it-scale.16.processes
0.02 ± 60% +20.0% 0.03 ± 66% +20.0% 0.03 ±
66% -20.0% 0.02 ± 50% will-it-scale.16.processes_idle
68587 +1.7% 69778 +2.7% 70408
+3.5% 70978 will-it-scale.per_process_ops

Thanks,
Amir.

2020-07-27 02:09:17

by Xing Zhengjun

[permalink] [raw]

Subject: Re: [LKP] Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On 7/24/2020 10:44 AM, Rong Chen wrote:
>
>
> On 7/21/20 11:59 PM, Amir Goldstein wrote:
>> On Tue, Jul 21, 2020 at 3:15 AM kernel test robot
>> <[email protected]> wrote:
>>> Greeting,
>>>
>>> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops
>>> due to commit:
>>>
>>>
>>> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold
>>> fsnotify() call into fsnotify_parent()")
>> Strange, that's a pretty dumb patch moving some inlined code from one
>> function to
>> another (assuming there are no fsnotify marks in this test).
>>
>> Unless I am missing something the only thing that changes slightly is
>> an extra d_inode(file->f_path.dentry) deference.
>> I can get rid of it.
>>
>> Is it possible to ask for a re-test with fix patch (attached)?
>

I apply the fix patch, the regression still exists.
=========================================================================================
tbox_group/testcase/rootfs/kconfig/compiler/nr_task/mode/test/cpufreq_governor/ucode:

lkp-csl-2ap2/will-it-scale/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3/gcc-9/16/process/open1/performance/0x5002f01

commit:
71d734103edfa2b4c6657578a3082ee0e51d767e
c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a
5c32fe90f2a57e7c4da06be51f705aec6affceb6 (the commit which the fix
patch apply based on)
7f66797f773621d0ef6718df0ef2cf849814d114 (the fix patch)

71d734103edfa2b4 c738fbabb0ff62d0f9a9572e56e 5c32fe90f2a57e7c4da06be51f7
7f66797f773621d0ef6718df0ef
---------------- --------------------------- ---------------------------
---------------------------
%stddev %change %stddev %change
%stddev %change %stddev
\ | \ | \
| \
229940 -9.8% 207333 -13.0% 199996
-11.7% 202927 will-it-scale.per_process_ops
3679048 -9.8% 3317347 -13.0% 3199942
-11.7% 3246851 will-it-scale.workload

> Hi Amir,
>
> We failed to apply this patch, could you tell us the base commit or the
> base branch?
>
> Best Regards,
> Rong Chen
> _______________________________________________
> LKP mailing list -- [email protected]
> To unsubscribe send an email to [email protected]

--
Zhengjun Xing

2020-07-27 11:07:30

by Jan Kara

[permalink] [raw]

Subject: Re: [fsnotify] c738fbabb0: will-it-scale.per_process_ops -9.5% regression

On Sun 26-07-20 14:52:47, Amir Goldstein wrote:
> On Fri, Jul 24, 2020 at 6:47 AM Amir Goldstein <[email protected]> wrote:
> >
> > On Fri, Jul 24, 2020 at 5:45 AM Rong Chen <[email protected]> wrote:
> > >
> > >
> > >
> > > On 7/21/20 11:59 PM, Amir Goldstein wrote:
> > > > On Tue, Jul 21, 2020 at 3:15 AM kernel test robot <[email protected]> wrote:
> > > >> Greeting,
> > > >>
> > > >> FYI, we noticed a -9.5% regression of will-it-scale.per_process_ops due to commit:
> > > >>
> > > >>
> > > >> commit: c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify() call into fsnotify_parent()")
> > > > Strange, that's a pretty dumb patch moving some inlined code from one
> > > > function to
> > > > another (assuming there are no fsnotify marks in this test).
> > > >
> > > > Unless I am missing something the only thing that changes slightly is
> > > > an extra d_inode(file->f_path.dentry) deference.
> > > > I can get rid of it.
> > > >
> > > > Is it possible to ask for a re-test with fix patch (attached)?
> > >
> > > Hi Amir,
> > >
> > > We failed to apply this patch, could you tell us the base commit or the
> > > base branch?
> > >
> >
> > Hi Rong,
> >
> > The patch is applied on top of the reported offending commit:
> > c738fbabb0ff62d0f9a9572e56e65d05a1b34c6a ("fsnotify: fold fsnotify()
> > call into fsnotify_parent()")
> >
> > I pushed it to my github:
> > https://github.com/amir73il/linux/commits/for_lkp
> >
>
> FWIW, I tried reproducing the reported regression on a local machine.
>
> I ran the test twice on each of the branch commits:
>
> 26dc3d2bff62 fsnotify: pass inode to fsnotify_parent()
> c738fbabb0ff fsnotify: fold fsnotify() call into fsnotify_parent()
> 71d734103edf fsnotify: Rearrange fast path to minimise overhead when
> there is no watcher
> 47aaabdedf36 fanotify: Avoid softlockups when reading many events
>
> Not only did I not observe a regression with the reported commit,
> but there was a slight improvement. And then there yet was another
> improvement with the fix commit on top of it.

I suspect this may be closely related to code generation, code cacheline
alignment etc. and thus depends heavily on a particular compiler version
and CPU. I've checked the commit myself and I agree it looks innocent so
for these reasons, I'm not particularly worried about this regression.

Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR