On Tue, Sep 12, 2023 at 11:14:42PM +0800, Tang, Feng wrote:
[...]
> >
> > Well that's the problem. Since I can't run the reproducer, there's
> > nothing I can do to troubleshoot the problem myself.
>
> We dug more into the perf and other profiling data from 0Day server
> running this case, and it seems that the new simple_offset_add()
> called by shmem_mknod() brings extra cost related with slab,
> specifically the 'radix_tree_node', which cause the regression.
>
> Here is some slabinfo diff for commit a2e459555c5f and its parent:
>
> 23a31d87645c6527 a2e459555c5f9da3e619b7e47a6
> ---------------- ---------------------------
>
> 26363 +40.2% 36956 slabinfo.radix_tree_node.active_objs
> 941.00 +40.4% 1321 slabinfo.radix_tree_node.active_slabs
> 26363 +40.3% 37001 slabinfo.radix_tree_node.num_objs
> 941.00 +40.4% 1321 slabinfo.radix_tree_node.num_slabs
>
> Also the perf profile show some difference
>
> 0.01 ±223% +0.1 0.10 ± 28% pp.self.shuffle_freelist
> 0.00 +0.1 0.11 ± 40% pp.self.xas_create
> 0.00 +0.1 0.12 ± 27% pp.self.xas_find_marked
> 0.00 +0.1 0.14 ± 18% pp.self.xas_alloc
> 0.03 ±103% +0.1 0.17 ± 29% pp.self.xas_descend
> 0.00 +0.2 0.16 ± 23% pp.self.xas_expand
> 0.10 ± 22% +0.2 0.27 ± 16% pp.self.rcu_segcblist_enqueue
> 0.92 ± 35% +0.3 1.22 ± 11% pp.self.kmem_cache_free
> 0.00 +0.4 0.36 ± 16% pp.self.xas_store
> 0.32 ± 30% +0.4 0.71 ± 12% pp.self.__call_rcu_common
> 0.18 ± 27% +0.5 0.65 ± 8% pp.self.kmem_cache_alloc_lru
> 0.36 ± 79% +0.6 0.96 ± 15% pp.self.__slab_free
> 0.00 +0.8 0.80 ± 14% pp.self.radix_tree_node_rcu_free
> 0.00 +1.0 1.01 ± 16% pp.self.radix_tree_node_ctor
>
> Some perf profile from a2e459555c5f is:
>
> - 17.09% 0.09% singleuser [kernel.kallsyms] [k] path_openat
> - 16.99% path_openat
> - 12.23% open_last_lookups
> - 11.33% lookup_open.isra.0
> - 9.05% shmem_mknod
> - 5.11% simple_offset_add
> - 4.95% __xa_alloc_cyclic
> - 4.88% __xa_alloc
> - 4.76% xas_store
> - xas_create
> - 2.40% xas_expand.constprop.0
> - 2.01% xas_alloc
> - kmem_cache_alloc_lru
> - 1.28% ___slab_alloc
> - 1.22% allocate_slab
> - 1.19% shuffle_freelist
> - 1.04% setup_object
> radix_tree_node_ctor
>
> Please let me know if you need more info.
>
> >
> > Is there any hope in getting this reproducer to run on Fedora?
>
> Myself haven't succeeded to reproduce it locally, will keep trying
> it tomorrow.
It can be reproduced on a local machien with CentOS 9 (similar to
Fedora ?), and some steps as:
* download source code
$ wget https://sourceforge.net/projects/aimbench/files/aim-suite9/Initial%20release/s9110.tar.Z
* untar the file and run 'make'
* create 2 files 's9workfile' and 'test.config'
$ cat s9workfile
# @(#) s9workfile:1.2 1/22/96 00:00:00
# AIM Independent Resource Benchmark - Suite IX Workfile
FILESIZE: 5M
disk_src
$ cat test.config
test
disk_src
200s
/dev/shm/
* run the test with cmd "./singleuser -nl < test.config"
The test case here is 'disk_src', so I picked one file 'disk_src.c'
and attached it for quick reference.
The kernel config of my CentOS is different from what 0Day used, so
the perf-profile and peformance score are a little different, but
the regression trend is the same, that commit a2e459555c5f has about
20% drop.
Also the test platform doesn't matter, I tried on several generation
of Xeon servers which can all reproduce it.
Thanks,
Feng