Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     kernel test robot <yujie.liu@intel.com>
CC:     Yu Zhao <yuzhao@google.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Hildenbrand <david@redhat.com>,
        David Howells <dhowells@redhat.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        SeongJae Park <sj@kernel.org>,
        Sven Schnelle <svens@linux.ibm.com>,
        Will Deacon <will@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "oe-lkp@lists.linux.dev" <oe-lkp@lists.linux.dev>,
        "lkp@intel.com" <lkp@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "ying.huang@intel.com" <ying.huang@intel.com>,
        "feng.tang@intel.com" <feng.tang@intel.com>,
        "zhengjun.xing@linux.intel.com" <zhengjun.xing@linux.intel.com>,
        "fengwei.yin@intel.com" <fengwei.yin@intel.com>
Subject: Re: [linus:master] will-it-scale.per_thread_ops -40.2% regression in
 mmap1 benchmark
Thread-Topic: [linus:master] will-it-scale.per_thread_ops -40.2% regression in
 mmap1 benchmark
Thread-Index: AQHZE5Dg5PRNV+lkn0WD/IKH1tst9K51gsqA
Date:   Mon, 19 Dec 2022 18:09:07 +0000
Message-ID: <20221219180857.u6opzhqqbbfxdj3h@revolver>
References: <202212191714.524e00b3-yujie.liu@intel.com>
In-Reply-To: <202212191714.524e00b3-yujie.liu@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?FsKv921CgEWz6kQsudrpVDKttXyJ+ljqphaUbhnS7+ZQricSYKD0E8YmrSGg?=
 =?us-ascii?Q?VDtlYytzLTgREfSvLUxcp9tB96M4YWdvaM9lXO0lh7mK7LNWQxofDR+K/Lc2?=
 =?us-ascii?Q?TprTOpG7WXPOepMIpjfKCOg39CpaxyDXlKNA9Nn5YMyrcsXFwwmAcNBtU/X3?=
 =?us-ascii?Q?3iRge4w7mZzpLzkcDF3d9rigThsWK1ZFE73HRqoBB82SozouIgHr6kBKEsef?=
 =?us-ascii?Q?t/rETWoK5yDKp0d26J6g1QUD1a0+Bue1++Uh9y3AesBDYQ0t4ETgmrNz28AJ?=
 =?us-ascii?Q?weufDmlxgMDjp6S0Sal/UqBjzUIgKgewmcV/y9J+H+m7kFIykCScqJEpbOr9?=
 =?us-ascii?Q?34fK5a5R+JiaKSLvKumAOrH0CzBGwyYd+0GtRQo1HaWb3sIAN6Gi4jNjH1N6?=
 =?us-ascii?Q?6pXuzYFniMkmWDL71wRzGGfMGrJtr9XbfMbQ4OJUiG9Bei2TApQ+siTlV3p8?=
 =?us-ascii?Q?oznUJ3RVwXfQzZIUSitxZdKnfyew+F5oWHmxLrf8IjRaWgbmCFjbRcdgLoXP?=
 =?us-ascii?Q?OmNAnRhQuMcHvAlB+qlwW1FDWRFvV0dwqQdy4EppDq8ywUuQmbLF9nXlrx0R?=
 =?us-ascii?Q?qe4lpSOGR2FB7zpqh/z2IVVIEg4GY7Kb4kqjxCyzsXgtpB4UdnezgEMH8y4Y?=
 =?us-ascii?Q?hxPB1ZWj1c2g1ThfzwYk2DUTumegUOtmOkDMORKU8PYDougiGyI6fMvtEPrg?=
 =?us-ascii?Q?8nPzLBMHJL5V4RvTn7j/bYj34dUfh7l2GGMCV0gR5p1d/I0ZGdeMrryVD0Rb?=
 =?us-ascii?Q?oSBZWWOEhVouOYmFGejJeLfFa5rlL13qtfoFYRx4Nt+JDSkZw4dWIl8s1hGx?=
 =?us-ascii?Q?VLg31zJ6SRiJLPGyGIgJdJyi5PZ/Oc8is5IYD6KUc7vH34Z2J+sI2tJkbcnN?=
 =?us-ascii?Q?LmiyUuiV/B9x3iX26sV8IZ0ysqiMcAA7aQTCa/AwehztjVYFy8dTI3Rxwj5X?=
 =?us-ascii?Q?x+dhEKe1Rbn9DlbZCqyK6hGKWW/GfE1CMWhVU7Z2/dZKeOdYxoisg0Gkcxa1?=
 =?us-ascii?Q?98Z2A/mr7gPj2SPCC8SjuA5Hy93rcdtcG5eAbiCmMUC8+N7T71SSlwgAPgvT?=
 =?us-ascii?Q?h3wyO+4/f/FN/w5HjorzsmXNKffMorJKBlJeFQdk96NcJI9TQ+S5AbMWSnV7?=
 =?us-ascii?Q?FdsUwWogBwDwERPKk/WsdR4ATwmzvccnHsQ1K6r1XkVp/rycjHeyVf/RyU7Z?=
 =?us-ascii?Q?gOn4neM6MQ0xGSu/m8c2PXRiLxlZ0LSrB5cElIDoiTshr7WpLZvombcE5/Ip?=
 =?us-ascii?Q?FOJDOCFt6KUrpHfLz/eGZ9ybm/qX2IxtAuYf+aVo+6HqxEKzOZeNRvUTUnZV?=
 =?us-ascii?Q?QHbvx0vVs7ywbx72t9b4OFy4Kn1WsrhIQyPNgQoSXQdB+fkg9BysWkZoHRDt?=
 =?us-ascii?Q?8XOZxR8Oj1b/jy5FLdb3eCza7dSnSlWdKzntgcbRka+jGw+XEKTUIQdQSutX?=
 =?us-ascii?Q?Z423uUz5+IyjYyVFP2KEdZUx3tjuMoAYB6wtZcjDi4L1MBqzqy8QsqXjeN3u?=
 =?us-ascii?Q?O/b1XtGmL98DBQ/8LbYa3Y0EiCoLExY8e5fijvsD/lbefUD2n/a/EiWtRSn/?=
 =?us-ascii?Q?P95mtXBKXuKDkAm4hf4zfj3tAajwUtbAzBa87Z5w3w0d+FYS9otKbjfArNNB?=
 =?us-ascii?Q?vA=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <FF669F782FE5374A90DA084B3FBB2106@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88267418-3984-4797-bdeb-08dae1ec1f42
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Dec 2022 18:09:07.9189
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: iXhOuHFznarZWk2RwmpQFKMfzxKjab6tap8zXF4LKXkVP6fffhKHrki0UMKFC8Ppm93BcRx4Q7D3cLn8MR0EsA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR10MB4161
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 phishscore=0
 mlxlogscore=999 bulkscore=0 malwarescore=0 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2212190161
X-Proofpoint-ORIG-GUID: Dl0cA_YWP-Hnr8PJhtnp2og-soEusnJv
X-Proofpoint-GUID: Dl0cA_YWP-Hnr8PJhtnp2og-soEusnJv
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

* kernel test robot <yujie.liu@intel.com> [221219 05:01]:
> Greetings,
>=20
> FYI, we noticed a -40.2% regression of will-it-scale.per_thread_ops
> between commit 524e00b36e8c and e15e06a83923 of mainline

Thank you for running this test.

We are aware of this regression.  The regression was taken as an
acceptable trade off for the gain on the read speed.  Applications
perform more reads than writes to the VMA tree.  The overfall
performance on real applications is either even or faster with the maple
tree.  This can be seen in the kernel build times where forked processes
are short lived and would be close to the worst case scenario.

This isn't to say we can't do better, and we are constantly working
towards faster performance.  Please continue to report on the
performance.

Looking specifically at mmap1, it is mapping then unmapping in a tight
loop.  The regression would be expected, considering the internals of
what is going on, but I don't believe this would ever happen in an
application that is doing what it is supposed to be doing.

If you find a real application that shows a performance regression,
please let us know.

>=20
> 524e00b36e8c5 mm: remove rb tree.
> 0c563f1480435 proc: remove VMA rbtree use from nommu
> d0cf3dd47f0d5 damon: convert __damon_va_three_regions to use the VMA iter=
ator
> c9dbe82cb99db kernel/fork: use maple tree for dup_mmap() during forking
> 3499a13168da6 mm/mmap: use maple tree for unmapped_area{_topdown}
> 7fdbd37da5c6f mm/mmap: use the maple tree for find_vma_prev() instead of =
the rbtree
> be8432e7166ef mm/mmap: use the maple tree in find_vma() instead of the rb=
tree.
> 2e3af1db17442 mmap: use the VMA iterator in count_vma_pages_range()
> f39af05949a42 mm: add VMA iterator
> d4af56c5c7c67 mm: start tracking VMAs with maple tree
> e15e06a839232 lib/test_maple_tree: add testing for maple tree
>=20
> in testcase: will-it-scale
> on test machine: 104 threads 2 sockets (Skylake) with 192G memory
> with following parameters:
>=20
> 	nr_task: 50%
> 	mode: thread
> 	test: mmap1
> 	cpufreq_governor: performance
>=20
> test-description: Will It Scale takes a testcase and runs it from 1 throu=
gh to n parallel copies to see if the testcase will scale. It builds both a=
 process and threads based test in order to see any differences between the=
 two.
> test-url: https://github.com/antonblanchard/will-it-scale
>=20
>=20
> We couldn't find out the commit that introduced this regression because
> some of above commits failed to boot during bisection, but looks it is
> related with maple tree code. Please check following details:

It is interesting that these issues were not detected by myself or other
build bots.  Perhaps there is a configuration option that wasn't tested.
In any rate, all of the listed commits were in preparation for the last
commit to remove the rb tree.  Regardless of which commit introduced the
regression, it is the fact that that the maple tree is slower on writes
that is being detected.

Thanks,
Liam