Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Date:   Wed, 13 Sep 2023 11:20:23 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Aaron Lu <aaron.lu@intel.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        "Dietmar Eggemann" <dietmar.eggemann@arm.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>,
        David Vernet <void@manifault.com>,
        "Nitin Tekchandani" <nitin.tekchandani@intel.com>,
        Daniel Jordan <daniel.m.jordan@oracle.com>,
        Tim Chen <tim.c.chen@intel.com>,
        Swapnil Sapkal <Swapnil.Sapkal@amd.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 1/1] sched/fair: ratelimit update to tg->load_avg
Message-ID: <ZQEqd6bNOOF9SoWI@chenyu5-mobl2.ccr.corp.intel.com>
References: <20230912065808.2530-1-aaron.lu@intel.com>
 <20230912065808.2530-2-aaron.lu@intel.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20230912065808.2530-2-aaron.lu@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?p7kXXUic9C8iuicB0QTHhwQMbBOSdIjXUydfT0tg/mt6HN0rDrX4sCQZ7l?=
 =?iso-8859-1?Q?bqP6qAQhUJgE0LGqd+9e5msTxGq8svJZHpR0mHGuc/1I8Fksow21Kj5ywB?=
 =?iso-8859-1?Q?27Z2NT6Pa+QumLhtRl+fg7aloDIfmCN85B/OJFtEvr7X4E8Vw3Cw0gFV7B?=
 =?iso-8859-1?Q?fxdUxmKKb/37lWclOm6Ir/moChY+e8QS1fG7amKJp3vgrnjpk4E9PUI9Nq?=
 =?iso-8859-1?Q?pObnAsBcHhxcHcXJuXieo6uC/iQBcI2O6q6k9evC+rKmiTkI/8vn34du3c?=
 =?iso-8859-1?Q?pNTJVXYPOpV9GVak9uo1NNTXkH5NnsyCMfF2c3koTKDrrKE8M+R2RWs3xD?=
 =?iso-8859-1?Q?mr3UPoDjCXVBwuqHKBsD2zomrSyA6pFjUebR0bSakTFjP4t04iuYGPEyOF?=
 =?iso-8859-1?Q?iFJIpW2G9jHY3LKIL7zz1RwXeU+ssmYtDSANXEgLOH/+fu8bra9bTn25OU?=
 =?iso-8859-1?Q?K89UBS/l+TfcBK0nGEkNcZG51/xqYcf0IglePoVLBVXcrGpFAUyvKSXhYx?=
 =?iso-8859-1?Q?Lcl0ndF7AfXXb8Yaow/lx7oFLqvi2IbTzg7C0wnJ5ij+BZeGl/PH9+BaZl?=
 =?iso-8859-1?Q?xnGuzp+r5x5Bb/YjEC3sDeedbHONi+OVYv1tpAkw9l0GnPgVomdeWTJCgs?=
 =?iso-8859-1?Q?huPBLT2ZEnr5k9tR5R04Xtt6pv/yShcsanFXVH58fJpbZN1xc6EB2p0s5F?=
 =?iso-8859-1?Q?48IrmMOTtjBlHnbLKgEUM9jdWbfHpkSPT37F932zrtoU2zqctRkl2ChpmP?=
 =?iso-8859-1?Q?/hhUW05kPkENm8QBw44oq5Zyi7881j6hveVfUyN1E+OTqb4q49UBRaXZzZ?=
 =?iso-8859-1?Q?PnWfXmq1xlh/al8oVdOsd5xwl2EvnP177sN2UnFmAmg1RqSNJwxWHJavr4?=
 =?iso-8859-1?Q?Qxlgr3SyF0Ng9t5nh2/zGWC8bIwwpU/355z9Oep8YWGkWnPgRGOUcnYiW0?=
 =?iso-8859-1?Q?/ZPfBpGLhiw0cBtKWK5wMTa+TG5uWIYZwuGScmnmgCS1qs0tXNoIQGIjRO?=
 =?iso-8859-1?Q?c3gf/y9P68iCqfn23ZPMJhmt8bKhM22EG/XDjiWvy01HEg+sShjNquwbI/?=
 =?iso-8859-1?Q?I6oLdq9mgG+3P1yt+Yd6dWTZ0/ZqBewJ3r8yg6Z4Ux0+891UIsSlyqUfZ4?=
 =?iso-8859-1?Q?At11SJU2+oNXuKqS1JQQ1EtnqT7upOaMPpKL47nZefHmz81FsxKDjaTW2w?=
 =?iso-8859-1?Q?YaL+0CmDyooQkE25RczoUa8/WZDV0bz+rHx4gG9WunRYlmRlnCBidjc/a+?=
 =?iso-8859-1?Q?cnMSrmHvsCX+ZbT2jVwgBd31PS9dAH9dlX+xNR3aZt6M1HIHAdKKL5NOZu?=
 =?iso-8859-1?Q?RgGiWq1NryTCydIqtNcSX5sWRbLzMFlaOJk8WayglmMwRfXyrz70gMcjEQ?=
 =?iso-8859-1?Q?DOWyemVwz4whdF4FqthgM7IKkqfqEyPRdENvGY8T1ubQ0kIwMzRcxZ3pyf?=
 =?iso-8859-1?Q?B3TgCHzfBb54SB8a64KLa1N6JsqTyQuIMx+yAc2R4BahgE72BdbQv5KY2/?=
 =?iso-8859-1?Q?TWucoDm4E2DGkvbMihqE1Bhc7jx/h4V9f6N2IhOmlHF7WRvBLa453cu61i?=
 =?iso-8859-1?Q?EwCYn1BHhF2V4jEzolAh2+uUgLQsJr1e+eyodkeS4C6dPbMG2AeZF6D+tM?=
 =?iso-8859-1?Q?ohW1KSd/29qPpR11drUI+L3sjxHuJ5YJDg?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 825b21cf-105b-4da2-2eed-08dbb40865ff
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2023 03:20:36.9412
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +MEtcWt4lpDTODpPQdLm4M8pWyCS5kpmqRPPLrsUrE5Rh7RsGS7wxyQq8l/Zh5RLh9ysF6bs8HpNykT7KASLPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7639
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 12 Sep 2023 20:20:51 -0700 (PDT)
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email

On 2023-09-12 at 14:58:08 +0800, Aaron Lu wrote:
> When using sysbench to benchmark Postgres in a single docker instance
> with sysbench's nr_threads set to nr_cpu, it is observed there are times
> update_cfs_group() and update_load_avg() shows noticeable overhead on
> a 2sockets/112core/224cpu Intel Sapphire Rapids(SPR):
> 
>     13.75%    13.74%  [kernel.vmlinux]           [k] update_cfs_group
>     10.63%    10.04%  [kernel.vmlinux]           [k] update_load_avg
> 
> Annotate shows the cycles are mostly spent on accessing tg->load_avg
> with update_load_avg() being the write side and update_cfs_group() being
> the read side. tg->load_avg is per task group and when different tasks
> of the same taskgroup running on different CPUs frequently access
> tg->load_avg, it can be heavily contended.
> 
> E.g. when running postgres_sysbench on a 2sockets/112cores/224cpus Intel
> Sappire Rapids, during a 5s window, the wakeup number is 14millions and
> migration number is 11millions and with each migration, the task's load
> will transfer from src cfs_rq to target cfs_rq and each change involves
> an update to tg->load_avg. Since the workload can trigger as many wakeups
> and migrations, the access(both read and write) to tg->load_avg can be
> unbound. As a result, the two mentioned functions showed noticeable
> overhead. With netperf/nr_client=nr_cpu/UDP_RR, the problem is worse:
> during a 5s window, wakeup number is 21millions and migration number is
> 14millions; update_cfs_group() costs ~25% and update_load_avg() costs ~16%.
> 
> Reduce the overhead by limiting updates to tg->load_avg to at most once
> per ms. The update frequency is a tradeoff between tracking accuracy and
> overhead. 1ms is chosen because PELT window is roughly 1ms and it
> delivered good results for the tests that I've done. After this change,
> the cost of accessing tg->load_avg is greatly reduced and performance
> improved. Detailed test results below.
> 
> ==============================
> postgres_sysbench on SPR:
> 25%
> base:   42382?19.8%
> patch:  50174?9.5%  (noise)
> 
> 50%
> base:   67626?1.3%
> patch:  67365?3.1%  (noise)
> 
> 75%
> base:   100216?1.2%
> patch:  112470?0.1% +12.2%
> 
> 100%
> base:    93671?0.4%
> patch:  113563?0.2% +21.2%
> 
> ==============================
> hackbench on ICL:
> group=1
> base:    114912?5.2%
> patch:   117857?2.5%  (noise)
> 
> group=4
> base:    359902?1.6%
> patch:   361685?2.7%  (noise)
> 
> group=8
> base:    461070?0.8%
> patch:   491713?0.3% +6.6%
> 
> group=16
> base:    309032?5.0%
> patch:   378337?1.3% +22.4%
> 
> =============================
> hackbench on SPR:
> group=1
> base:    100768?2.9%
> patch:   103134?2.9%  (noise)
> 
> group=4
> base:    413830?12.5%
> patch:   378660?16.6% (noise)
> 
> group=8
> base:    436124?0.6%
> patch:   490787?3.2% +12.5%
> 
> group=16
> base:    457730?3.2%
> patch:   680452?1.3% +48.8%
> 
> ============================
> netperf/udp_rr on ICL
> 25%
> base:    114413?0.1%
> patch:   115111?0.0% +0.6%
> 
> 50%
> base:    86803?0.5%
> patch:   86611?0.0%  (noise)
> 
> 75%
> base:    35959?5.3%
> patch:   49801?0.6% +38.5%
> 
> 100%
> base:    61951?6.4%
> patch:   70224?0.8% +13.4%
> 
> ===========================
> netperf/udp_rr on SPR
> 25%
> base:   104954?1.3%
> patch:  107312?2.8%  (noise)
> 
> 50%
> base:    55394?4.6%
> patch:   54940?7.4%  (noise)
> 
> 75%
> base:    13779?3.1%
> patch:   36105?1.1% +162%
> 
> 100%
> base:     9703?3.7%
> patch:   28011?0.2% +189%
> 
> ==============================================
> netperf/tcp_stream on ICL (all in noise range)
> 25%
> base:    43092?0.1%
> patch:   42891?0.5%
> 
> 50%
> base:    19278?14.9%
> patch:   22369?7.2%
> 
> 75%
> base:    16822?3.0%
> patch:   17086?2.3%
> 
> 100%
> base:    18216?0.6%
> patch:   18078?2.9%
> 
> ===============================================
> netperf/tcp_stream on SPR (all in noise range)
> 25%
> base:    34491?0.3%
> patch:   34886?0.5%
> 
> 50%
> base:    19278?14.9%
> patch:   22369?7.2%
> 
> 75%
> base:    16822?3.0%
> patch:   17086?2.3%
> 
> 100%
> base:    18216?0.6%
> patch:   18078?2.9%
> 
> Reported-by: Nitin Tekchandani <nitin.tekchandani@intel.com>
> Suggested-by: Vincent Guittot <vincent.guittot@linaro.org>
> Signed-off-by: Aaron Lu <aaron.lu@intel.com>
> Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Tested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Reviewed-by: David Vernet <void@manifault.com>
> Tested-by: Swapnil Sapkal <Swapnil.Sapkal@amd.com>
> ---
>

Since we know that this patch brings good improvement for netperf,
hackbench, I did some further verification on tbench/schbench on Ice Lake
Xeon Platinum 8360Y, and it reports good result too:

schbench
========
case            	load    	baseline(std%)	compare%( std%)
normal          	1-mthreads	 1.00 (  1.70)	 +0.00 (  0.00)
normal          	2-mthreads	 1.00 (  2.32)	 -0.62 (  5.24)
normal          	4-mthreads	 1.00 (  3.17)	 -1.86 (  3.11)

tbench
======
case            	load    	baseline(std%)	compare%( std%)
loopback        	36-threads	 1.00 (  2.80)	 +1.85 (  0.45)
loopback        	72-threads	 1.00 (  0.27)	 -0.20 (  0.51)
loopback        	108-threads	 1.00 (  0.06)	+21.92 (  0.10)
loopback        	144-threads	 1.00 (  1.47)	+28.42 (  0.11)

Tested-by: Chen Yu <yu.c.chen@intel.com>

thanks,
Chenyu